A deep neural network image segmentation method based on smoothness constraint
By using a loss function based on distance penalty and smoothness constraint, the problems of non-smoothness and missegmentation in image segmentation are solved, achieving higher segmentation accuracy and stability, and improving the performance of the image segmentation model.
Patent Information
- Application Number
- CN202310910962.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-07-24
- Publication Date
- 2025-11-18
- Estimated Expiration
- 2043-07-24
AI Technical Summary
Existing image segmentation loss functions tend to result in unsmooth segmentation and missegmented small regions when dealing with imbalanced datasets. Furthermore, existing loss functions are difficult to tune and it is hard to measure the accuracy of the segmentation results from a global perspective.
We employ a content loss function based on distance penalty, an unsupervised length loss function, and an area compensation loss function. By converting segmentation labels into a signed distance function and combining it with the LeakyReLU activation function, we define a smoothness-constrained loss function to ensure the smoothness and accuracy of the segmentation results.
It effectively avoids missegmented small regions, improves the smoothness and accuracy of segmentation results, and enhances the performance of evaluation indicators such as Dice coefficient, HD, JSC and APD.
Smart Images

Figure CN117036695B_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of deep neural network technology, and specifically relates to a deep neural network image segmentation method based on smoothness constraints. Background Technology
[0002] Image segmentation is the technique and process of dividing an image into several specific regions with unique properties and extracting targets of interest. With the rapid development of deep learning in recent years, deep neural networks (DNNs) have made significant progress in image analysis due to their powerful feature extraction capabilities. Their construction process mainly includes two parts: model structure design and model parameter training.
[0003] Loss functions are a distance metric used to evaluate the model's predicted values and the true labels. They play an important role in improving the accuracy and stability of model segmentation. Image segmentation loss functions can be broadly classified into four categories: distribution-based loss, region-based loss, boundary-based loss, and composite loss.
[0004] The most commonly used distribution-based loss function is the Binary Cross-Entropy loss function. In classification tasks, a Sigmoid or Softmax activation function combined with the Binary Cross-Entropy loss function is frequently used. When the number of foreground pixels is much smaller than the number of background pixels (i.e., the number of y=0 pixels is much greater than the number of y=1 pixels), the y=0 component in the loss function will dominate, causing a dataset skew. This ultimately leads to the model's predictions being severely biased towards the background, resulting in poor performance in practical applications. Additionally, there is the Weighted Binary Cross-Entropy loss function, which adds positive sample weights to the Binary Cross-Entropy loss function. It can achieve better results in cases of imbalanced sample sizes, but requires manual adjustment of the weights of difficult samples, making parameter tuning more challenging.
[0005] A common region-based loss function is the Dice loss function (DiceLoss), which is adapted from the Dice coefficients. However, the Dice loss function is particularly unfavorable for small targets. When only foreground and background are considered, even a small target with some mispredicted pixels will cause significant changes in Dice, leading to drastic gradient shifts and unstable training. Furthermore, the Dice loss function weights FP and FN equally, which in practice results in high segmentation accuracy but low recall.
[0006] The third type of loss function is boundary-based loss functions, such as Hausdorff distance loss. However, due to its non-convexity, it is not widely used. The last type of loss function is based on composite loss functions, such as exponential logarithmic loss. It forces the network to focus on inaccurate predictions, thus incorporating finer segmentation boundaries and more accurate data distributions. However, the added parameter weights bring considerable challenges to parameter tuning.
[0007] While existing loss functions have made significant progress, they are still point-to-point comparison loss metrics, which compare each pixel in the segmented result with its corresponding label. This approach makes it difficult to measure the accuracy of the segmentation result from a global perspective, resulting in unsmooth segmentation results and the appearance of missegmented small regions. Summary of the Invention
[0008] The purpose of this invention is to address the shortcomings of existing multi-level inverters by proposing a deep neural network image segmentation method based on smoothness constraints. The specific scheme is as follows:
[0009] The first aspect of this invention provides a smoothness loss function based on curve evolution for a neural network segmentation model, comprising:
[0010] When used for image segmentation, a distance-penalized content loss function L is used to measure the loss of pixels in the segmented image. content ;
[0011] When used for image prediction, the unsupervised length loss function L predicts the boundary length of the mask. length ;as well as,
[0012] Used in image segmentation to compensate for the length loss function L length The area compensation loss function L for over-segmentation prediction caused by over-constraint. area ;
[0013] The smoothness loss function convLetLoss based on curve evolution is defined as:
[0014]
[0015] Where α is the length loss function L length The weights, β is the area compensation loss function L area The weights are given by w, where w is the image width.
[0016] A second aspect of the present invention provides a deep neural network segmentation model based on smoothness constraints, wherein the smoothness loss function based on curve evolution is used as the loss function of the neural network segmentation model.
[0017] A third aspect of the present invention provides a deep neural network segmentation device based on smoothness constraints:
[0018] Memory; and
[0019] A processor coupled to the memory is configured to implement the deep neural network segmentation model based on smoothness constraints when executing instructions stored in the memory.
[0020] This invention has outstanding substantive features and significant progress compared to the prior art, specifically:
[0021] This invention first converts segmentation labels into a signed distance function and proposes a content loss metric based on distance penalty. For mis-segmented pixels, the loss is positively correlated with the distance to the label boundary, effectively avoiding mis-segmented small regions. Then, an unsupervised length loss metric is proposed, which minimizes the boundary length of the prediction mask based on the level set method, thus ensuring the smoothness of the segmentation result. Finally, to address the issue of mis-segmentation caused by the potential over-constraint of the length loss function leading to the disappearance of the segmentation target in the prediction image, an area compensation loss metric is proposed to provide an outward "compensation" loss during image segmentation, thereby solving the problems of unsmooth segmentation results and mis-segmented small regions associated with existing loss functions. Attached Figure Description
[0022] Figure 1 This is an image of the real contour label in Embodiment 1 of the present invention after distance transformation. Detailed Implementation
[0023] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention are described clearly and completely below, but the present invention is not limited to these embodiments:
[0024] Example 1
[0025] A method for calculating smoothness loss based on curve evolution for neural network segmentation models includes:
[0026] Content loss measurement
[0027] In image segmentation, a content loss function L based on distance penalty is used. content Perform a loss measurement on the pixels of the segmented image.
[0028] Neural network segmentation models belong to the category of supervised training when performing segmentation tasks. Therefore, before the neural network segmentation model performs image segmentation tasks, the true contour labels of the image need to be initialized to SDF form. This embodiment uses a distance transformation function to perform this initialization process. In the original contour labels, the pixel values of the background are generally set to 0, and the pixel values of the segmentation target region are set to 1. The distance transformation function is used to calculate the distance of each pixel to the nearest 0-value pixel. The loss of each pixel is positively correlated with the distance from the boundary of the contour label. The input is a grayscale image. Figure 1 As shown, after distance transformation, the brighter the point on the image, the farther away it is from the zero point.
[0029] To enable the neural network segmentation model to output a form similar to SDF, the activation function of the last output layer needs to be set to the LeakyReLU function. Unlike the commonly used Sigmoid and ReLU functions, the LeakyReLU function is a widely used variant of the ReLU activation function, and its output has a small slope for negative inputs. Since the derivative is always non-zero, this reduces the occurrence of silent neurons, allows gradient-based learning, and solves the problem of neurons not learning after the ReLU function enters the negative region. LeakyReLU preserves the values on the negative axis, ensuring that information on the negative axis is not completely lost.
[0030] The LeakyReLU activation function is defined as follows:
[0031]
[0032] Content loss function L based on distance penalty content Defined as:
[0033]
[0034] Let Ω∈R be the image region. Ω→R represents the segmentation result output by the neural network segmentation model. Ω→R is the distance map converted from the manually drawn result.
[0035] Length loss metric
[0036] In image prediction, an unsupervised length loss function L is used. length Predict the boundary length of the mask.
[0037] Training neural network models using content loss functions has shortcomings, such as the inability to guarantee the smoothness of segmentation edges. To address this, this embodiment defines the following length loss metric:
[0038]
[0039] P(x, y) = 0 represents the target boundary predicted by the neural network, L length Integrating along the target boundary is equivalent to calculating the number of boundary points, i.e., the boundary length. During backpropagation, minimizing L... length This means minimizing the curve segment that encloses the same area, i.e., making the curve as smooth as possible, thus ensuring the smoothness of the prediction boundary.
[0040] In the area compensation loss function L area In this model, the integration region is the set of boundary points P(x, y) = 0. Since the ideal value of P can only be positive or negative, and since P is defined in discrete space, it does not have a value of 0. Therefore, the output P of the neural network segmentation model needs to be transformed using the following formula:
[0041]
[0042] Where H(x) is the Heaviside function, which is defined as follows:
[0043]
[0044] After the Heaviside function transformation, The value is 0 in the target region and 1 in the background region. Therefore, the set of boundary points P(x,y)=0 can be represented as The set of points with non-zero gradients within an image region, i.e., the set of boundary points, can be defined as: ,in, Points where the gradient magnitude is greater than 0, and points where the gradient is non-zero;
[0045] Therefore, the length loss function L length The following formula can be used instead:
[0046]
[0047] It should be noted that the Heaviside function has an infinite derivative region at x=0, which may cause the gradient to fail to update during the training of the neural network segmentation model. Therefore, this embodiment uses the following function to approximate the Heaviside function:
[0048]
[0049] is a coefficient used to scale the weight of x. In the comparison experiment between cross-entropy and this function, It was set to 1 in the ablation experiment. The weight was set to 0.25, which has a certain impact on the experiment;
[0050] During backpropagation, the following function is used to replace the derivative of the Heaviside function, namely the Dirac function:
[0051] .
[0052] Area compensation loss measurement
[0053] To address the issues of potential over-constraint in length loss metrics during image segmentation, leading to over-segmentation and the disappearance of segmented targets in the predicted image, and the potential over-segmentation of blurred-boundary targets, an area-compensating loss function L is defined. area This problem is solved by providing an outward "compensation" for the loss during the segmentation process.
[0054] Among them, the area compensation loss function L area Defined as:
[0055]
[0056] Where w is the image width.
[0057] Smoothness loss function based on curve evolution
[0058] The smoothness loss function ConvLetLoss, based on curve evolution, is used for calculation and is defined as follows:
[0059]
[0060] Where α is the length loss function L length The weights are used to improve performance on datasets with different distributions; β is the area-compensation loss function L. area The weights are negative. To ensure consistency in the value ranges of the content loss function and the length loss function, the length loss function is weighted by a factor of 4 times the image width in the curve evolution-based smoothness loss function ConvLetLoss.
[0061] Other loss functions, such as cross-entropy and Dice Loss, can be used to experiment on the ACDC2017 left ventricular and epicardial data. However, experiments on small regions prone to misclassification and regions with rough boundaries show that the proposed method performs better overall in terms of Dice coefficient, HD, JSC, and APD evaluation metrics. Furthermore, a length loss metric L can be added to the original loss function. length and area compensation loss metric L area The results show that the method proposed in this patent performs better than the original loss function method in terms of evaluation metrics, proving the effectiveness of the L proposed in this embodiment. length and L area This is an effective part for improving segmentation accuracy.
[0062] Example 2
[0063] This embodiment provides a deep neural network segmentation model based on smoothness constraints, using the smoothness loss calculation method based on curve evolution described in Embodiment 1 as the loss function of the neural network segmentation model.
[0064] Example 3
[0065] This embodiment provides a deep neural network segmentation device based on smoothness constraints, including...
[0066] Memory; and
[0067] A processor coupled to the memory is configured to, when executing instructions stored in the memory, implement the deep neural network segmentation model based on smoothness constraints as described in Embodiment 2.
[0068] The memory may include, for example, system memory, fixed non-volatile storage media, etc. System memory may store, for example, the operating system, application programs, boot loader, and other programs.
[0069] The device may also include input / output interfaces, network interfaces, and storage interfaces. These interfaces, as well as the memory and processor, can be connected via, for example, a bus. The input / output interfaces provide connection interfaces for input / output devices such as monitors, mice, keyboards, and touchscreens. The network interfaces provide connection interfaces for various networked devices. The storage interfaces provide connection interfaces for external storage devices such as SD cards and USB flash drives.
[0070] Those skilled in the art will understand that embodiments of this disclosure can be provided as methods, systems, or computer program products. Therefore, this disclosure can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this disclosure can take the form of a computer program product embodied on one or more computer-non-transitory readable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer program code.
[0071] This disclosure is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this disclosure. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create a machine for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.
[0072] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.
[0073] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.
Claims
1. A method for calculating smoothness loss based on curve evolution for neural network segmentation models, characterized in that, include: In image segmentation, a content loss function L based on distance penalty is used. content Measure the loss of pixels in the segmented image; The distance-penalized content loss function L content Defined as: , Let Ω∈R be the image region. Ω→R represents the segmentation result output by the neural network segmentation model. Ω→R is the distance map transformed from the manually drawn result; w is the image width; In image prediction, an unsupervised length loss function L is used. length Predict the boundary length of the mask; and, In the image segmentation process, the area-compensated loss function L is used. area Compensation for length loss function L length Over-constraints leading to over-segmentation and over-prediction; The length loss function L length Defined as: , ▽ represents the gradient. for The result after the Heaviside function transformation is: , Where H(x) is the Heaviside function, which is defined as follows: ; Area compensation loss function L area Defined as: , The smoothness loss function convLetLoss, based on curve evolution, is used for calculation and is defined as follows: , Where α is the length loss function L length The weights, β is the area compensation loss function L area The weights are given by w, where w is the image width.
2. The smoothness loss calculation method based on curve evolution according to claim 1, characterized in that, The smoothness loss function based on curve evolution also includes: Before performing image segmentation tasks, neural network segmentation models initialize the true contour labels of images in SDF form.
3. The smoothness loss calculation method based on curve evolution according to claim 2, characterized in that, The initialization process is implemented using a distance change function: In the original outline label, set the background pixel value to 0 and the target region pixel value to 1; Calculate the distance of each pixel in the contour label to the nearest pixel with a value of 0; The magnitude of pixel loss is positively correlated with the distance from the outline label boundary.
4. The smoothness loss calculation method based on curve evolution according to claim 2, characterized in that, Set the activation function of the last output layer of the neural network segmentation model to the LeakyReLU function.
5. The smoothness loss calculation method based on curve evolution according to claim 1, characterized in that: During the training of the neural network segmentation model, the following function is used to approximate the Heaviside function: , is a coefficient used to scale the weight of x; During backpropagation, the Dirac function is used. To replace the derivative of the Heaviside function, where: 。 6. A method for constructing a deep neural network segmentation model based on smoothness constraints, characterized in that: The smoothness loss calculation method based on curve evolution as described in any one of claims 1-5 is used as the loss function of the neural network segmentation model.
7. A deep neural network segmentation device based on smoothness constraints, characterized in that: Memory; as well as A processor coupled to the memory, the processor being configured to, upon execution of instructions stored in the memory, implement the deep neural network segmentation model based on smoothness constraints as described in claim 6.
Citation Information
Patent Citations
priori shape constraint-based BCA-UNet liver segmentation method
CN112561860A
Gradient normalization systems and methods for adaptive loss balancing in deep multitask networks
US20190130275A1
