An image semantic segmentation method based on total variation data augmentation with salt and pepper noise

By adding salt and pepper noise to the medical image dataset and using a full-variable regularization denoising model to expand the dataset, combined with neural network training, the problems of low image segmentation accuracy and insufficient samples are solved, and the tumor segmentation of PET/CT images are improved Robustness and accuracy.

CN114299004BActive Publication Date: 2024-11-15SUZHOU JUNYUEYUAN INTELLIGENT TECHNOLOGY CO LTD
View PDF 5 Cites 0 Cited by

Patent Information

Application Number
CN202111605820.X
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2021-12-25
Publication Date
2024-11-15
Estimated Expiration
2041-12-25

AI Technical Summary

Technical Problem

The existing image segmentation algorithms have problems such as low segmentation accuracy, inability to perform accurate semantic segmentation, and insufficient sample data in medical image processing, especially in the case of tumor segmentation in PET/CT images, which are difficult to effectively solve the problems of low contrast and noise.

Method used

Using a method of augmentation of full-variable data based on salt and pepper noise, a variety of denoising results are generated to expand the data set and combined with nerves The network is trained to improve the robustness of image segmentation.

Benefits of technology

The robustness of the image segmentation algorithm is significantly improved through data augmentation technology, and can generate more accurate tumor segmentation results under different noise conditions, enhance the performance of deep learning models, and is suitable for tumor segmentation tasks of PET/CT images.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114299004B_ABST
    Figure CN114299004B_ABST
Patent Text Reader

Abstract

The present invention relates to an image semantic segmentation method based on salt and pepper noise total variation data augmentation, wherein the method performs data augmentation based on a total variation regularization model, and applies a neural network technology to the augmented data set to realize a method for tumor segmentation in medical PET / CT images, wherein the method applies a neural network method to the augmented data set to realize tumor segmentation in medical PET / CT images. The present invention adds such image data as samples to the data set, minimizes the optimization problem in the total variation regularization denoising model, iteratively generates denoising results that meet different accuracy requirements, and further expands the data set, so that the obtained image segmentation results are more robust.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention belongs to the technical field of image processing and discloses an image semantic segmentation method based on salt and pepper noise total variation data augmentation. Background Art

[0002] In the field of medical radiology, positron emission tomography (PET) and computed tomography (CT) are often used to obtain tumor images. PET images display metabolic information of human tissues, with low resolution. The highlighted part of the image can roughly represent the tumor area in the image, with a fuzzy edge image. In contrast, CT images have high resolution and clear tumor edges, but they display structural information of the human body, and the contrast between tumor tissue and surrounding normal tissue is low. Accurate tumor segmentation from PET\CT images is of great significance in this discipline.

[0003] The total variation model has good performance in image denoising, image fusion, image segmentation and other fields, and has achieved certain theoretical and experimental results. Compared with deep learning methods, the image segmentation accuracy of this method is not high, and it cannot perform accurate semantic segmentation, but the model has the advantages of not requiring training samples and the research on the solution algorithm is relatively mature. When using medical instruments to collect image information, there is usually loss, resulting in image blur, image noise and other related problems.

[0004] Deep learning is effective in extracting image features. Shallow information can display local features of the image, while deep information involves a gradually expanding perceptual field and gradually abstracting the displayed global features. When using supervised deep learning algorithms, a large amount of sample data is usually required, and the actual segmentation results need to be manually annotated by doctors. However, medical images face the problem of relatively few samples and high costs for sampling and making data sets.

[0005] Existing data augmentation techniques expand existing data sets by adding noise to images, rotating them, etc. In practical applications, the present invention further expands the data set in a targeted manner, so that the obtained image segmentation results are more robust. Summary of the invention

[0006] In view of the above technical problems, the purpose of the present invention is to provide a medical image data augmentation method based on a total variation regularization model. The causes of salt and pepper noise may be sudden and strong interference to the image signal, analog-to-digital converter or bit transmission errors, etc. For example, a failed sensor causes the pixel value to be the minimum value, and a saturated sensor causes the pixel value to be the maximum value. In view of the fact that the existing acquired medical images may have been denoised, we also add this type of image data as samples to the data set, and iterate to generate denoising results that meet different accuracy requirements by minimizing the optimization problem in the total variation regularization denoising model, and further expand the data set to make the obtained image segmentation results more robust.

[0007] The technical solution provided by the present invention is:

[0008] A method for image semantic segmentation based on salt and pepper noise total variation data augmentation, the method performs data augmentation based on a total variation regularization model, and applies neural network technology to the augmented data set to achieve a method for tumor segmentation in medical PET / CT images, comprising the following steps:

[0009] Step 1: Get the sample image information in the data set. Assume that the pixel value at the pixel point (x, y) in the image Ω is I(x, y). Without loss of generality, the pixel value of the entire image is simply mapped as I, and the label image corresponding to the image is L;

[0010] Step 2: Add salt and pepper noise n with different signal-to-noise ratios to the image to obtain the noise image f, that is,

[0011] f = I + n;

[0012] Step 3: Use a total variation regularization image denoising model to remove noise and obtain a denoised result image;

[0013] Step 3.1: Construct the total variation regularized image denoising model applicable to specific noise:

[0014]

[0015] Among them, the symbol, min u It means that the function takes the minimum value of the desired denoised image pixel value u. It represents the distribution derivative of the image pixel value. Its components are the partial derivatives with respect to the horizontal x and the vertical y, respectively. It aims to control the pixel value u of the image to have the characteristic of taking a constant value in slices. ||·||1 represents the 1-norm. λ>0 is the regularization parameter in the model. The second term ||uf||1 aims to control u so that it does not deviate too much from the obtained f. Compared with the 2-norm, the 1-norm can better reflect the sparse characteristics of noise.

[0016] Step 3.2: Solve the total variation regularized image denoising model using a primal-dual algorithm;

[0017] The total variation regularized image denoising model is equivalently written in the following primal dual form:

[0018]

[0019] Where p is the dual variable, the dual space it belongs to is denoted by Y, and P is a convex set of the following form

[0020] P={p∈Y:||p|| ∞ <1}

[0021] And ||p|| ∞ represents the infinite norm, δ P is the indicator function of the set P. Thus, the following primal-dual iterative algorithm can be obtained:

[0022]

[0023] p k+1 =argmin p ∫ Ω <u k+1 , div p>+δ P (p)

[0024]

[0025] where div p represents the divergence of the dual variable p and θ is the super gradient parameter.

[0026] Step 3.3: Select different accuracy requirements, obtain the image denoising results under the requirements, and calculate the error at each step of the algorithm iteration Different error tolerances tol can be selected respectively (for example, tol=10 -2 , 10 -3 , 10 -4 , 10 -5 ), output the corresponding sample denoised image

[0027] Step 4: Compare the noise image f with the denoising result image The expanded image data set is obtained by corresponding to the labeled image L corresponding to the sample. Since the added salt and pepper noise has different signal-to-noise ratios, there are multiple noise images f, and the denoising results obtained under different error tolerances are also multiple. The labeled image they correspond to is still L, so the expansion from the unique corresponding relationship (I, L) is realized to form an augmented data set;

[0028] Step 5: Build a neural network model for training and prediction.

[0029] Step 5.1: Build a suitable, end-to-end fully convolutional neural network model, whose macro architecture is a V-net model network, which includes the number of convolutional network layers, convolution kernel size, step size, number of training rounds, optimizer and learning rate;

[0030] Step 5.2: Bring the augmented data set as input into the V-net model network for training.

[0031] Step 5.3: After the network training is completed, the new medical PET / CT images that need to be segmented and processed are respectively substituted into the V-net model network to obtain the final segmentation results.

[0032] Beneficial effects of the present invention:

[0033] Compared with the existing image data augmentation methods, such as simple noise addition, rotation, enlargement, reduction and other technologies, the technical advantage of the present invention is that it can further improve the robustness of the deep learning algorithm: according to the characteristics that the existing acquired medical images may have been denoised, according to the sparse characteristics of salt and pepper noise images, a total variation denoised image is designed, and this type of image data is also added to the data set as a sample, thereby achieving further expansion of the data sample set. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 This is a model structure diagram for data augmentation in the present invention;

[0035] Figure 2 An example diagram of the data augmentation process of the present invention;

[0036] Figure 3 It is a model structure diagram of the present invention;

[0037] Figure 4 Salt and pepper noise graphs corresponding to different signal-to-noise ratios. DETAILED DESCRIPTION

[0038] See also Figure 1-Figure 4 As shown, the present invention will be further described in detail below with reference to the accompanying drawings in conjunction with specific examples.

[0039] The structural model of the method of the present invention is as follows Figure 3 It is shown that the invention is further described by way of examples, but is not intended to limit the scope of the invention in any way.

[0040] step:

[0041] Step 1: Get the sample image information in the data set, and set the pixel value at the pixel point (x, y) in the image Ω to be I(x, y). Without loss of generality, the pixel value of the entire image is simply recorded as mapping I, and the label image corresponding to the image is set to L;

[0042] Step 2: Add salt and pepper noise n with different signal-to-noise ratios to the image to obtain the noise image f, that is,

[0043] f = I + n;

[0044] Step 3: Use the total variation regularization image denoising model to remove noise and obtain the denoised result image;

[0045] Step 3.1: Construct a total variation image denoising model suitable for specific noise;

[0046]

[0047] The symbol min u It means that the function takes the minimum value of the desired denoised image pixel value u. It represents the distribution derivative of the image pixel value. Its components are the partial derivatives about the horizontal and vertical directions respectively. It aims to control the pixel value u of the image to have the characteristic of taking constant values ​​in slices. ||·||1 represents the 1-norm. λ>0 is the regularization parameter in the model. The second term ||uf||1 aims to control u so that it does not deviate too much from the obtained f. Compared with the 2-norm, the 1-norm can better reflect the sparse characteristics of noise.

[0048] Step 3.2: Solve the image denoising model using the primal-dual algorithm;

[0049] The above minimization model can be equivalently written in the following primal dual form:

[0050]

[0051] Where p is the dual variable, the dual space it belongs to is denoted by Y, and P is a convex set of the following form

[0052] P={p∈Y:||p|| ∞ <1}

[0053] And ||p|| ∞ represents the infinite norm, δ P is the indicator function of the set P. Thus, the following primal-dual iterative algorithm can be obtained:

[0054]

[0055] p k+1 =argminp ∫ Ω ≤u k+1 , div p>+δ P (p)

[0056]

[0057] where div p represents the divergence of the dual variable p and θ is the super gradient parameter.

[0058] Step 3.3: Select different accuracy requirements and obtain the image denoising results under the requirements; calculate the error at each step of the algorithm iteration Different error tolerances tol can be selected respectively (for example, tol=10 -2 , 10 -3 , 10 -4 , 10 -5 ), output the corresponding sample denoised image

[0059] Step 4: Compare the noise image f with the denoising result image The expanded image dataset is obtained by corresponding to the labeled image L corresponding to the sample. Since the added salt and pepper noise has different signal-to-noise ratios, there are multiple noise images, and the denoising results obtained under different error tolerances are also multiple. The labeled image they correspond to is still L, so the expansion from the unique corresponding relationship (I, L) is achieved, that is, data augmentation.

[0060] Step 5: Build a neural network model for training and prediction.

[0061] Step 5.1: Build a suitable, end-to-end fully convolutional neural network model, whose macro architecture is a V-net model network, which includes the number of convolutional network layers, convolution kernel size, step size, number of training rounds, optimizer and learning rate;

[0062] Step 5.2: Bring the augmented data set as input into the V-net model network for training.

[0063] Step 5.3: After the network training is completed, the new medical PET / CT images that need to be segmented and processed are respectively substituted into the V-net model network to obtain the final segmentation results.

[0064] In addition to processing medical PET / CT images, the present invention is also applicable to other images.

Claims

1. An image semantic segmentation method based on total variation data augmentation with salt and pepper noise, Its characteristics are: The method performs data augmentation based on a total variation regularization model and applies neural network technology to the augmented data set to achieve a method for tumor segmentation in medical PET / CT images, comprising the following steps: Step 1: Get the sample image information in the data set, and set the pixel value at the pixel point (x, y) in the image Ω to be I(x, y). Without loss of generality, the pixel value of the entire image is simply recorded as mapping I, and the label image corresponding to the image is L; Step 2: Add salt and pepper noise n with different signal-to-noise ratios to the image to obtain the noise image f, that is, f = I + n; Step 3: Use a total variation regularization image denoising model to remove noise and obtain a denoised result image; Step 3.1: Construct the total variation regularized image denoising model applicable to specific noise: Among them, the symbol, min u It means that the function takes the minimum value of the desired denoised image pixel value u. Represents the distribution derivative of the image pixel value, whose components are the partial derivatives about the horizontal x and vertical y, respectively, which aims to control the pixel value u of the image to have the characteristic of taking a constant value in slices, ||·|| 1 represents the 1-norm, λ>0 is the regularization parameter in the model, and the second term ||uf|| 1 Aims to control u so that it does not deviate too much from the obtained f. Compared with the 2-norm, the 1-norm can better reflect the sparse characteristics of noise; Step 3.2: Solve the total variation regularized image denoising model using a primal-dual algorithm; The total variation regularized image denoising model is equivalently written in the following primal dual form: Where p is the dual variable, the dual space it belongs to is denoted by Y, and P is a convex set of the following form P={p∈Y:||p|| ∞ <1} And ||p|| ∞ represents the infinite norm, δ P is the indicator function of the set P; thus, the following primal-dual iterative algorithm can be obtained p k+1 =argmin p ∫ Ω <u k+1 ,divp>+δ P (p) where divp represents the divergence of the dual variable p, θ is the super gradient parameter, Step 3.3: Select different accuracy requirements, obtain the image denoising results under the requirements, and calculate the error at each step of the algorithm iteration Take different degrees of error tolerance tol, and take tol=10 -2 , 10 -3 , 10 -4 , 10 -5 , output the corresponding sample denoised image Step 4: Compare the noise image f with the denoising result image They are respectively matched with the labeled image L corresponding to the sample to obtain the expanded image data set. Since the added salt and pepper noise has different signal-to-noise ratios, there are multiple noise images f, and there are also multiple denoising results obtained under different error tolerances. The labeled image they correspond to is still L, so the expansion from the unique corresponding relationship (I, L) is achieved to form an augmented data set.

2. The image semantic segmentation method based on salt and pepper noise total variation data augmentation according to claim 1, Its characteristics are: After obtaining the augmented data set, a neural network model is built for training and prediction. The steps are as follows: Step 5.1: Build a suitable, end-to-end fully convolutional neural network model, whose macro architecture is a V-net model network, which includes the number of convolutional network layers, convolution kernel size, step size, number of training rounds, optimizer and learning rate; Step 5.2: Bring the augmented data set as input into the V-net model network for training; Step 5.3: After the network training is completed, the new medical PET / CT images that need to be segmented and processed are respectively substituted into the V-net model network to obtain the final segmentation results.

Citation Information

Patent Citations

  • TV (total variation) image noise removal method based on noise priori constraint

    CN106355561A

  • An image mixed noise elimination method based on a noise learning neural network model

    CN109685743A

  • Infrared image deblurring method under salt and pepper noise

    CN109767404A

  • High-noise image denoising method based on deep convolutional network

    CN110648292A

  • Ancient book Chinese character image denoising method based on progressive generative adversarial

    CN113792743A