A mask generation method based on total variation denoising

Abstract

The present invention discloses a mask generation method based on total variation denoising, belonging to the field of image processing. The method comprises using a web crawler to crawl an original image and modify the original image to obtain a tampered image to obtain original data; discarding the tampered image with a different pixel size from the original image and subtracting the original image from the corresponding tampered image to obtain a subtraction image dataset; using the maximum stable extreme value region method and non-maximum suppression method based on the subtraction image dataset to obtain text information of several subtraction images; performing total variation denoising on the subtraction images in the subtraction image dataset to obtain a processed image dataset; and superimposing the processed image in the processed image dataset and the text information of the corresponding subtraction image to obtain a mask. The present invention solves the problems of low mask acquisition efficiency and insufficient quality.

Description

Technical Field

[0001] The present invention belongs to the field of image processing, and in particular relates to a mask generation method based on total variation denoising. Background Art

[0002] Whether in culture, economy, or society, tools that can easily identify these manipulated images are urgently needed. Numerous deep learning models are currently working to address this issue, but they also face significant challenges. The effectiveness of a deep learning model's recognition performance depends largely on the quality of the dataset used during training. Existing datasets all have challenges: the Defacto dataset is artificially synthesized and may not fully reflect real-world manipulation. The manipulated images collected in the CASIA v1.0 dataset are edited using a limited number of manipulation methods and may not represent the full range of manipulated images. The NIST 2016 dataset is more comprehensive, but it is proprietary to the National Institute of Standards and Technology. The COVERAGE dataset has a small sample size, preventing deep learning models from achieving deep learning. The CASIA v2.0 dataset suffers from issues such as imbalanced and limited subject distribution, image quality and metadata, and potential overfitting. These issues can prevent models trained on this dataset from generalizing well.

[0003] One existing technical solution is data labeling, which is crucial for the effective operation of most artificial intelligence algorithms. Artificial intelligence algorithms are data-driven. This means that to achieve artificial intelligence, we must first teach computers the human ability to understand and judge things, allowing them to learn this recognition capability.

[0004] The process of data labeling involves manually labeling data to provide machine systems with learning samples. Data labeling involves labeling the data that needs to be recognized and distinguished by the machine, allowing the computer to continuously learn the characteristics of this data, ultimately enabling the computer to independently recognize it.

[0005] Image annotation quality standards: The quality of image annotation depends on the accuracy of pixel identification. The closer the annotated pixel is to the edge pixels of the annotated object, the higher the annotation quality and the greater the difficulty. If the image annotation requires 100% accuracy, the error between the annotated pixel and the edge pixels of the annotated object should be within 1 pixel.

[0006] The annotation process of image data is as follows:

[0007] (1) Data cleaning: Eliminate data quality issues such as missing values, noisy data, and duplicate data.

[0008] (2) Data labeling: divide labeling tasks, formulate labeling standards, and carry out labeling tasks.

[0009] (3) Labeling inspection: Labeling quality is inspected by a labeling auditor or a machine quality inspection mechanism. Summary of the Invention

[0010] In view of the above-mentioned deficiencies in the prior art, the present invention provides a mask generation method based on total variation denoising to solve the problems of low efficiency and insufficient quality of mask acquisition.

[0011] In order to achieve the above-mentioned object of the invention, the technical solution adopted by the present invention is: a mask generation method based on total variation denoising, comprising the following steps:

[0012] S1. Using a web crawler to crawl the original image and modify the original image to obtain the tampered image, to obtain the original data;

[0013] S2, discarding the tampered image with a different pixel size from the original image, and subtracting the original image from the corresponding tampered image to obtain a subtraction image dataset;

[0014] S3. Based on the subtraction image data set, using the maximum stable extreme value region method and the non-maximum suppression method, obtain text information of several subtraction images;

[0015] S4, performing total variation denoising processing on the subtraction images in the subtraction image dataset to obtain a processed image dataset;

[0016] S5. Superimpose the text information of the processed image and the corresponding subtraction image in the processed image data set to obtain a mask.

[0017] The beneficial effects of the present invention are as follows: the present invention can be automatically run on a processor, thereby reducing labor costs and improving mask acquisition efficiency; by utilizing the maximum stable extreme value region method and the non-maximum suppression method, the text information of the image can be acquired, the image background and the image text information can be separated, the loss of the image text information can be avoided, and the quality of the mask can be improved; and by total variation denoising, the quality of the mask can be improved and the image noise can be reduced; by using a web crawler to crawl the original image and modifying the original image to obtain the tampered image, the sample diversity of the tampered image is increased.

[0018] Furthermore, each original image in the original data in step S1 has several corresponding tampered images.

[0019] The beneficial effect of the above further solution is that the acquired tampered images of various types are matched one-to-one with their original images, which facilitates subsequent image processing.

[0020] Furthermore, the expression of the subtraction image data set in step S2 is:

[0021] U=[A 0,xp ,A 1,xp ,…,A i,xp ,…,A n,xp ]

[0022] A i,xp =A i,0 -B i,r ; i=1,...,n

[0023] Among them, U is the subtraction image dataset; A i,xp is the subtraction image set corresponding to the i-th original image; A n,xp is the subtraction image set corresponding to the nth original image; A i,0 is the i-th original image; B i,r is the rth tampered image corresponding to the i-th original image; i is the original image number; n is the total number of original images; r is the tampered image number.

[0024] The beneficial effects of the above further solution are: discarding unusable images to improve image processing efficiency; at the same time, sequentially numbering each image to facilitate subsequent image processing.

[0025] Furthermore, the step S3 is specifically as follows:

[0026] S301, obtaining a grayscale image of each subtraction image in a subtraction image dataset;

[0027] S302, performing binarization processing on the grayscale images of each subtraction image to obtain a binary image of each grayscale threshold;

[0028] S303, respectively calculate the change rate of the connected area in the binary image of each gray threshold:

[0029]

[0030] Where v(t) is the rate of change of the connected area in the binary image with grayscale threshold t; Q t+Δ is the connected area in the binary graph with grayscale threshold t+Δ; Q t-Δ is the connected area in the binary graph with grayscale threshold t-Δ; Q t is the connected area in the binary graph with grayscale threshold t; Δ is the small increment of grayscale threshold; |·| is the absolute value function;

[0031] S304, obtaining the area where the change rate of the connected area in the binary image of each gray threshold is the local minimum, and obtaining the maximum stable extreme value area set of each binary image;

[0032] S305, setting a comparison threshold;

[0033] S306 , performing non-maximum suppression on the maximum stable extreme value region sets of each binary image according to the comparison threshold to remove overlap, thereby obtaining text information of a plurality of subtracted images.

[0034] The beneficial effects of the above further scheme are: extracting text information separately to avoid being identified as noise and processed in total variation denoising; using the maximum stable extreme value region method, the area of ​​text information can be identified at a faster speed, saving computing time.

[0035] Furthermore, the step S306 is specifically as follows:

[0036] S3061. Calculate the confidence of the target selection box in the maximum stable extreme value region set of the current binary image;

[0037] S3062. Obtain the target selection box with the highest confidence in the maximum stable extreme value region set of the current binary image as the comparison selection box;

[0038] S3063. Obtain a set of remaining selection boxes based on the comparison selection boxes.

[0039] S3064. Calculate the intersection-over-union ratio of each selection box in the remaining selection box set and the comparison selection box;

[0040] S3065: Based on the intersection-and-union ratios of each selection box in the remaining selection box set and the comparison selection box, remove the selection boxes whose intersection-and-union ratios are greater than the comparison threshold from the remaining selection box set to obtain a first selection box set;

[0041] S3066: Determine whether there is only one selection box in the first selection box set. If so, obtain the text information of the subtraction image corresponding to the current binary image and proceed to step S3067. Otherwise, update the maximum stable extreme value region set of the current binary image to the first selection box set and return to step S3062.

[0042] S3067. Determine whether the current binary image is the last binary image. If so, integrate the text information of the subtraction images corresponding to each binary image to obtain text information of several subtraction images, and proceed to step S4. Otherwise, update the current binary image to the next binary image, and return to step S3061.

[0043] The beneficial effect of the above further solution is: when using the maximum stable extreme value region method, many overlapping regions will appear. Using the non-maximum suppression method to select the best region in the overlapping regions can better lock the text information.

[0044] Furthermore, the step S4 is specifically as follows:

[0045] S401, obtaining a numerical approximate solution of an anisotropic diffusion equation for each pixel point according to a current subtraction image in the subtraction image data set;

[0046] S402, obtaining a gradient based on a numerical approximate solution of an anisotropic diffusion equation for each pixel;

[0047] S403. According to the gradient, a diffusion coefficient whose intensity and direction are related to the gradient is obtained;

[0048] S404. Based on the diffusion coefficient whose intensity and direction are related to the gradient, Taylor approximate diffusion is performed using the original image corresponding to the current subtraction image to obtain a first image:

[0049]

[0050] Where minTV(f) is the first image; f is the original image; λ is a constant; f0 is the image after denoising; is the square of the partial derivative of the pixel in the x direction; is the square of the partial derivative of the pixel in the y direction; dx is the differential of x; dy is the differential of y; Ω is the integration area;

[0051] S405, performing binarization processing on the first image to obtain a second image;

[0052] S406, performing erosion and dilation operations on the second image to obtain a processed image corresponding to the current subtracted image;

[0053] S407. Determine whether the current subtraction image is the last subtraction image in the subtraction image dataset. If so, obtain the processed image dataset based on the processed images corresponding to each subtraction image. Otherwise, update the current subtraction image to the next subtraction image, and return to step S401.

[0054] The beneficial effects of the above further solution are: a large amount of noise in the subtraction image is removed through total variation denoising; and corrosion and dilation operations are performed simultaneously to make the subtraction image close to the ideal effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] Figure 1 Flow chart of the method of the present invention.

[0056] Figure 2 Schematic diagram of the method of the present invention. DETAILED DESCRIPTION

[0057] The specific embodiments of the present invention are described below to facilitate understanding of the present invention by those skilled in the art. However, it should be clear that the present invention is not limited to the scope of the specific embodiments. For those skilled in the art, as long as various changes are within the spirit and scope of the present invention as defined and determined by the appended claims, these changes are obvious, and all inventions and creations utilizing the concepts of the present invention are protected.

[0058] like Figure 1 and Figure 2 As shown, in one embodiment of the present invention, a mask generation method based on total variation denoising includes the following steps:

[0059] S1. Using a web crawler to crawl the original image and modify the original image to obtain the tampered image, to obtain the original data;

[0060] S2, discarding the tampered image with a different pixel size from the original image, and subtracting the original image from the corresponding tampered image to obtain a subtraction image dataset;

[0061] S3. Based on the subtraction image data set, using the maximum stable extreme value region method and the non-maximum suppression method, obtain text information of several subtraction images;

[0062] S4, performing total variation denoising processing on the subtraction images in the subtraction image dataset to obtain a processed image dataset;

[0063] S5. Superimpose the text information of the processed image and the corresponding subtraction image in the processed image data set to obtain a mask.

[0064] Each original image in the original data in step S1 has several corresponding tampered images.

[0065] The expression of the subtraction image data set in step S2 is:

[0066] U=[A 0,xp ,A 1,xp ,…,A i,xp ,…,A n,xp ]

[0067] A i,xp =A i,0 -B i,r ; i=1,...,n

[0068] Among them, U is the subtraction image dataset; A i,xp is the subtraction image set corresponding to the i-th original image; A n,xp is the subtraction image set corresponding to the nth original image; A i,0 is the i-th original image; B i,ris the rth tampered image corresponding to the i-th original image; i is the original image number; n is the total number of original images; r is the tampered image number.

[0069] In this embodiment, the subtraction of two images: the subtraction of the original image and the tampered image is simply to perform an AB operation on two images with the same pixel size, and discard the image with a different pixel size from the original image.

[0070] The step S3 is specifically as follows:

[0071] S301, obtaining a grayscale image of each subtraction image in a subtraction image dataset;

[0072] S302, performing binarization processing on the grayscale images of each subtraction image to obtain a binary image of each grayscale threshold;

[0073] S303, respectively calculate the change rate of the connected area in the binary image of each gray threshold:

[0074]

[0075] Where v(t) is the rate of change of the connected area in the binary image with grayscale threshold t; Q t+Δ is the connected area in the binary graph with grayscale threshold t+Δ; Q t-Δ is the connected area in the binary graph with grayscale threshold t-Δ; Q t is the connected area in the binary graph with grayscale threshold t; Δ is the small increment of grayscale threshold; |·| is the absolute value function;

[0076] S304, obtaining the area where the change rate of the connected area in the binary image of each gray threshold is the local minimum, and obtaining the maximum stable extreme value area set of each binary image;

[0077] S305, setting a comparison threshold;

[0078] S306 , performing non-maximum suppression on the maximum stable extreme value region sets of each binary image according to the comparison threshold to remove overlap, thereby obtaining text information of a plurality of subtracted images.

[0079] In this embodiment, the maximum stable extreme region method is first used in text extraction, assuming all three possible holdings of a grayscale image i. Pixels below the threshold are called "black", and pixels above or equal to the threshold are called "white". If a movie with a threshold image is shown, when the frame corresponds to the threshold, a white image will first be seen. Subsequently, black spots corresponding to local intensity minima will appear and grow. At a certain moment, the regions corresponding to the two local minima will merge. The final image will be black. The set of all connected branches in all frames in the movie is the set of all maximum regions; the minimum region can be obtained by inverting the intensity of I and running the same process.

[0080] The step S306 is specifically as follows:

[0081] S3061. Calculate the confidence of the target selection box in the maximum stable extreme value region set of the current binary image;

[0082] S3062. Obtain the target selection box with the highest confidence in the maximum stable extreme value region set of the current binary image as the comparison selection box;

[0083] S3063. Obtain a set of remaining selection boxes based on the comparison selection boxes.

[0084] S3064. Calculate the intersection-over-union ratio of each selection box in the remaining selection box set and the comparison selection box;

[0085] S3065: Based on the intersection-and-union ratios of each selection box in the remaining selection box set and the comparison selection box, remove the selection boxes whose intersection-and-union ratios are greater than the comparison threshold from the remaining selection box set to obtain a first selection box set;

[0086] S3066: Determine whether there is only one selection box in the first selection box set. If so, obtain the text information of the subtraction image corresponding to the current binary image and proceed to step S3067. Otherwise, update the maximum stable extreme value region set of the current binary image to the first selection box set and return to step S3062.

[0087] S3067. Determine whether the current binary image is the last binary image. If so, integrate the text information of the subtraction images corresponding to each binary image to obtain text information of several subtraction images, and proceed to step S4. Otherwise, update the current binary image to the next binary image, and return to step S3061.

[0088] In this embodiment, the maximum stable extreme region method is used. For the same text, many rectangular boxes will appear to select it. Non-maximum suppression is used to remove less effective rectangular boxes and select the most effective one. In object detection, NMS is a post-processing method that removes overlapping bounding boxes. After applying the bounding box prediction method described above, a set of detections D is generated for a specific object category in the image. Each detection is defined by a bounding box and a score. The detections in D are sorted by score, and the highest-scoring detection is greedily selected. Detections are skipped when the bounding box of a previously selected detection covers at least 50% of the bounding box. Using this theory, overlap can be removed, as shown in the figure below.

[0089] The step S4 is specifically as follows:

[0090] S401, obtaining a numerical approximate solution of an anisotropic diffusion equation for each pixel point according to a current subtraction image in the subtraction image data set;

[0091] S402, obtaining a gradient based on a numerical approximate solution of an anisotropic diffusion equation for each pixel;

[0092] S403. According to the gradient, a diffusion coefficient whose intensity and direction are related to the gradient is obtained;

[0093] S404. Based on the diffusion coefficient whose intensity and direction are related to the gradient, Taylor approximate diffusion is performed using the original image corresponding to the current subtraction image to obtain a first image:

[0094]

[0095] Where minTV(f) is the first image; f is the original image; λ is a constant; f0 is the image after denoising; is the square of the partial derivative of the pixel in the x direction; is the square of the partial derivative of the pixel in the y direction; dx is the differential of x; dy is the differential of y; Ω is the integration area;

[0096] S405, performing binarization processing on the first image to obtain a second image;

[0097] S406, performing erosion and dilation operations on the second image to obtain a processed image corresponding to the current subtracted image;

[0098] S407. Determine whether the current subtraction image is the last subtraction image in the subtraction image dataset. If so, obtain the processed image dataset based on the processed images corresponding to each subtraction image. Otherwise, update the current subtraction image to the next subtraction image, and return to step S401.

[0099] In this embodiment, a subtraction image containing a large amount of noise is processed by cyclically calculating the first-order and second-order partial derivatives of each pixel in the x and y directions. To ensure that the result does not differ too much from the original image, a fidelity term is added when solving the gradient minimum. The value of each point is calculated according to the formula, and then the above operation is iterated 100 times. The final image is still not usable directly because it is even more blurred. The image is eroded twice and then binarized with a threshold of 15 to obtain an image with less noise. In this embodiment, the number of erosion and dilation operations can be set according to the specific conditions of the image.

Claims

1. A mask generation method based on total variation denoising, characterized in that: The following steps are involved: S1. Using a web crawler to crawl the original image and modify the original image to obtain the tampered image, to obtain the original data; S2, discarding the tampered image with a different pixel size from the original image, and subtracting the original image from the corresponding tampered image to obtain a subtraction image dataset; S3. Based on the subtraction image data set, using the maximum stable extreme value region method and the non-maximum suppression method, obtain text information of several subtraction images; S4, performing total variation denoising processing on the subtraction images in the subtraction image dataset to obtain a processed image dataset; the step S4 is specifically as follows: S401, obtaining a numerical approximate solution of an anisotropic diffusion equation for each pixel point according to a current subtraction image in the subtraction image data set; S402, obtaining a gradient based on a numerical approximate solution of an anisotropic diffusion equation for each pixel; S403. According to the gradient, a diffusion coefficient whose intensity and direction are related to the gradient is obtained; S404. Based on the diffusion coefficient whose intensity and direction are related to the gradient, Taylor approximate diffusion is performed using the original image corresponding to the current subtraction image to obtain a first image: in, is the first image; is the original image; is a constant; is the image after noise reduction; is the square of the partial derivative of the pixel in the x direction; is the square of the partial derivative of the pixel in the y direction; is the differential of x; is the differential of y; is the integration area; S405, performing binarization processing on the first image to obtain a second image; S406, performing erosion and dilation operations on the second image to obtain a processed image corresponding to the current subtracted image; S407: Determine whether the current subtraction image is the last subtraction image in the subtraction image dataset. If so, obtain a processed image dataset based on the processed images corresponding to each subtraction image. Otherwise, update the current subtraction image to the next subtraction image, and return to step S401. S5. Superimpose the text information of the processed image and the corresponding subtraction image in the processed image data set to obtain a mask.

2. The mask generation method based on total variation denoising according to claim 1, characterized in that: Each original image in the original data in step S1 has several corresponding tampered images.

3. The mask generation method based on total variation denoising according to claim 1, characterized in that: The expression of the subtraction image data set in step S2 is: in, is a subtraction image dataset; For the A set of subtraction images corresponding to the original images; For the A set of subtraction images corresponding to the original images; For the original images; For the The original image corresponding to the a tampered image; Number the original image; is the total number of original images; Number the tampered image.

4. The mask generation method based on total variation denoising according to claim 1, characterized in that: The step S3 is specifically as follows: S301, obtaining a grayscale image of each subtraction image in a subtraction image dataset; S302, performing binarization processing on the grayscale images of each subtraction image to obtain a binary image of each grayscale threshold; S303, respectively calculate the change rate of the connected area in the binary image of each gray threshold: in, The grayscale threshold is The rate of change of connected regions in the binary graph; The grayscale threshold is Connected regions in a binary graph of ; The grayscale threshold is Connected regions in a binary graph of ; The grayscale threshold is Connected regions in a binary graph of ; is a small increment of the grayscale threshold; is the absolute value function; S304, obtaining the area where the change rate of the connected area in the binary image of each gray threshold is the local minimum, and obtaining the maximum stable extreme value area set of each binary image; S305, setting a comparison threshold; S306 , performing non-maximum suppression on the maximum stable extreme value region sets of each binary image according to the comparison threshold to remove overlap, thereby obtaining text information of a plurality of subtracted images.

5. The mask generation method based on total variation denoising according to claim 4, characterized in that: The step S306 is specifically as follows: S3061. Calculate the confidence of the target selection box in the maximum stable extreme value region set of the current binary image; S3062. Obtain the target selection box with the highest confidence in the maximum stable extreme value region set of the current binary image as the comparison selection box; S3063. Obtain a set of remaining selection boxes based on the comparison selection boxes. S3064. Calculate the intersection-over-union ratio of each selection box in the remaining selection box set and the comparison selection box; S3065: Based on the intersection-and-union ratios of each selection box in the remaining selection box set and the comparison selection box, remove the selection boxes whose intersection-and-union ratios are greater than the comparison threshold from the remaining selection box set to obtain a first selection box set; S3066: Determine whether there is only one selection box in the first selection box set. If so, obtain the text information of the subtraction image corresponding to the current binary image and proceed to step S3067. Otherwise, update the maximum stable extreme value region set of the current binary image to the first selection box set and return to step S3062. S3067. Determine whether the current binary image is the last binary image. If so, integrate the text information of the subtraction images corresponding to each binary image to obtain text information of several subtraction images, and proceed to step S4. Otherwise, update the current binary image to the next binary image, and return to step S3061.

Images