An image content noise processing method based on granular ball computing

Abstract

The present invention belongs to the field of image processing, and particularly relates to an image content noise processing method based on granular ball computing. The method includes obtaining the gradients of each pixel point of the picture to be processed; taking the pixel point with the minimum current gradient as the center, spreading outwards in a rectangular shape, and repeating the iteration to form multiple different rectangular regions; calculating the Euclidean distance from each pixel point to the center of its corresponding rectangular region, processing the reciprocals of the Euclidean distances of all pixel points to obtain a weight list for each rectangular region; multiplying the weight list of each rectangular region by the pixel value of the corresponding pixel point to obtain the pixel values of each part after clustering, thereby completing the clustering operation on the picture to be processed; performing resampling processing on the picture to be processed after the clustering operation based on the Gaussian distribution. The image processed by the present invention can have more anti-noise and anti-attack capabilities.

Description

Technical Field

[0001] The present invention belongs to the field of image processing, and particularly relates to an image content noise processing method based on granular sphere computing. Background Art

[0002] As the visual basis for humans to perceive things, images are an important basis for humans to obtain information from the outside world and an important way to perceive the outside world. In this context, it has become increasingly important to quickly identify and process images in various ways. In recent years, with the rapid development of deep learning technology, deep neural networks have been widely used in fields such as the image field.

[0003] Although deep neural networks have made significant breakthroughs in fields such as image recognition, and in some scenarios, the efficiency and accuracy of image recognition models exceed that of the human eye. The vulnerability shown by deep neural network recognition models when facing content noise such as adversarial examples still poses a hidden danger to their wide application. An adversarial example refers to a picture obtained by adding some tiny perturbations that are imperceptible to the human eye to a clean picture, resulting in the model making a wrong judgment during inference. The existence of adversarial examples will threaten the application of deep learning in the field of security sensitivity. There are various ways to generate adversarial examples, such as white-box attacks and black-box attacks. How to effectively defend against these attacks so that the neural network model can work properly is one of the urgent problems to be solved in the current neural network field.

[0004] Existing methods for processing content noise such as adversarial examples can generally be divided into two aspects: the data level and the model level. At the data processing level, such as adversarial training, by adding the generated adversarial examples to the training set, continuously expanding the scale of the training samples; such as JPEG image compression, removing unimportant parts of the image. At the model structure level, defense distillation models, GAN-based defense methods, defense frameworks such as MagNet have also been continuously proposed. However, all of the above methods have their own limitations. The adversarial training method consumes a great deal of time, the image compression method will reduce the classification accuracy of normal pictures, the defense distillation method cannot resist CW attacks, and most importantly, the above image content noise processing methods can only defend against one or several types of the same type of attacks, and their performance when facing unknown types of attacks is not satisfactory. Summary of the Invention

[0005] To solve the problems existing in the existing methods for processing image content noise, the present invention discovers that granule sphere computing is a structured thinking method, and its core is to granulate the problem object - divide it into small particles for calculation. Granule sphere computing methods have now been widely applied to machine learning algorithms and achieved good results, such as granule sphere k-means, granule sphere SVM, etc. In the field of images, how to granulate an image is a very meaningful task. Therefore, the present invention proposes a method for processing image content noise based on granule sphere computing. The present invention clusters the picture in a rectangular manner and then performs resampling operations using the Gaussian distribution. During this process, interference factors such as fine noise are excluded to a certain extent, and the picture after resampling still contains the main information of the original picture. When using the picture processed by this method to train a neural network, the classification performance of the trained neural network is significantly improved when facing different content noises or adversarial sample attacks.

[0006] A method for processing image content noise based on granule sphere computing according to the present invention, the method comprising:

[0007] Obtain the gradients of each pixel point of the picture to be processed;

[0008] Taking the pixel point with the minimum current gradient as the center, spreading outwards in a rectangular shape and repeating the iteration to form multiple different rectangular regions;

[0009] Calculate the Euclidean distance from each pixel point to the center of its corresponding rectangular region, and process the reciprocal of the Euclidean distances of all pixel points to obtain a weight list for each rectangular region;

[0010] Multiply the weight list of each rectangular region by the pixel value of the corresponding pixel point to obtain the pixel values of each part after clustering, thereby completing the clustering operation on the picture to be processed;

[0011] Perform resampling processing on the picture to be processed after the clustering operation based on the Gaussian distribution.

[0012] Advantages of the present invention:

[0013] When the present invention processes image content noise using the granule sphere computing method, it first clusters the picture in a rectangular manner and then performs resampling operations using the Gaussian distribution. During this process, interference factors such as fine noise are excluded to a certain extent, and the picture after resampling still contains the main information of the original picture. When using the picture processed by this method to train a neural network, the defense performance of the neural network is significantly improved when facing noise or adversarial sample attacks. In addition, this method does not target adversarial samples generated by a certain special type of attack. Theoretically, for any adversarial samples generated based on the original picture or other content noises added to the original picture, this method has a certain defense effect. Existing experimental data can ensure the feasibility of the solution of the present invention. Description of the Drawings

[0014] Figure 1 is a flowchart of the method for processing image content noise based on granule sphere calculation in an embodiment of the present invention;

[0015] Figure 2 is a schematic diagram of clustering pictures with the center of the minimum gradient point in an embodiment of the present invention;

[0016] Figure 3 is a schematic diagram of resampling pixel points according to the Gaussian distribution by using the data saved by clustering in an embodiment of the present invention.

[0017] Figure 4 is a schematic diagram of obtaining the weight list (value) corresponding to each pixel point in an embodiment of the present invention;

[0018] Figure 5 is a schematic diagram of multiplying pixel points by the weight list to complete the clustering operation in an embodiment of the present invention. Detailed Embodiments

[0019] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0020] Figure 1 is a flowchart of a method for processing image content noise based on granule sphere calculation in an embodiment of the present invention, as Figure 1 shown, the processing method includes:

[0021] 101. Obtain the gradients of each pixel point of the picture to be processed;

[0022] In this step, the picture to be processed refers to an adversarial sample formed by adding perturbations to a clean picture after an adversarial attack. The gradient of the pixel point refers to the change rate of a certain pixel of the picture to be processed in the x and y directions. For the edge part of the image, its gray value changes greatly, and the gradient value is also large; for the relatively smooth part of the image, its gray value changes little, and the corresponding gradient value is also small. Generally, the image gradient calculates the edge information of the image.

[0023] 102. Take the pixel point with the current minimum gradient as the center, spread outwards in a rectangular shape, and repeat the iteration to form multiple different rectangular regions;

[0024] Figure 2It is a schematic diagram of the process of spreading outwards in a rectangle with the pixel point having the minimum gradient in the embodiment of the present invention as the center. As Figure 2 shown, initially, the point with the minimum gradient among all pixel points is selected and used as the center. First, outlier points are defined as pixel points within the rectangular area whose gray value exceeds a certain threshold compared to the center point. The purity threshold is set as 1 minus the ratio of the number of outlier points within the rectangular area to the total number of pixel points. Before the clustering area meets the threshold condition, it spreads outwards in a rectangle from the center, and the diameter of each spread increases by 1.

[0025] In a specific embodiment of the present invention, with the pixel point having the minimum current gradient as the center, it spreads outwards in a rectangular shape and repeats the iteration to form multiple different rectangular regions, including selecting the pixel point at the remaining minimum gradient as the center for each iteration and spreading outwards in a rectangular shape until the pre-set purity threshold is reached and the spread terminates.

[0026] Among them, outlier points are defined as pixel points within the rectangular area whose gray value exceeds a certain preset gray value (set to 20 - 40 in this embodiment) compared to the center point. Purity is defined as 1 minus the ratio of the number of outlier points within the rectangular area to the total number of pixel points in this area (initially set to 0.90 - 0.99 in this embodiment, and the purity threshold will also increase dynamically as the rectangular area expands. When the increase reaches the acceptable maximum value, it will no longer change, and the acceptable maximum value is set to 0.99). The expansion of the rectangular area means that the total number of pixel points it contains is also increasing. If the purity threshold is fixed at a certain value, it will lead to insufficient clustering, and the entire clustering process will terminate prematurely, resulting in more single-sample points and affecting the effects of subsequent operations such as resampling.

[0027] 103. Calculate the Euclidean distance from each pixel point to the center of its corresponding rectangular area, and process the reciprocals of the Euclidean distances of all pixel points to obtain the weight list of each rectangular area;

[0028] In the embodiment of the present invention, the reciprocals of the Euclidean distances from each pixel point to the current rectangular area are processed through a normalization function. When a pixel point corresponds to only one rectangular area, the corresponding weight can be directly regarded as 1; when a pixel point corresponds to multiple different rectangular areas, a weight list of each pixel point in the current rectangular area is obtained, and the sum of the weights in the normalized weight list is 1; specifically, when a certain pixel point exists in multiple rectangular areas, that is, when there is an overlap of rectangular areas, a weight list form can be used to determine the weight value of each pixel point in a certain rectangular area. As Figure 4 shown, a picture containing 9 pixel points is divided into two areas after clustering. Among them, pixel point 5 corresponds to two rectangles at the same time. Assuming that the distances from the centers of the two rectangles to pixel point 5 are the same, after normalization, the two weight values corresponding to this point are both 0.5, and the corresponding weights of the remaining pixel points are all 1.

[0029] 104. Multiply the weight list of each rectangular region by the pixel value of the corresponding pixel point to obtain the pixel values of each part after clustering, thereby completing the clustering operation on the picture to be processed;

[0030] In the embodiment of the present invention, the pixel value of each pixel point of each class is multiplied by the corresponding weight value in the weight list, that is, the pixel value of the class is updated, so as to complete the clustering operation on the image to be processed. As Figure 5 shown, a picture containing 9 pixel points is divided into two regions after clustering, and the corresponding weight list (value) of each pixel point is obtained. After multiplying the pixel value by the weight, the updated pixel is obtained.

[0031] 105. Resample the picture to be processed after the clustering operation based on the Gaussian distribution.

[0032] In the embodiment of the present invention, first obtain a target image matrix of the same size as the picture to be processed and conforming to the same distribution, and then modify the value of each pixel point: for a point whose value is greater than the maximum pixel value saved, fix its value to the corresponding maximum value; for a point whose value is less than the minimum pixel value saved, fix its value to the corresponding minimum value.

[0033] Figure 3 This is a schematic diagram of resampling each pixel point according to the Gaussian distribution using the clustering data in the embodiment of the present invention. As Figure 3 shown, first obtain a matrix of the same size as the original picture and conforming to the same distribution, and then modify the value of each point: for a point whose value is greater than the maximum pixel value saved, fix its value to the corresponding maximum value; for a point whose value is less than the minimum pixel value saved, fix its value to the corresponding minimum value. Accordingly, the pixel values of the resampled picture are fixed within a certain range.

[0034] In the embodiment of the present invention, the Gaussian distribution function is expressed as:

[0035]

[0036] where f i (x) represents the probability density function of the Gaussian distribution of the i-th rectangular region, μ i represents the pixel mean value in the i-th rectangular region of the picture to be processed, and σ i represents the pixel standard deviation in the i-th rectangular region of the picture to be processed. μ i and σ i will be dynamically calculated and saved during the clustering process.

[0037] It can be understood that, in the embodiments of the present invention, the parameter i refers to the i-th rectangular region. Starting from the point with the minimum gradient, a rectangular region is obtained after n (not fixed, adaptively determining the number of clustering times and stopping when the purity threshold is reached) times of clustering. Each rectangular region corresponds to a pixel mean and variance. Therefore, for each rectangular region, the specific clustering operation is determined by its own distribution. The pixel mean and standard deviation of the region obtained through the clustering operation make the improved Gaussian function more adaptable to the distribution of each rectangular region, so that the overall new image obtained by resampling is smoother, and the pixels obtained by sampling each part are prevented from appearing overly fragmented.

[0038] In a preferred embodiment of the present invention, the method may further include step 106; the step 106 includes:

[0039] 106. Input the image to be processed after resampling into the target image classification model and train the target image classification model.

[0040] The image to be processed in this embodiment is processed through the above several steps before entering the neural network. The pixel values of the processed image are fixed within a certain range, reducing the impact of individual outliers on the whole.

[0041] Those of ordinary skill in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing relevant hardware through a program. The program can be stored in a computer-readable storage medium, and the storage medium can include: ROM, RAM, disk, or optical disc, etc.

[0042] Although the embodiments of the present invention have been shown and described, for those of ordinary skill in the art, it can be understood that various changes, modifications, substitutions, and variations can be made to these embodiments without departing from the principles and spirit of the present invention. The scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. An image content noise processing method based on granular ball computing, characterized in that The method includes: Obtaining the gradients of each pixel point of the picture to be processed; Centering on the pixel point with the minimum current gradient, spreading outwards in a rectangular shape and repeating the iteration to form multiple different rectangular regions; the process of centering on the pixel point with the minimum current gradient, spreading outwards in a rectangular shape and repeating the iteration to form multiple different rectangular regions includes selecting the pixel point at the location with the minimum remaining gradient as the center for each iteration and spreading outwards in a rectangular shape until the purity of the pixel points in the rectangular region reaches a pre-set purity threshold to terminate the spreading; wherein, the purity is defined as 1 minus the ratio of the number of heterogeneous points in the rectangular region to the total number of pixel points in the region, and a heterogeneous point is defined as a pixel point in the rectangular region whose gray value exceeds a certain preset gray value from the gray value of the center point; Calculating the Euclidean distance from each pixel point to the center of its corresponding rectangular region, and processing the reciprocals of the Euclidean distances of all pixel points to obtain a weight list for each rectangular region; Multiplying the weight list of each rectangular region by the pixel value of the corresponding pixel point to obtain the pixel values of each part after clustering, thereby completing the clustering operation on the picture to be processed; Performing resampling processing on the picture to be processed after the clustering operation based on the Gaussian distribution.

2. The image content noise processing method based on granule ball calculation according to claim 1, wherein, The process of calculating the Euclidean distance from each pixel point to the center of its corresponding rectangular region, and processing the reciprocals of the Euclidean distances of all pixel points to obtain a weight list for each rectangular region includes processing the reciprocal of the Euclidean distance from each pixel point to the current rectangular region through a normalization function. When a pixel point corresponds to only one rectangular region, the corresponding weight is directly regarded as 1; when a pixel point corresponds to multiple different rectangular regions, a weight list for each pixel point in the current rectangular region is obtained, and the sum of the weights in the normalized weight list is 1.

3. A method for processing image content noise based on granular computing according to claim 1, characterized in that The process of multiplying the weight list of each rectangular region by the pixel value of the corresponding pixel point to obtain the pixel values of each part after clustering, thereby completing the clustering operation on the picture to be processed includes multiplying the pixel value of the pixel points in each class by the corresponding weight value in the weight list, that is, updating the pixel values of the class, thereby completing the clustering operation on the picture to be processed.

4. A method for processing image content noise based on granule computing according to claim 1, characterized in that, The process of performing resampling processing on the picture to be processed after the clustering operation based on the Gaussian distribution includes first obtaining a target image matrix of the same size as the picture to be processed and conforming to the same distribution, and then modifying the values of each pixel point: for a point whose value is greater than the maximum pixel value saved, its value is fixed to the corresponding maximum value; for a point whose value is less than the minimum pixel value saved, its value is fixed to the corresponding minimum value.

5. A method for processing image content noise based on granule computing according to claim 4, characterized in that The Gaussian distribution function is expressed as: Among them, f i (x) represents the probability density function of the Gaussian distribution of the i-th rectangular region, and μ i represents the pixel mean within the i-th rectangular region of the image to be processed, and σ i represents the pixel standard deviation within the i-th rectangular region of the image to be processed. μ i and σ i will be dynamically calculated and saved during the clustering process.

6. A method for processing image content noise based on granule computing according to claim 1, characterized in that After performing resampling processing on the picture to be processed after the clustering operation based on the Gaussian distribution, it further includes inputting the resampled picture to be processed into a target image classification model to train the target image classification model.

Images