A method for detecting violation images using a small amount of data samples

Abstract

The application discloses a kind of rule violation image detection methods using small amount of data samples learning, it is characterized in that, including following steps:1) data enhancement;2) the image after data enhancement is input to the model newly designed and is trained;This method can identify a variety of types of rule violation image, high-dimensional data processing capacity is strong, generalization ability is strong, precision and robustness are high.

Description

Technical Field

[0001] This invention relates to artificial intelligence and illegal data detection technology, specifically an illegal image detection method that uses a small number of data samples for learning. Background Technology

[0002] Traditional illegal image detection algorithms often require manual feature extraction when given an input image, and they are weak in processing complex high-dimensional data and lack generalization ability, thus limiting their accuracy and robustness. Moreover, current research on illegal data identification, both domestically and internationally, is mostly focused on independent types of illegal data, such as adult images, gun images, or violent images, in order to identify that type of illegal image sample from a large number of samples to be detected.

[0003] Traditional machine learning methods for detecting adult images mainly fall into two categories: methods based on skin color and texture features, and pattern recognition methods based on handcrafted features. Grega et al. proposed an algorithm for automatically detecting and recognizing dangerous situations in closed-circuit television systems, capable of detecting firearms with a low false alarm rate. However, the final commercially available algorithm was a perception support system because such an automated system cannot assess the overall situation and therefore cannot determine the severity of potential dangers in the current environment. Déniz et al., in their research on violence detection, combined psychological research to propose a new algorithm that estimates extreme acceleration through consecutive frames and uses extreme acceleration as the primary detection feature. Summary of the Invention

[0004] The purpose of this invention is to address the shortcomings of existing technologies by providing a method for detecting illegal images using a small number of data samples. This method can identify and recognize various types of illegal images, has strong high-dimensional data processing capabilities, strong generalization ability, and high accuracy and robustness.

[0005] The technical solution to achieve the objective of this invention is:

[0006] A method for detecting violations using a small number of data samples includes the following steps:

[0007] 1) Data augmentation: Preprocessing of sample data in the class-imbalanced violation image dataset: The classes other than the class with the most samples in the class-imbalanced violation image dataset are expanded, that is, by adding Gaussian noise, adding salt and pepper noise, adding Poisson noise, image rotation and image flipping to augment the dataset. At the same time, the image size is uniformized. Through data augmentation, the number of training samples is expanded.

[0008] 2) Input the data-enhanced images into the newly designed model for training, including:

[0009] 2-1) Preliminary feature extraction: The input image is first passed through a convolutional layer with a kernel size of 7x7 to extract features and obtain a feature map. Then, a BN layer is used to standardize the feature map obtained in step 1), so that the elements in the feature map have zero mean and unit variance. The activation function is Leaky ReLU. Finally, MaxPool is used to reduce the size of the feature map and reduce the computational cost of subsequent processing.

[0010] 2-2) Further feature operations are performed on the feature map input multi-path residual blocks: The feature map processed in step 2-1) is processed in three paths, as follows:

[0011] 2-2-1) After receiving the feature map processed in step 2-1), the first path is processed by a convolutional layer composed of 1x1 convolutional kernels. Linear combinations are performed between different channels. After the 1x1 convolutional layer, a BN layer and a Leaky ReLU activation function are used. The 1x1 convolutional layer, BN layer and Leaky ReLU activation function are considered as the first combination. In the first path, three such combinations are stacked, and the output is the feature map processed by the first combination.

[0012] 2-2-2) After receiving the feature map processed in step 2-1), the second path inputs a convolutional layer composed of 3x3 convolutional kernels for processing. This extracts local features in the image and detects the relationship between pixels. The 3x3 convolutional layer uses a BN layer and a Leaky ReLU activation function. The 3x3 convolutional layer, BN layer and Leaky ReLU activation function are considered as the second combination. In the second path, three such combinations are stacked, and the output is the feature map processed by the second combination.

[0013] 2-2-3) After receiving the feature map processed in step 2-1), the third path inputs a convolutional layer composed of 5x5 convolutional kernels for processing to detect features over a wider range. After the 5x5 convolutional layer, a BN layer and a Leaky ReLU activation function are used. The 5x5 convolutional layer, BN layer and Leaky ReLU activation function are considered as the third combination. In the third path, three such combinations are stacked, and the output is the feature map processed by the third combination.

[0014] 2-2-4) Receive the feature maps processed by the three channels in steps 2-2-1), 2-2-2), and 2-2-3), as well as the feature map given in step 2-1). Add these four feature maps together and output the feature map to the subsequent layers.

[0015] 2-3) The ResBlock block is used to receive the feature map given in step 2-2-4) for feature extraction processing, and the output feature map is given to the subsequent layers;

[0016] 2-4) The AvgPool layer averages the elements of each feature map in groups, using a 2x2 kernel. The size of the processed feature map is one-quarter of the original size.

[0017] 2-5) FC Layer: Using the feature maps processed in step 2-4), specific classification is performed. Specifically, the model knows the true classification of the samples during training. If the model's prediction differs from the true classification during training, the loss function penalizes the model. Starting from the back end of the model, parameters are gradually updated backward to correct erroneous parameters, enabling the model to provide more accurate predictions. The loss function is improved because in real-world datasets, there is feature overlap between samples. Therefore, as the number of samples increases, the marginal benefit the model can extract from the data decreases. An anti-feature overlap cross-entropy loss function is used, introducing a hyperparameter to describe the degree of feature overlap. The cross-entropy loss function is modified according to the class imbalance of the dataset. The modified loss function allows the model to better extract features from fewer classes during training, improving the model's performance in class imbalance scenarios. The anti-feature overlap cross-entropy loss function is defined as shown in the formula:

[0018]

[0019] This allows the model to learn the ability to classify using feature maps;

[0020] 2-6) Input all training set images into the model as in step 2) to train the model, and finally obtain a model for classifying and identifying illegal data;

[0021] 2-7) Using the model trained in step 2-6), input the image to be detected and give a prediction of whether it is illegal data, which is used to classify and identify illegal image data.

[0022] In the field of identifying illegal images, obtaining a large amount of effective training data faces significant challenges. Compared to ordinary everyday images, which are relatively easy to obtain, the collection of illegal image data is subject to strict ethical and legal constraints, leading to a severe imbalance in the number of samples of different categories in the training dataset. This technical solution alleviates the problems of the difficulty in obtaining illegal image data and the high cost of labeling it.

[0023] This technical solution expands the number of existing small-class samples through data augmentation. These samples with slight noise or perturbation are helpful for the model to extract features.

[0024] This technical solution employs a multi-path residual module. By using multi-path residual modules with different convolutional kernel sizes, the model can extract features at different levels. The leaky ReLU activation function is used to avoid neuron deactivation.

[0025] This technical solution takes into account the feature overlap of most classes of samples and adopts a new anti-feature overlap loss function. The loss function can be weighted according to the actual training data to form a loss function that is more in line with the current dataset.

[0026] This method can identify and authenticate various types of illegal images, has strong high-dimensional data processing capabilities, strong generalization ability, and high accuracy and robustness. Attached Figure Description

[0027] Figure 1 This is a schematic diagram of the model structure and training flowchart for an example. Detailed Implementation

[0028] The present invention will be further described below with reference to the accompanying drawings and embodiments, but this is not intended to limit the scope of the invention.

[0029] Example:

[0030] Reference Figure 1 A method for detecting violations using a small number of data samples includes the following steps:

[0031] 1) Data augmentation: Preprocessing of sample data in the class-imbalanced violation image dataset: The classes other than the class with the most samples in the class-imbalanced violation image dataset are expanded, that is, by adding Gaussian noise, adding salt and pepper noise, adding Poisson noise, image rotation and image flipping to augment the dataset. At the same time, the image size is uniformized. Through data augmentation, the number of training samples is expanded.

[0032] 2) Input the data-enhanced images into the newly designed model for training, including:

[0033] 2-1) Preliminary feature extraction: The input image is first passed through a convolutional layer with a kernel size of 7x7 to extract features and obtain a feature map. Then, a BN layer is used to standardize the feature map obtained in step 1), so that the elements in the feature map have zero mean and unit variance. The activation function is Leaky ReLU. Finally, MaxPool is used to reduce the size of the feature map and reduce the computational cost of subsequent processing.

[0034] 2-2) Further feature operations are performed on the feature map input multi-path residual blocks: The feature map processed in step 2-1) is processed in three paths, as follows:

[0035] 2-2-1) After receiving the feature map processed in step 2-1), the first path is processed by a convolutional layer composed of 1x1 convolutional kernels. Linear combinations are performed between different channels. After the 1x1 convolutional layer, a BN layer and a Leaky ReLU activation function are used. The 1x1 convolutional layer, BN layer and Leaky ReLU activation function are considered as the first combination. In the first path, three such combinations are stacked, and the output is the feature map processed by the first combination.

[0036] 2-2-2) After receiving the feature map processed in step 2-1), the second path inputs a convolutional layer composed of 3x3 convolutional kernels for processing. This extracts local features in the image and detects the relationship between pixels. The 3x3 convolutional layer uses a BN layer and a Leaky ReLU activation function. The 3x3 convolutional layer, BN layer and Leaky ReLU activation function are considered as the second combination. In the second path, three such combinations are stacked, and the output is the feature map processed by the second combination.

[0037] 2-2-3) After receiving the feature map processed in step 2-1), the third path inputs a convolutional layer composed of 5x5 convolutional kernels for processing to detect features over a wider range. After the 5x5 convolutional layer, a BN layer and a Leaky ReLU activation function are used. The 5x5 convolutional layer, BN layer and Leaky ReLU activation function are considered as the third combination. In the third path, three such combinations are stacked, and the output is the feature map processed by the third combination.

[0038] 2-2-4) Receive the feature maps processed by the three channels in steps 2-2-1), 2-2-2), and 2-2-3), as well as the feature map given in step 2-1). Add these four feature maps together and output the feature map to the subsequent layers.

[0039] 2-3) The ResBlock block is used to receive the feature map given in step 2-2-4) for feature extraction processing, and the output feature map is given to the subsequent layers;

[0040] 2-4) The AvgPool layer averages the elements of each feature map in groups. In this example, the size of the feature map after processing with a 2x2 kernel is one-quarter of the size before processing.

[0041] 2-5) FC Layer: Using the feature maps processed in step 2-4), specific classification is performed. Specifically, the model knows the true classification of the samples during training. If the model's prediction differs from the true classification during training, the loss function penalizes the model. Starting from the back end of the model, parameters are gradually updated backward to correct erroneous parameters, enabling the model to provide more accurate predictions. The loss function is improved because in real-world datasets, there is feature overlap between samples. Therefore, as the number of samples increases, the marginal benefit the model can extract from the data decreases. An anti-feature overlap cross-entropy loss function is used, introducing a hyperparameter to describe the degree of feature overlap. The cross-entropy loss function is modified according to the class imbalance of the dataset. The modified loss function allows the model to better extract features from fewer classes during training, improving the model's performance in class imbalance scenarios. The anti-feature overlap cross-entropy loss function is defined as shown in the formula:

[0042]

[0043] This allows the model to learn the ability to classify using feature maps;

[0044] 2-6) Input all training set images into the model as in step 2) to train the model, and finally obtain a model for classifying and identifying illegal data;

[0045] 2-7) Using the model trained in step 2-6), input the image to be detected and give a prediction of whether it is illegal data, which is used to classify and identify illegal image data.

[0046] In this example, the experiment used five violation categories: adult photos, adult anime images, sexy images, guns, and violent conflicts, as well as two non-violation categories: ordinary portraits and ordinary anime images. The training data consisted mostly of easily obtainable non-violation images, with a small portion consisting of more difficult-to-obtain violation images. In other words, it is not necessary to have the same number of violation images as non-violation images; a smaller sample size of violation images is sufficient to achieve better accuracy.

Claims

1. A method for detecting violations using a small number of data samples, characterized in that, Includes the following steps: 1) Data augmentation: Preprocessing of sample data in the class-imbalanced violation image dataset: Expand the classes in the class-imbalanced violation image dataset except for the class with the most samples by adding Gaussian noise, adding salt and pepper noise, adding Poisson noise, image rotation and image flipping to augment the dataset, and uniformize the image size after data augmentation. 2) Input the data-enhanced images into the newly designed model for training: including: 2-1) Preliminary feature extraction: The input image is first passed through a convolutional layer with a kernel size of 7x7 to extract features and obtain a feature map. Then, a BN layer is used to standardize the feature map so that the elements in the feature map have zero mean and unit variance. The activation function is Leaky ReLU. Finally, the size of the feature map is reduced by MaxPool. 2-2) Further feature operations are performed on the feature map input multi-path residual blocks: The feature map processed in step 2-1) is processed in three paths, as follows: 2-2-1) After receiving the feature map processed in step 2-1), the first path is processed by a convolutional layer composed of 1x1 convolutional kernels. Linear combinations are performed between different channels. After the 1x1 convolutional layer, a BN layer and a Leaky ReLU activation function are used. The 1x1 convolutional layer, BN layer and Leaky ReLU activation function are considered as the first combination. In the first path, three such combinations are stacked, and the output is the feature map processed by the first combination. 2-2-2) After receiving the feature map processed in step 2-1), the second path inputs a convolutional layer composed of 3x3 convolutional kernels for processing to extract local features in the image and detect the relationship between pixels. The 3x3 convolutional layer uses a BN layer and a Leaky ReLU activation function. The 3x3 convolutional layer, BN layer and Leaky ReLU activation function are considered as the second combination. In the second path, three such combinations are stacked, and the output is the feature map processed by the second combination. 2-2-3) After receiving the feature map processed in step 2-1), the third path inputs a convolutional layer composed of 5x5 convolutional kernels for processing. After the 5x5 convolutional layer, a BN layer and a Leaky ReLU activation function are used. The 5x5 convolutional layer, BN layer and Leaky ReLU activation function are considered as the third combination. In the third path, three such combinations are stacked, and the output is the feature map processed by the third combination. 2-2-4) Receive the feature maps processed by the three channels in steps 2-2-1), 2-2-2), and 2-2-3), as well as the feature map given in step 2-1). Add these four feature maps together and output the feature map to the subsequent layers. 2-3) The ResBlock block is used to receive the feature map given in step 2-2-4) for feature extraction processing, and the output feature map is given to the subsequent layers; 2-4) The AvgPool layer averages the elements of each feature map in groups, and the size of the feature map after processing with a 2x2 kernel is one-quarter of the original size. 2-5) FC Layer: Using the feature maps processed in step 2-4), during model training, if the model's prediction differs from the true classification, the loss function will penalize the model. Parameters are updated gradually from the back end of the model to the front, correcting erroneous parameters. The improved loss function employs the inverse feature overlap cross-entropy loss function, introducing a hyperparameter to describe the degree of feature overlap. The cross-entropy loss function is modified based on the imbalance of the dataset classes. The definition of the inverse feature overlap cross-entropy loss function is shown in the formula: 2-6) Input all training set images into the model in the manner of steps 2-1) to 2-5) to finally obtain the model used for classifying and identifying illegal data; 2-7) Using the model trained in step 2-6), input the image to be detected and give a prediction of whether it is illegal data, which is used to classify and identify illegal image data.

Images