Image data processing method and device

Abstract

One embodiment of the present application provides an image data processing method and apparatus, comprising: obtaining an image dataset; for each pixel in the image dataset, calculating the weighted median of the pixel values ​​within a preset domain window centered on each pixel, and then updating the weighted median to the pixel value of the corresponding pixel; terminating the data preprocessing process in response to reaching a preset termination condition, and then updating the image dataset to obtain an updated image dataset; in response to a first prediction category being different from each second prediction category and a first confidence level being different from each second confidence level, determining the image dataset as an abnormal image dataset and outputting it; or in response to each second prediction category being the same as the first prediction category and the difference between the first confidence level and each second confidence level exceeding a threshold, determining the image dataset as an abnormal image dataset and outputting it. This greatly reduces the significant threat posed by adversarial samples to the security of the model.

Description

Technical Field

[0001] One or more embodiments of the present application relate to the field of data processing technology, specifically to the field of deep learning technology, and more particularly to an image data processing method and apparatus. Background Art

[0002] The widespread application of deep learning has gradually integrated a wave of high-tech AI technologies into production and daily life. AI ushered in a new round of technological iterations in the second decade of the 21st century, demonstrating performance far exceeding previous levels in many key areas. This is particularly true in the field of vision. Deep neural networks, powered by massive amounts of data, have enabled a large number of neural vision-based applications to become widely used in daily life, including in scenarios requiring high confidentiality and security, such as facial recognition, autonomous driving, and drone reconnaissance. The application of deep neural networks in these scenarios has made the security and robustness of neural networks a key research focus within AI security.

[0003] Although neural networks surpass human performance in image recognition and object detection, they are vulnerable to adversarial examples. Adversarial examples involve deliberately adding subtle perturbations to a normal dataset, causing the neural network model to misidentify semantic information and produce incorrect output. Adversarial examples pose a significant threat to model security. Summary of the Invention

[0004] In view of this, the purpose of one or more embodiments of the present application is to propose an image data processing method and device to solve the problem that adversarial samples pose a huge threat to the security of the model.

[0005] Based on the above objectives, one or more embodiments of the present application provide an image data processing method, including:

[0006] Get image dataset;

[0007] executing a data preprocessing process based on the image dataset, wherein the data preprocessing process includes: calculating, for each pixel point in the image dataset, a weighted median of pixel values ​​within a preset region window centered on each pixel point, and then updating the weighted median to a pixel value of the corresponding pixel point; terminating the data preprocessing process in response to reaching a preset termination condition, and then updating the image dataset to obtain an updated image dataset;

[0008] Randomly extracting a deep neural network model from a preset bagged deep neural network model set as a first deep neural network model, and inputting the updated image dataset into the first deep neural network model to obtain a first prediction category and a first confidence level;

[0009] Determine the other deep neural network models in the preset bagged deep neural network model set except the first deep neural network model as second deep neural network models, and then input the updated image dataset into the second deep neural network model to obtain each second prediction category and each second confidence level;

[0010] In response to the first prediction category being different from each second prediction category and the first confidence being different from each second confidence, the image dataset is determined to be an abnormal image dataset and output; or in response to the second prediction category being the same as the first prediction category and the difference between the first confidence and each second confidence exceeds a threshold, the image dataset is determined to be an abnormal image dataset and output.

[0011] Optionally, the preset termination condition includes:

[0012] At least one of the following is achieved: a threshold of the number of repeated iterations is reached, and a pixel value of the pixel does not change during the iteration process.

[0013] Optionally, the architectural difference between each deep neural network model in the preset bagged deep neural network model set is greater than a preset threshold.

[0014] Optionally, after randomly extracting a deep neural network model from a preset bagged deep neural network model set as the first deep neural network model, the method further includes:

[0015] The neuron parameters and the output of the activation function layer of the randomly selected first deep neural network model are randomly perturbed to the same extent.

[0016] Optionally, before randomly extracting a deep neural network model from the preset bagged deep neural network model set, the image data processing method further includes:

[0017] Obtain a training set, the training set including images stored in pixel-level matrix data;

[0018] The training set is divided into ten equal parts, with each part containing the same proportion of each image category;

[0019] For each deep neural network model in the bagged deep neural network model set, nine of the ten equally divided training sets are taken to form the training set of each deep neural network model, and then each deep neural network model is trained using the projected gradient descent method to obtain the optimal hyperparameters of each deep neural network model and various parameters in the model, and then the final trained preset bagged deep neural network model set is obtained.

[0020] One or more embodiments of the present application further provide an image data processing device, including:

[0021] an acquisition unit configured to acquire an image dataset;

[0022] a data preprocessing unit configured to perform a data preprocessing process based on the image data set, wherein the data preprocessing process includes: calculating, for each pixel point in the image data set, a weighted median of pixel values ​​within a preset region window centered on each pixel point, and then updating the weighted median to the pixel value of the corresponding pixel point; terminating the data preprocessing process in response to reaching a preset termination condition, and then updating the image data set to obtain an updated image data set;

[0023] A first prediction unit is configured to randomly extract a deep neural network model from a preset bagged deep neural network model set as a first deep neural network model, and input the updated image dataset into the first deep neural network model to obtain a first prediction category and a first confidence level;

[0024] a second prediction unit, configured to determine other deep neural network models in the preset bagged deep neural network model set except the first deep neural network model as second deep neural network models, and then input the updated image dataset into the second deep neural network model to obtain respective second prediction categories and respective second confidence levels;

[0025] An abnormal data set determination unit is configured to, in response to the first prediction category being different from each second prediction category and the first confidence being different from each second confidence, determine that the image data set is an abnormal image data set and output it; or, in response to the second prediction category being the same as the first prediction category and the difference between the first confidence and each second confidence exceeds a threshold, determine that the image data set is an abnormal image data set and output it.

[0026] Optionally, the preset termination condition includes:

[0027] At least one of the following is achieved: a threshold of the number of repeated iterations is reached, and a pixel value of the pixel does not change during the iteration process.

[0028] Optionally, the architectural difference between each deep neural network model in the preset bagged deep neural network model set is greater than a preset threshold.

[0029] Optionally, the image data processing apparatus further includes a random perturbation unit configured to:

[0030] The neuron parameters and the output of the activation function layer of the randomly selected first deep neural network model are randomly perturbed to the same extent.

[0031] Optionally, the image data processing apparatus further includes a model training unit configured to:

[0032] Obtain a training set, the training set including images stored in pixel-level matrix data;

[0033] The training set is divided into ten equal parts, with each part containing the same proportion of each image category;

[0034] For each deep neural network model in the bagged deep neural network model set, nine of the ten equally divided training sets are taken to form the training set of each deep neural network model, and then each deep neural network model is trained using the projected gradient descent method to obtain the optimal hyperparameters of each deep neural network model and various parameters in the model, and then the final trained preset bagged deep neural network model set is obtained.

[0035] As can be seen from the above, the image data processing method and device provided by one or more embodiments of the present application obtain an image data set; based on the image data set, perform a data preprocessing process, wherein the data preprocessing process includes: for each pixel point in the image data set, calculate the weighted median of the pixel values ​​in a preset domain window centered on each pixel point, and then update the weighted median to the pixel value of the corresponding pixel point; in response to reaching a preset termination condition, terminate the data preprocessing process, and then update the image data set to obtain an updated image data set; randomly extract a deep neural network model from the preset bagged deep neural network model set as the first deep neural network model, and input the updated image data set into the first deep neural network model. The method uses a deep neural network model to obtain a first prediction category and a first confidence level; determines the other deep neural network models in the preset bagged deep neural network model set except the first deep neural network model as second deep neural network models, and then inputs the updated image dataset into the second deep neural network model to obtain each second prediction category and each second confidence level; in response to the first prediction category being different from each second prediction category and the first confidence level being different from each second confidence level, determines that the image dataset is an abnormal image dataset and outputs it; or in response to the second prediction category being the same as the first prediction category and the difference between the first confidence level and each second confidence level exceeds a threshold, determines that the image dataset is an abnormal image dataset and outputs it. This improves the robust detection capability of the deep learning model and the security of the related system, and greatly reduces the huge threat posed by adversarial samples to the security of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] In order to more clearly illustrate one or more embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only one or more embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0037] Figure 1 This is a flowchart of an image data processing method according to one embodiment of the present application;

[0038] Figure 2 This is a flowchart of an image data processing method according to one embodiment of the present application;

[0039] Figure 3 This is a schematic structural diagram of an image data processing device according to one embodiment of the present application;

[0040] Figure 4 This is a schematic diagram of the hardware structure of an image data processing electronic device according to one embodiment of the present application. DETAILED DESCRIPTION

[0041] In order to make the objectives, technical solutions and advantages of the present disclosure more clearly understood, the present disclosure is further described in detail below in conjunction with specific embodiments and with reference to the accompanying drawings.

[0042] It should be noted that, unless otherwise defined, the technical terms or scientific terms used in one or more embodiments of the present application should have the usual meanings understood by people with ordinary skills in the field to which the present disclosure belongs. The "first", "second" and similar words used in one or more embodiments of the present application do not indicate any order, quantity or importance, but are only used to distinguish different components. "Include" or "comprise" and similar words mean that the elements or objects appearing before the word include the elements or objects listed after the word and their equivalents, without excluding other elements or objects. "Connect" or "connected" and similar words are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. "Up", "down", "left", "right" and the like are only used to indicate relative positional relationships. When the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0043] Figure 1 The following is a flow chart of an image data processing method according to an embodiment of the present application. The image data processing method according to the embodiment of the present application includes:

[0044] Step S101: Acquire an image dataset.

[0045] The image data processing method of the present embodiment can be executed by a server, which can obtain an image dataset via a wired or wireless connection. In step S101, the acquired image dataset can be a raw dataset corresponding to the attack and defense requirements of the neural network model. Based on the spatial domain gradient smoothing strategy, the image is subjected to spatial domain gradient smoothing, image classification using the pre-training dataset, and classification using the test dataset to obtain an optimized dataset.

[0046] Specifically, the original data set can be an existing widely used data set, or it can be a custom data set uploaded by the user or server. The original data needs to be expressed as a "picture of pixel-level matrix data storage." The original data is used to train neural network models, such as classification systems, face recognition and other neural network models. Gradient smoothing in the spatial domain is required before training. In the present disclosure, the image data set is mainly used to train diverse deep neural network models.

[0047] Step S102: Execute a data preprocessing process based on the image data set, wherein the data preprocessing process includes: for each pixel point in the image data set, calculate the weighted median of the pixel values ​​within a preset region window centered on each pixel point, and then update the weighted median to the pixel value of the corresponding pixel point. In response to reaching a preset termination condition, terminate the data preprocessing process, and then update the image data set to obtain an updated image data set.

[0048] Specifically, the preset termination conditions include:

[0049] At least one of the following is achieved: a threshold of the number of repeated iterations is reached, and a pixel value of the pixel does not change during the iteration process.

[0050] For example, based on the spatial domain gradient smoothing strategy, the weighted median of the pixel values ​​in the domain window corresponding to each pixel point (x, y) in the image data set is calculated, and the weighted median is updated to the pixel value of the currently calculated pixel point.

[0051] The original dataset is processed in the spatial domain using a spatial gradient smoothing strategy to obtain an optimized and updated image dataset. Spatial domain smoothing performs domain median processing on the images contained in the training dataset. However, this median processing method does not simply median the selected domain and the currently selected pixel. Instead, a weight is selected. This weight represents the balance between the domain value and the current pixel value. A larger weight indicates a greater weighting of the domain value over the current pixel value. This weight can be set between 0 and 1.

[0052] Specifically, different image data processing methods can be used based on the needs of training and application scenarios. For example, the averaging method and the median method of spatial domain filtering can be used. The difference between the averaging method and the median method lies in the different ways of calculating the neighborhood value. Different image processing methods are suitable for different styles. For example, the averaging method can be used for images with high noise and many disturbances, while the median method can be used for images with many edges and sharpness. The calculated weighted median method can be used to iteratively update the pixel value of each pixel in the image dataset until the pixel value is stable and unchanged.

[0053] For example, for step S102, the image dataset is subjected to a spatial domain smoothing weighted median method, and a 5*5 domain (the embodiment of the present application does not specifically limit the size of the domain) median method is calculated for each color channel and each pixel. If the pixel is located at the edge of the image and a 5*5 median method centered on the pixel cannot be formed, the pixel value of the pixel is the weighted median value obtained by the intersection of the 5*5 matrix centered on the pixel and the image, and then the pixel value of each pixel in the optimized image dataset is updated to obtain a new image dataset.

[0054] Step S103: randomly extract a deep neural network model from the preset bagged deep neural network model set as the first deep neural network model, and input the updated image data set into the first deep neural network model to obtain a first prediction category and a first confidence level.

[0055] Specifically, the architectural differences between the respective deep neural network models in the preset bagged deep neural network model set are greater than a preset threshold.

[0056] Specifically, before randomly extracting a deep neural network model from the preset bagged deep neural network model set, the method further includes:

[0057] Obtain a training set, the training set including images stored in pixel-level matrix data;

[0058] The training set is divided into ten equal parts, with each part containing the same proportion of each image category;

[0059] For each deep neural network model in the bagged deep neural network model set, nine of the ten equally divided training sets are taken to form the training set of each deep neural network model, and then each deep neural network model is trained using the projected gradient descent method to obtain the optimal hyperparameters of each deep neural network model and various parameters in the model, and then the final trained preset bagged deep neural network model set is obtained.

[0060] For example, based on the currently commonly used advanced deep neural network models, multiple, for example, ten neural network models and diversified defense models are selected, and each deep neural network model is trained.

[0061] Training can be done in stages. For example, each stage uses different hyperparameters, such as the step size, learning rate, and optimization method. Each stage of training builds on the results of the previous stage.

[0062] In step S103, because there are numerous neural network models, to enrich their diversity, the training set needs to be divided into multiple parts, for example, ten. Each model is then trained on only nine of the ten parts. Within each part, it is important to ensure that the proportion of each image data category is uniform to ensure uniform model training performance.

[0063] The training data in the training set is optimized using weighted median processing in the spatial domain. This processing allows the deep model to learn the most pristine images with the least adversarial perturbations. This allows the deep model to learn the optimal set of images from a completely clean dataset. After each deep model is fully trained, its parameters are fully trained.

[0064] In step S103, the ten neural network models can be grouped into a bagged deep neural network model set, and indiscriminate extraction can be performed from these ten neural networks. At this time, it is necessary to ensure that the probability of each model being extracted is consistent and the model distribution conforms to a uniform distribution.

[0065] For step S103, the neuron parameters of the deep neural network model are randomly perturbed. The perturbation needs to obey the Gaussian distribution and is modeled as the Gaussian distribution N(μ, σ 2 C). Where μ is the mean of the distribution, σ is the learning step size, and C is the covariance matrix, which means that each neuron parameter will fluctuate randomly to cope with the disturbance of the image data to be tested.

[0066] For step S103 , μ is the mean of the distribution, which may be a small perturbation of 1E-5, that is, a random perturbation with an expected amplitude of 1E-5.

[0067] In step S103, the neuron parameters of the selected deep neural network are updated, including the parameters of each neuron, such as the parameters of the bias and weight.

[0068] In step S103, the sample to be tested is input into one of the deep neural network models extracted from the bagged deep neural network model set.

[0069] For step S103, the activation layer of one of the deep neural network models extracted from the bagged deep neural network model set that the sample to be tested passes through is randomly perturbed. The perturbation needs to also conform to the Gaussian distribution N(μ, σ 2 C). Where μ is the mean of the disturbance and σ is the deviation.

[0070] For step S103, the perturbation of the activation layer further eliminates the impact of the adversarial perturbation, preventing the adversarial perturbation from affecting the model output with high confidence.

[0071] In step S104, the other deep neural network models in the preset bagged deep neural network model set except the first deep neural network model are determined as second deep neural network models, and then the updated image data set is input into the second deep neural network model to obtain each second prediction category and each second confidence.

[0072] For step S104, the sample to be tested is input into other deep neural network models that have not been selected in the bagged deep neural network model set (that is, the second deep neural network model. It can be understood that there can be multiple second deep neural network models) to obtain the output, that is, the second prediction categories and the second confidence levels output by each second deep neural network model.

[0073] Step S105: In response to the first prediction category being different from each second prediction category and the first confidence being different from each second confidence, the image dataset is determined to be an abnormal image dataset and output; or in response to the second prediction category being the same as the first prediction category and the difference between the first confidence and each second confidence exceeds a threshold, the image dataset is determined to be an abnormal image dataset and output.

[0074] Compare the output category and confidence of the deep neural network model extracted from the bagged deep neural network model set (i.e., the first deep neural network model) with the predicted value and confidence of the other deep neural network models not extracted (i.e., the second deep neural network model). It will be understood that the "predicted value" mentioned in the embodiments of the present application refers to the "predicted category" for the input image data.

[0075] For example, since there are nine other protected models (that is, deep neural network models that are not extracted from the bagged deep neural network model set), it cannot be ensured that the outputs and confidence levels of these nine models are the same. Therefore, the outputs of these nine models need to be processed. If the output of the experimental deep neural network model (that is, the first deep neural network model) is completely different from the output of the control group (that is, each second deep neural network model), then the input image data is considered to be a sample carrying adversarial perturbations, otherwise the input image data is determined to be normal image data; or if the output category of the experimental group (that is, the first deep neural network model) is the same as the output category of the control group (that is, each second deep neural network model), but the confidence level exceeds the threshold (that is, the first confidence level is different from each second confidence level, and the difference between the first confidence level and each second confidence level exceeds the confidence level threshold), then the input image data is determined to be a sample carrying adversarial perturbations, otherwise the input image data is determined to be normal image data.

[0076] The embodiment of the present application obtains an image data set; based on the image data set, performs a data preprocessing process, wherein the data preprocessing process includes: for each pixel point in the image data set, calculating the weighted median of the pixel values ​​in a preset domain window centered on each pixel point, and then updating the weighted median to the pixel value of the corresponding pixel point; in response to reaching a preset termination condition, terminates the data preprocessing process, and then updates the image data set to obtain an updated image data set; randomly extracts a deep neural network model from a preset bagged deep neural network model set as a first deep neural network model, and inputs the updated image data set into the first deep neural network model to obtain a first preprocessed image data set. The method comprises the following steps: determining the prediction category and the first confidence level; determining the other deep neural network models in the preset bagged deep neural network model set except the first deep neural network model as the second deep neural network model, and then inputting the updated image dataset into the second deep neural network model to obtain each second prediction category and each second confidence level; in response to the first prediction category being different from each second prediction category and the first confidence level being different from each second confidence level, determining the image dataset as an abnormal image dataset and outputting it; or in response to the second prediction category being the same as the first prediction category and the difference between the first confidence level and each second confidence level exceeding a threshold, determining the image dataset as an abnormal image dataset and outputting it. This improves the robust detection capability of the deep learning model and the security of the related system, and greatly reduces the huge threat posed by adversarial samples to the security of the model.

[0077] Figure 2 The figure is a flowchart of an image data processing method according to an embodiment of the present application.

[0078] like Figure 2As shown, the image data processing method of the embodiment of the present application includes:

[0079] Step S201: Acquire an image dataset.

[0080] The acquired image dataset is divided into ten parts, each containing the same proportion of categories.

[0081] Step S202: Execute a data preprocessing process based on the image data set, wherein the data preprocessing process includes: for each pixel point in the image data set, calculate the weighted median of the pixel values ​​within a preset region window centered on each pixel point, and then update the weighted median to the pixel value of the corresponding pixel point. In response to reaching a preset termination condition, terminate the data preprocessing process, and then update the image data set to obtain an updated image data set.

[0082] For example, for each pixel point (x, y), there is a domain window. The weighted median of the pixel values ​​within the domain window size is calculated, and the value of the pixel point is calculated using the median formula. The specific formula is as follows:

[0083] G(x, y) = βMedian(I p，q∈Ω (p, q))+I(x, y) / Median(I p，q∈Ω (p, q))+I(x, y)*I(x, y)

[0084] Among them, β is a configurable weight parameter, Ω is the domain of (x, y), (p, q) is the coordinate in the domain of Ω, I(x, y) is the pixel value of the original image, Median(I p,q∈Ω (p,q)) represents the median of all pixel values ​​in the original image. The size of the Ω domain is a configurable parameter like β.

[0085] Repeat step S202 until the pixel value no longer changes or the number of iterations n is reached, to obtain an updated image data set.

[0086] Furthermore, based on the gradient smoothing strategy, we can use the image dataset to enhance the training method to obtain a stronger and more robust training set, which will directly improve the effectiveness and reliability of subsequent image data analysis. The following steps are included:

[0087] Step S2021: input a two-dimensional image f(x, y), and initialize the domain window size (e.g., a 5*5 window) and the number of iterations n;

[0088] Step S2022, initialize the number of loops i = 1, set J_i as the output image of the i-th iteration, where J_0 = I;

[0089] Step S2023, the number of loops i is incremented by 1. If i ≤ n, the pixel value of each pixel Ji-1(x, y) of the output image Ji of the i-th iteration is calculated and the above steps are continued. If i > n, the above steps S2021 to S2023 are continued.

[0090] Step S2024: For each pixel point (x, y), there is a neighborhood window, and the median of the pixel values ​​within the neighborhood window size is calculated:

[0091] Re-use the median formula to calculate the value of the pixel; G(x, y) = βMedian(I p，q∈Ω (p, q))+I(x, y) / Median(I p，q∈Ω (p, q))+I(x, y)*I(x, y)

[0092] Among them, β is a configurable weight parameter, Ω is the domain of (x, y), (p, q) is the coordinate in the domain of Ω, I(x, y) is the pixel value of the original image, Median(I p,q∈Ω (p,q)) represents the median of all pixel values ​​in the original image. The size of the Ω domain is a configurable parameter like β.

[0093] Step S2025, replacing the value of the pixel with the result of the above-mentioned domain median calculation;

[0094] Repeat the above steps S2021 to S2025 until the value of no pixel in the image data set changes or the number of iterations n is reached, and then stop the preprocessing of the image data set.

[0095] In the embodiment of the present invention, pixel smoothing in the image space is to correct the value of the pixel according to the median of the pixel's neighborhood, thereby reducing the intensity difference between adjacent pixels.

[0096] Step S203: randomly extract a deep neural network model from the preset bagged deep neural network model set as the first deep neural network model.

[0097] For example, the bagged deep neural network model set is a selection of ten most advanced deep neural network models with large differences, such as:

[0098] AlexNet, VGG, GoogleNet, ResNet, DenseNet, Transformer, Xception, NASNet, EfficientNet, SENet, ShuffleNet, etc. Ensure that the architecture of each deep neural network model is different and has sufficient diversity.

[0099] At least one of a public dataset or a custom dataset that matches the usage scenario of the pre-trained deep neural network model is selected as the original dataset. The original dataset includes images stored in pixel-level matrix data. This original dataset is the image dataset obtained in step S101. A deep neural network model is trained using nine of the ten divided image datasets. This ensures that the difference between the image dataset used by each deep neural network model and the image datasets used by other deep neural network models is not empty and has an intersection. The image dataset is divided into ten equal parts to ensure that each image data category is equally represented in each image dataset, preventing the deep neural network model from being affected by the image dataset category during evaluation. The deep neural network model training process is mainly divided into two steps, with different training parameters in each step to address model overfitting and instability. Specifically, the deep neural network model is trained using projected gradient descent. In the first few epochs, an optimal set of parameters is found to achieve a temporary optimal model performance. Based on the performance of the first epoch, another set of optimal parameters is searched for in the next few epochs to achieve optimal model performance. An epoch represents the completion of a forward computation and backpropagation process after all data is fed into the deep neural network model. The length of an epoch is related to the diversity of the image dataset; the greater the diversity, the larger the epoch. Through training, the optimal hyperparameters and various parameters of all ten deep neural network models are obtained. The ten deep neural network models described above are grouped together into a set called the "bagged deep neural network model set." This bagged deep neural network model set is used only to detect malicious input and is not used to predict classification results for normal images. To detect malicious input image data, a single deep neural network model is randomly selected from the bagged deep neural network model set using a uniform distribution.

[0100] The principles of steps S201 to S203 are similar to those of steps S101 to S103 and will not be described in detail here.

[0101] Step S204: Perform random perturbations of the same degree on the randomly selected neuron parameters and the output of the activation function layer of the first deep neural network model, for example, with an amplitude of 1E-5.

[0102] Randomly fluctuate the neuron parameters and output of the activation function layer of a randomly selected deep neural network model from the set of bagged deep neural network models, where the parameters include weights and biases.

[0103] A small adjustment is made to the neuron parameters in the extracted deep neural network model, such as a random perturbation of 1E-5. The image data to be tested is input into one of the deep neural network models extracted from the bagged deep neural network model set. The image data to be tested passes through the hidden layer of the extracted deep neural network model, and the output of its activation function layer is also randomly perturbed to the same extent as the neuron parameters of the deep neural network.

[0104] Step S205: Input the updated image dataset into the first deep neural network model to obtain a first prediction category and a first confidence level.

[0105] In step S206, the other deep neural network models in the preset bagged deep neural network model set except the first deep neural network model are determined as second deep neural network models, and then the updated image data set is input into the second deep neural network model to obtain each second prediction category and each second confidence.

[0106] Step S207: In response to the first prediction category being different from each second prediction category and the first confidence being different from each second confidence, the image dataset is determined to be an abnormal image dataset and output; or in response to the second prediction category being the same as the first prediction category and the difference between the first confidence and each second confidence exceeds a threshold, the image dataset is determined to be an abnormal image dataset and output.

[0107] Compare the predicted category and confidence output by the first deep neural network model with the predicted category and confidence of each second deep neural network model. If the predicted categories are different or the predicted categories are the same but the confidence difference exceeds a threshold, the input image data is determined to be malicious input; otherwise, the input image data is determined to be normal image data, that is, not malicious input.

[0108] The image data processing method of the embodiment of the present application is based on the diversity of deep neural networks and uses methods such as fine-tuning of deep neural network parameters to model the randomness of unknown attacks. In order to prevent black-box attacks on neural networks, the diversity and randomness of the target model are further improved, and the migration ability and attack intensity of the opponent's black-box attacks are tested, thereby improving the robust detection capability of the deep learning model and the security of related systems. This method requires a variety of neural networks with different architectures and data sets from different sources, and randomly perturbs the randomly extracted neural network model, thereby improving the neural network's ability to recognize adversarial perturbations.

[0109] The image data processing method provided in the embodiment of the present application uses weighted median processing of spatial domain gradient smoothing, a training method of a neural network model, and the use of multiple neural network output comparison to enhance the diversity of the neural network, and by perturbing the parameters of the experimental network with Gaussian distribution, it ensures that the opponent cannot obtain the data and parameters of the real experimental network, so that the opponent's attack is in a black box state and cannot achieve the expected attack behavior, thereby protecting the input and output of the neural network. The defense capability of the deep neural network against attacks is improved, and the security of deep learning related systems is further improved. The accuracy and efficiency of the neural network's adversarial defense are improved, and the security of deep learning related systems is improved.

[0110] like Figure 3 As shown, the present application further provides an image data processing device 300 , comprising an acquisition unit 301 , a data preprocessing unit 302 , a first prediction unit 303 , a second prediction unit 304 and an abnormal data set determination unit 305 .

[0111] The acquisition unit 301 is configured to acquire an image dataset.

[0112] The data preprocessing unit 302 is configured to perform a data preprocessing process based on the image data set, wherein the data preprocessing process includes: for each pixel point in the image data set, calculating the weighted median of the pixel values ​​within a preset range window centered on each pixel point, and then updating the weighted median to the pixel value of the corresponding pixel point. In response to reaching a preset termination condition, the data preprocessing process is terminated, and the image data set is updated to obtain an updated image data set.

[0113] The first prediction unit 303 is configured to randomly extract a deep neural network model from a preset bagged deep neural network model set as a first deep neural network model, and input the updated image data set into the first deep neural network model to obtain a first prediction category and a first confidence level.

[0114] The second prediction unit 304 is configured to determine the other deep neural network models in the preset bagged deep neural network model set except the first deep neural network model as the second deep neural network model, and then input the updated image data set into the second deep neural network model to obtain each second prediction category and each second confidence.

[0115] The abnormal data set determination unit 305 is configured to determine that the image data set is an abnormal image data set and output it in response to the first prediction category being different from each second prediction category and the first confidence being different from each second confidence; or to determine that the image data set is an abnormal image data set and output it in response to each second prediction category being the same as the first prediction category and the difference between the first confidence and each second confidence exceeds a threshold.

[0116] In some embodiments, the preset termination condition includes at least one of: reaching a threshold of the number of repeated iterations, and the pixel value of the pixel point no longer changing during the iteration process.

[0117] In some embodiments, the architectural differences between the respective deep neural network models in the preset bagged deep neural network model set are greater than a preset threshold.

[0118] In some embodiments, the image data processing apparatus further includes Figure 3 The random perturbation unit not shown in the figure is configured to perform random perturbations of the same degree on the neuron parameters and the output of the activation function layer of the randomly selected first deep neural network model.

[0119] In some embodiments, the image data processing apparatus further includes Figure 3 The model training unit not shown in the figure is configured to: obtain a training set, the training set including images stored in pixel-level matrix data; divide the training set into ten equal parts, with each image category having the same proportion in each part; for each deep neural network model in the bagged deep neural network model set, take nine parts of the ten-part training set to constitute the training set of each deep neural network model, and then train each deep neural network model using the projected gradient descent method to obtain the optimal hyperparameters of each deep neural network model and various parameters in the model, and then obtain the final trained preset bagged deep neural network model set.

[0120] It should be noted that the method of one or more embodiments of the present application can be performed by a single device, such as a computer or server. The method of this embodiment can also be applied in a distributed scenario, where multiple devices cooperate with each other to complete the method. In such a distributed scenario, one of the multiple devices may only perform one or more steps of the method of one or more embodiments of the present application, and the multiple devices will interact with each other to complete the method.

[0121] The foregoing description describes specific embodiments of the present application. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims can be performed in an order different from that described in the embodiments and still achieve the desired results. Furthermore, the processes depicted in the accompanying drawings do not necessarily require the specific order shown or the sequential order to achieve the desired results. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0122] For the convenience of description, the above devices are described as being functionally divided into various modules. Of course, when implementing one or more embodiments of the present application, the functions of each module can be implemented in the same or multiple software and / or hardware.

[0123] The apparatus of the above embodiment is used to implement the corresponding method in the above embodiment and has the beneficial effects of the corresponding method embodiment, which will not be described in detail here.

[0124] Figure 4 10. A more specific hardware structure diagram of an electronic device for image data processing provided in an embodiment of the present application is shown. The device may include: a processor 1010, a memory 1020, an input / output interface 1030, a communication interface 1040, and a bus 1050. The processor 1010, the memory 1020, the input / output interface 1030, and the communication interface 1040 are communicatively connected to each other within the device via the bus 1050.

[0125] The processor 1010 can be implemented using a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present application.

[0126] The memory 1020 can be implemented in the form of ROM (Read Only Memory), RAM (Random Access Memory), static storage devices, dynamic storage devices, etc. The memory 1020 can store an operating system and other application programs. When the technical solutions provided in the embodiments of the present application are implemented through software or firmware, the relevant program codes are stored in the memory 1020 and are called and executed by the processor 1010.

[0127] The input / output interface 1030 is used to connect input / output modules to implement information input and output. The input / output modules can be configured as components within the device (not shown in the figure) or can be externally connected to the device to provide corresponding functions. Input devices may include a keyboard, mouse, touch screen, microphone, various sensors, etc., and output devices may include a display, speaker, vibrator, indicator light, etc.

[0128] The communication interface 1040 is used to connect to a communication module (not shown) to enable communication between the device and other devices. The communication module can communicate via a wired method (such as USB, network cable, etc.) or a wireless method (such as mobile network, WiFi, Bluetooth, etc.).

[0129] The bus 1050 comprises a path for transmitting information between the various components of the device (eg, the processor 1010 , the memory 1020 , the input / output interface 1030 , and the communication interface 1040 ).

[0130] It should be noted that although the above device only shows the processor 1010, the memory 1020, the input / output interface 1030, the communication interface 1040, and the bus 1050, in a specific implementation, the device may also include other components necessary for normal operation. In addition, it will be understood by those skilled in the art that the above device may only include the components necessary to implement the embodiments of the present application, and does not necessarily include all the components shown in the figure.

[0131] The computer-readable media of this embodiment include permanent and non-permanent, removable and non-removable media that can be used to store information by any method or technology. The information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, read-only compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device.

[0132] Those skilled in the art should understand that the discussion of any of the above embodiments is merely illustrative and is not intended to imply that the scope of the present disclosure (including the claims) is limited to these examples. Based on the concept of the present disclosure, the technical features in the above embodiments or different embodiments may be combined, the steps may be implemented in any order, and there are many other variations of different aspects of one or more embodiments of the present application as above, which are not provided in detail for the sake of simplicity.

[0133] In addition, to simplify the description and discussion, and in order not to make one or more embodiments of the present application difficult to understand, known power / ground connections to integrated circuit (IC) chips and other components may or may not be shown in the provided figures. In addition, the device can be shown in the form of a block diagram to avoid making one or more embodiments of the present application difficult to understand, and this also takes into account the following fact, that is, the details of the implementation of these block diagram devices are highly dependent on the platform on which one or more embodiments of the present application will be implemented (that is, these details should be fully within the scope of understanding of those skilled in the art). Where specific details (e.g., circuits) are set forth to describe exemplary embodiments of the present disclosure, it will be apparent to those skilled in the art that one or more embodiments of the present application can be implemented without these specific details or with changes in these specific details. Therefore, these descriptions should be considered illustrative rather than restrictive.

[0134] Although the present disclosure has been described in conjunction with specific embodiments thereof, many alternatives, modifications, and variations of these embodiments will be apparent to those skilled in the art based on the foregoing description. For example, other memory architectures (e.g., dynamic RAM (DRAM)) may use the embodiments discussed.

[0135] The one or more embodiments of the present application are intended to encompass all such substitutions, modifications, and variations that fall within the broad scope of the appended claims. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the one or more embodiments of the present application should be included within the scope of protection of this disclosure.

Claims

1. A method for processing image data, characterized in that: include: Get image dataset; executing a data preprocessing process based on the image dataset, wherein the data preprocessing process comprises: calculating, for each pixel point in the image dataset, a weighted median of pixel values ​​within a preset region window centered on each pixel point, and then updating the weighted median to a pixel value of the corresponding pixel point; terminating the data preprocessing process in response to reaching a preset termination condition, thereby updating the image dataset to obtain an updated image dataset; Randomly extracting a deep neural network model from a preset bagged deep neural network model set as a first deep neural network model, and inputting the updated image dataset into the first deep neural network model to obtain a first prediction category and a first confidence level; Determining the other deep neural network models in the preset bagged deep neural network model set except the first deep neural network model as second deep neural network models, and then inputting the updated image dataset into the second deep neural network model to obtain each second prediction category and each second confidence level; In response to the first prediction category being different from the respective second prediction categories and the first confidence being different from the respective second confidences, the image dataset is determined to be an abnormal image dataset and output; or in response to the respective second prediction categories being the same as the first prediction category and the differences between the first confidence and the respective second confidences exceeding a threshold, the image dataset is determined to be an abnormal image dataset and output.

2. The method according to claim 1, characterized in that The preset termination conditions include: At least one of the following is achieved: a threshold of the number of repeated iterations is reached, and a pixel value of the pixel does not change during the iteration process.

3. The method according to claim 1, characterized in that The architectural difference between each deep neural network model in the preset bagged deep neural network model set is greater than a preset threshold.

4. The method according to claim 1, wherein After randomly extracting a deep neural network model from the preset bagged deep neural network model set as the first deep neural network model, the method further includes: The randomly selected neuron parameters and the output of the activation function layer of the first deep neural network model are randomly perturbed to the same extent.

5. The method according to claim 1, wherein Before randomly extracting a deep neural network model from the preset bagged deep neural network model set, the method further includes: Obtain a training set, the training set comprising images stored in pixel-level matrix data; The training set is divided into ten equal parts, with each part containing the same proportion of each image category; For each deep neural network model in the bagged deep neural network model set, nine of the ten equally divided training sets are taken to constitute the training set of each deep neural network model, and then each deep neural network model is trained using the projected gradient descent method to obtain the optimal hyperparameters of each deep neural network model and various parameters in the model, and then obtain the final trained preset bagged deep neural network model set.

6. An image data processing device, characterized in that: include: an acquisition unit configured to acquire an image dataset; a data preprocessing unit configured to perform a data preprocessing process based on the image dataset, wherein the data preprocessing process includes: calculating, for each pixel point in the image dataset, a weighted median of pixel values ​​within a preset region window centered on each pixel point, and then updating the weighted median to the pixel value of the corresponding pixel point; terminating the data preprocessing process in response to reaching a preset termination condition, and then updating the image dataset to obtain an updated image dataset; A first prediction unit is configured to randomly extract a deep neural network model from a preset bagged deep neural network model set as a first deep neural network model, and input the updated image dataset into the first deep neural network model to obtain a first prediction category and a first confidence level; a second prediction unit, configured to determine other deep neural network models in the preset bagged deep neural network model set except the first deep neural network model as second deep neural network models, and then input the updated image dataset into the second deep neural network model to obtain respective second prediction categories and respective second confidence levels; An abnormal data set determination unit is configured to, in response to the first prediction category being different from the respective second prediction categories and the first confidence being different from the respective second confidences, determine that the image data set is an abnormal image data set and output it; or, in response to the respective second prediction categories being the same as the first prediction category and the differences between the first confidence and the respective second confidences exceeding a threshold, determine that the image data set is an abnormal image data set and output it.

7. The device according to claim 6, characterized in that The preset termination conditions include: At least one of the following is achieved: a threshold of the number of repeated iterations is reached, and a pixel value of the pixel does not change during the iteration process.

8. The device according to claim 6, characterized in that The architectural difference between each deep neural network model in the preset bagged deep neural network model set is greater than a preset threshold.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the method according to any one of claims 1 to 5 is implemented.

10. A non-transitory computer-readable storage medium, characterized in that The non-transitory computer-readable storage medium stores computer instructions, and the computer instructions are used to cause the computer to execute the method according to any one of claims 1 to 5.

Images