An adversarial sample attack pooling method and system for deep neural network image classification tasks

Abstract

The application belongs to the technical field of deep neural network, and discloses an anti-adversarial sample attack pooling method for a deep neural network image classification task.The application distinguishes the importance of multiple feature mappings of adjacent feature mean values based on a statistical principle, and mixes maximum feature mappings for weighting, so that the convolution network not only reduces the cost of weight update in the error back propagation process, but also has good interpretability.The application further analyzes the role of the pooling operation in the robustness of adversarial sample attack.The experimental results show that in the image classification task, the pooling method proposed by the application can effectively improve the robustness against adversarial sample attack on the Cifar100 and Cifar10 data sets under the premise of ensuring the network prediction effect.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of deep neural networks, and particularly relates to an anti-adversarial sample attack pooling method and system for a deep neural network image classification task. BACKGROUND

[0002] Deep neural networks can automatically extract effective features of targets through layer-by-layer abstraction, and have been widely applied to research fields such as computer vision, speech recognition and natural language processing, and have achieved remarkable results. Convolutional neural networks are a special structure of deep neural networks, and are often used for target recognition, image classification and other related tasks. The pooling operation in the convolutional neural network is a key step to reduce the dimension of features. Through the pooling operation, the number of parameter or weight updates can be reduced, and the calculation cost can be reduced. According to the update mode of the convolutional weight by the pooling operation, the pooling operation can be divided into three categories: (1) updating all weight parameters, such as average pooling (Average Pooling), rank-based average pooling (Rank-Based Average Pooling) and weighted pooling (Weighted Pooling); (2) updating a single weight parameter, such as maximum pooling (MaxPooling), row-wise maximum pooling (Row-wise MaxPooling), global maximum pooling (Global MaxPooling) and stochastic pooling (Stochastic Pooling); (3) updating part of the weight parameters, such as mixed pooling (Mixed Pooling) and approximate median pooling (Approximate Median Pooling). Among them, the two most commonly used pooling methods are maximum pooling and average pooling.

[0003] Although the pooling operation is simple and effective in aggregating local features of an image, in previous studies, the role of the pooling operation in weight updating of the convolutional network and the robustness of the network has not been fully analyzed. The approximate median pooling proposed in the literature is inspired by human vision in extracting image features, that is, color blocks in the boundary transition region are more easily perceived, so the average value of multiple feature maps close to the median of the pooling is taken as the result of the pooling, and the role of the pooling operation in weight updating is discussed. However, the approximate median pooling does not distinguish the importance of multiple feature maps adjacent to the median of the pooling in image recognition. Considering the important role of the maximum feature map in image recognition, the present application proposes a lightweight pooling method based on statistical principles, called adjacent median mixed weighted pooling (AMMW Pooling, AdjacentMedianMixed WeightedPooling) based on the literature. The pooling method can reduce the number of weights that need to be updated during the learning process of the convolutional neural network, the update strategy has good interpretability, and also enhances the robustness of the convolutional neural network under adversarial sample attack.

[0004] Through the above analysis, the problems and defects of the prior art are:

[0005] Convolutional neural networks are often used for various image processing tasks, and pooling operation is one of the important components of the network. Although the pooling operation is simple and effective in aggregating local features of an image, the influence of the pooling operation on the number of weight updates during network training and the robustness of the model when facing adversarial sample attack has not been fully studied. SUMMARY

[0006] In view of the problems existing in the prior art, the present application provides an anti-adversarial sample attack pooling method for deep neural network image classification tasks.

[0007] The present application is implemented as follows: an anti-adversarial sample attack pooling method for deep neural network image classification tasks comprises:

[0008] S101 step, adjacent feature mapping median weighting strategy, the step comprises: adjacent feature mapping of the original signal, observing multiple feature values in the adjacent region of each data point in the signal, using the median as a measure, and setting a weighting coefficient according to the statistical distribution to reduce the influence of noise and outliers, so as to ensure that the extracted features are more representative and stable in the local range;

[0009] Step S102, effective feature selection, the step includes: performing statistical analysis on the mapping features obtained in step S101, using indicators such as data distribution, correlation analysis and variance to screen out redundant and noisy data, and retaining only the effective features that play a key role in the overall signal recognition and classification.

[0010] Step S103, Feature weight calculation, the steps include: quantifying and evaluating the effective features selected in step S102 according to statistical principles, calculating the weight of each feature, and converting the contribution of each feature into a numerical representation through normalization and weighted summation methods.

[0011] Step S104, weighted pooling of the maximum hybrid feature, includes: on the basis of the weights calculated in step S103, a hybrid weighted pooling strategy is adopted to combine the maximum feature in the local region with the overall weighted summation result, so as to extract the most significant local feature information while retaining the overall statistical information, thereby achieving efficient feature fusion and reducing data dimensionality.

[0012] Furthermore, the weighting strategy for the median of the neighbor feature map:

[0013] It includes two steps: effective feature selection and feature weight calculation. Effective feature selection is used to determine which features participate in the pooling operation, and feature weight calculation will calculate the weight values ​​of these features during the model training process.

[0014] Furthermore, the selection of effective features:

[0015] The value of the p-th neuron in the (l+1)-th layer (the current pooling layer) corresponds to a U×U feature map region of size in the l-th layer obtained after convolution extraction. The median of these U×U feature maps is quickly found using the "minimum contrast sum" method. p The pooling operation expands outwards from the median value, selecting features that are close to the median value for weighted pooling. Features that are not selected are considered to have a pooling weight of 0.

[0016] Furthermore, the feature mapping region of the U×U is selected to include median p Inside and near the median p We perform weighted pooling on n (n≤U×U) feature maps, where each n feature map is pooled with respect to the median... p Arranged in ascending order of value around the center, the leftmost feature map has the smallest value, and the rightmost feature map has the largest value; these n feature maps will be divided into three parts: the blue squares represent the parts including the median p Within this range, the feature maps whose values ​​are closest to medianp are m1 in number; orange squares represent the next closest median. pcharacteristics mapping, the number of which is m2; the remaining green squares represent the characteristics mapping with values at both ends, the number of which is m3; the number distribution of the three parts is calculated in the manner of m3 = n - (m1 + m2); according to the parity of the size of the pooling kernel, the corresponding median p may be 1 (UxU takes an odd number) or 2 (UxU takes an even number), in order to make the median p The number of characteristics mapping participating in the pooling on both sides is the same, and the number of characteristics mapping participating in the pooling is combined with the parity of the selected characteristics mapping n, so that the number of m1, m2 and m3 is slightly adjusted in the actual distribution process.

[0017] Further, the feature weight calculation is:

[0018] The selection method of the effective characteristics mapping, the n characteristics mapping to be weighted is divided into three parts according to the distance from the median p of the characteristics mapping, the characteristics mapping closer to the median p plays a greater role in the network function, so according to the "3σ" principle of normal distribution, the characteristics in the three parts are weighted; the role of each neuron in each part is not much different, so the strategy of equally distributing the weight is adopted; specifically, the pooling weight corresponding to each characteristics mapping in the middle part is

[0019]

[0020] The pooling weight corresponding to each characteristics mapping in the second part is

[0021]

[0022] The pooling weight corresponding to each characteristics mapping in the most two sides is

[0023]

[0024] In the model training process, the error δ(l+1, p) needs to be back propagated, at this time, the pooling weight calculated in formulas (1) to (3) is distributed to the n neurons in the corresponding area of the previous layer, and the corresponding gradient is calculated, and the update of the convolution kernel is calculated.

[0025] Further, the weighted pooling of mixed maximum characteristics is:

[0026] Considering that the maximum characteristics mapping contains the texture information in the picture, therefore, in addition to selecting the n characteristics mapping near the median, the maximum characteristics mapping is also integrated to participate in the pooling operation; in the pooling process, the weight of the maximum characteristics mapping is denoted as w max , at this time, the remaining n adjacent median pThe total full weight of the feature mapping is 1-w max The weights of the remaining three parts are calculated according to formulas (4)-(6), and this pooling operation is called mixed weighted pooling of adjacent feature mapping median values.

[0027]

[0028] In the back propagation process of the error delta (l+1, p), the pooling weights calculated in formulas (4)-(6) and w max are distributed to n+1 neurons in the corresponding area of the previous layer, and the corresponding gradient is calculated to update the corresponding convolution kernel.

[0029] Another object of the present application is to provide a lightweight mixed weighted pooling system based on statistical principles, comprising:

[0030] A weighting strategy module for the weighting strategy of adjacent feature mapping median values.

[0031] A feature selection module for effective feature selection.

[0032] A calculation module for feature weight calculation.

[0033] A weighted pooling module for mixed maximum feature weighted pooling.

[0034] Another object of the present application is to provide a computer device comprising a memory and a processor, the memory storing a computer program, the computer program being executed by the processor to make the processor execute the steps of the anti-adversarial sample attack pooling method for deep neural network image classification tasks.

[0035] Another object of the present application is to provide a computer readable storage medium storing a computer program, the computer program being executed by a processor to make the processor execute the steps of the anti-adversarial sample attack pooling method for deep neural network image classification tasks.

[0036] Another object of the present application is to provide an information data processing terminal for implementing the lightweight mixed weighted pooling system based on statistical principles.

[0037] In combination with the above technical solutions and the technical problems solved, the technical solution to be protected by the present application has the following advantages and positive effects:

[0038] First, the application effectively solves the problems of serious signal data noise interference and inaccurate feature extraction in the prior art by using an anti-adversarial sample attack pooling method for deep neural network image classification tasks, and improves the stability and reliability of data processing in industrial applications.

[0039] The application aims at the problem that the traditional method fails to effectively eliminate noise and outliers in the data preprocessing stage, leading to inaccurate feature extraction, and realizes smooth processing of local features by using a neighboring feature mapping median weighting strategy, reduces the influence of noise on signal analysis, and thus improves the accuracy of overall signal recognition.

[0040] In industrial real-time monitoring and large-scale data processing scenarios, the traditional technology is difficult to meet the needs due to slow processing speed and high consumption of computing resources. The application simplifies the data dimension through effective feature selection and feature weight calculation, and significantly reduces the system operation complexity by using a hybrid weighted pooling strategy, thereby improving the data processing efficiency and providing technical support for real-time applications.

[0041] In addition, the application realizes efficient integration of information by using a hybrid pooling method that fuses local maximum features and overall weighted sum results, so that key features can be quickly and accurately captured in actual industrial applications, effectively supporting intelligent judgment and response of the decision system in complex scenarios, thereby solving the problems of response delay, resource waste and misjudgment risk in the prior art in industrial applications.

[0042] The application method distinguishes the importance of multiple feature mappings of the neighboring feature median based on statistical principles, and mixes the maximum feature mapping for weighting, so that the convolution network not only reduces the cost of weight update in the error backpropagation process, but also has good interpretability. The application also analyzes the role of the pooling operation in the robustness of adversarial sample attacks. Experimental results show that in the image classification task, the pooling method proposed by the application can effectively improve the robustness against adversarial sample attacks on Cifar100 and Cifar10 data sets while ensuring the network prediction effect.

[0043] The application overcomes the deficiency of the traditional pooling method that does not utilize the statistical regularity among features, and through dynamic adjustment of the feature mappings participating in the pooling, the number of network updated parameters is significantly reduced, the training efficiency of the network is improved, the prediction accuracy of the network in ordinary scenarios is improved, and the network robustness in adversarial sample attack scenarios is improved.

[0044] Secondly, the anti-adversarial sample attack pooling method for deep neural network image classification tasks is proposed to solve the problems of low signal data processing efficiency and inaccurate feature extraction in the prior art. Through steps such as adjacent feature mapping median weighting, effective feature selection and feature weight calculation, the noise influence is effectively suppressed, and the key information in the signal is fully preserved, thereby solving the problems of feature redundancy and noise interference in the traditional method.

[0045] In industrial applications, especially for large-scale real-time signal processing scenarios, data processing speed and accuracy are crucial. The technical scheme uses a hybrid maximum feature weighted pooling strategy to extract local significant features while also integrating overall statistical information, achieving data dimension reduction and efficient fusion. This method not only significantly improves signal processing speed, but also significantly improves data accuracy and stability, meeting the high requirements of industrial application scenarios.

[0046] Compared with the prior art, the scheme of the present application has obvious reduction in hardware resource consumption and computational complexity, with the advantage of lightweight, especially suitable for real-time signal processing requirements in embedded devices and mobile terminals. By reducing data redundancy and noise interference, the system can respond and process input signals faster, greatly improving the overall performance and service life of the device, providing a cost-effective solution for industrial applications.

[0047] In summary, the technical scheme of the present application not only effectively solves the shortcomings of the prior art in feature extraction, data noise reduction, processing efficiency and hardware resource utilization, but also achieves double improvement in data processing quality and real-time response capability in industrial applications. This anti-adversarial sample attack pooling method for deep neural network image classification tasks brings significant technological progress to the field of signal data processing, promoting the development of related industries in the direction of intelligentization and precision. BRIEF DESCRIPTION OF DRAWINGS

[0048] Figure 1 is the flow chart of the anti-adversarial sample attack pooling method for deep neural network image classification tasks provided by the embodiment of the present application.

[0049] Figure 2 is the lightweight hybrid weighted pooling system structure block diagram based on statistical principles provided by the embodiment of the present application.

[0050] Figure 3 is the effective feature selection and quantity allocation schematic diagram provided by the embodiment of the present application.

[0051] Figure 4 is the prediction accuracy mean graph of ResNet-20 on Cifar100 with the same pooling with weight update quantity n change provided by the embodiment of the present application.

[0052] Figure 5 is a prediction accuracy graph of ResNet-20 on Cifar100 for different pooling provided by an embodiment of the present application.

[0053] Figure 6 is a prediction accuracy graph of ResNet-20 on Cifar10 for different pooling provided by an embodiment of the present application.

[0054] Figure 7 is a prediction accuracy mean graph of ResNet-20 on Cifar10 for the same pooling with the number of weight updates n provided by an embodiment of the present application. DETAILED DESCRIPTION

[0055] In order to make the purpose, technical scheme and advantages of the present application clearer, the present application will be further described in detail below with embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and do not limit the present application.

[0056] As shown in Figure 1 , the anti-adversarial sample attack pooling method for deep neural network image classification tasks provided by an embodiment of the present application includes the following steps:

[0057] S101, a weighting strategy of the median of adjacent feature mapping;

[0058] S102, effective feature selection;

[0059] S103, feature weight calculation;

[0060] S104, weighted pooling of mixed maximum features.

[0061] (S101, a weighting strategy of the median of adjacent feature mapping):

[0062] In the initial stage of signal data processing, first, the original signal is subjected to adjacent feature mapping, and a median weighting strategy is adopted. Specifically, for each data point in the signal, the system will examine a plurality of feature values in the adjacent region thereof, use the median as a measurement index, and set a weighting coefficient according to the statistical distribution to reduce the influence of noise and outliers, so as to ensure that the extracted features are more representative and stable in the local range.

[0063] (S102, effective feature selection):

[0064] After the adjacent weighting processing, the system preliminarily screens all the mapped features, and evaluates the features by statistical analysis method. Redundant and noise data are screened out by using data distribution, correlation analysis and variance indicators, and only effective features which play a key role in the overall signal identification and classification are reserved. This step lays a solid foundation for subsequent feature weight calculation, and ensures that the main information is highlighted in the processing process.

[0065] (S103 feature weight calculation):

[0066] After selecting the effective features, the system quantitatively evaluates the importance of each feature according to statistical principles, and calculates the weight of each feature. By normalization processing and weighted summation method, the contribution degree of each feature is converted into numerical representation, so that the feature can affect the final result according to its weight in subsequent mixed pooling. This weight distribution mechanism based on statistical data effectively reflects the relative importance of each feature in the overall signal, and enhances the discrimination ability of the model.

[0067] (S104 mixed maximum feature weighted pooling):

[0068] Finally, the system adopts a mixed weighted pooling strategy to combine the maximum feature in the local region with the overall weighted summation result on the basis of the calculated weight. This step can not only extract the most significant local feature information, but also retain the overall statistical information, realizing efficient fusion of features. Through this mixed way, the system not only improves the response ability to key features, but also reduces the data dimension, ensuring that the entire signal processing process is lightweight and has high accuracy.

[0069] The weighted strategy of the median of the adjacent feature mapping provided by the embodiment of the application:

[0070] It includes two steps of effective feature selection and feature weight calculation. The effective feature selection is used to determine which features participate in the pooling operation, and the feature weight calculation calculates the weight value corresponding to these features in the model training process.

[0071] The effective feature selection provided by the embodiment of the application:

[0072] The value of the pth neuron of the l+1 layer, i.e. the current pooling layer, corresponds to a corresponding feature mapping region of the l layer with a size of UxU obtained after convolution extraction; for the UxU feature mapping, the "minimum contrast sum" method is used to quickly find the median p of the feature mapping, and the selected part of the features adjacent to the median is expanded to both sides with the median as the center to perform weighted pooling operation, and the pooling weight of the features not selected is regarded as 0.

[0073] The UxU feature mapping area provided by the embodiment of the application selects the median p The n(n≤UxU) feature mappings adjacent to the median p are weighted and pooled, the n feature mappings are arranged in order according to the value size with the median p at the center, the leftmost feature mapping is the smallest, and the rightmost feature mapping is the largest; the n feature mappings are divided into three parts: the blue square represents the feature mapping including the median p , the value of which is closest to the median p , and the number of the feature mappings is m1; the orange square represents the feature mapping which is second closest to the median p , and the number of the feature mappings is m2; the remaining green square represents the feature mapping with the value at both ends, and the number of the feature mappings is m3; the number distribution of the three parts is calculated in the manner of m3=n-(m1+m2); according to the parity of the size of the pooling kernel, the corresponding median p may be 1 (UxU takes an odd number) or 2 (UxU takes an even number), in order to make the number of feature mappings participating in the pooling on both sides of the median p the same, and in combination with the parity of the number n of selected feature mappings, the number of m1, m2 and m3 will be slightly adjusted in the actual distribution process.

[0074] The feature weight calculation provided by the embodiment of the application is as follows:

[0075] The selection method of the effective feature mapping divides the n feature mappings needing weighting into three parts according to the distance from the median p of the feature mapping, the feature mapping closer to the median p plays a greater role in the network function, therefore, according to the “3σ” principle of the normal distribution, the feature in the three parts is weighted and distributed; the role of each neuron in each part is not much different, and therefore the strategy of average weight distribution is adopted; specifically, the pooling weight corresponding to each feature mapping in the middle part is

[0076]

[0077] The pooling weight corresponding to each feature mapping in the second part is

[0078]

[0079] The pooling weight corresponding to each feature mapping in the most two sides part is

[0080]

[0081] In the model training process, the error δ(l+1, p) needs to be back propagated, at this time, the calculated pooling weight in formulas (1)-(3) is distributed to n neurons in the corresponding region of the previous layer, and the corresponding gradient is calculated to update the convolution kernel.

[0082] The mixed maximum feature weighting pooling provided by the embodiment of the application has the following advantages:

[0083] Considering that the maximum feature mapping contains the texture information in the picture, in addition to selecting n feature mappings with median values in the adjacent feature mappings, the maximum feature mapping is also integrated to participate in the pooling operation; in the pooling process, the weight of the maximum feature mapping is denoted as w max At this time, the total weight of the remaining n adjacent median p feature mappings is 1-w max The weights of the remaining three parts are calculated according to formulas (4)-(6), and this pooling operation is referred to as mixed weighting pooling of median values in adjacent feature mappings.

[0084]

[0085] In the back propagation process of the error δ(l+1, p), the calculated pooling weights in formulas (4)-(6) and w max are distributed to n+1 neurons in the corresponding region of the previous layer, and the corresponding gradient is calculated to update the corresponding convolution kernel.

[0086] As shown in Figure 2 , the lightweight mixed weighting pooling system based on the statistical principle provided by the embodiment of the application comprises:

[0087] A weighting strategy module is configured to perform a weighting strategy of median values in adjacent feature mappings.

[0088] A feature selection module is configured to perform effective feature selection.

[0089] A calculation module is configured to calculate feature weights.

[0090] A weighted pooling module is configured to perform mixed maximum feature weighting pooling.

[0091] Another purpose of the application is to provide a computer device comprising a memory and a processor, wherein the memory stores a computer program, and the computer program is executed by the processor to enable the processor to perform the steps of the anti-adversarial sample attack pooling method for deep neural network image classification tasks.

[0092] Another object of the present application is to provide a computer readable storage medium storing a computer program, which, when executed by a processor, causes the processor to perform the steps of the method for adversarial sample attack pooling for deep neural network image classification tasks.

[0093] Another object of the present application is to provide an information data processing terminal for implementing the lightweight hybrid weighted pooling system based on statistical principles.

[0094] The present application is embodied as follows:

[0095] 1. Pooling operation of convolutional neural network

[0096] 1.1. Convolutional neural network structure

[0097] A convolutional neural network is a kind of deep feedforward neural network. A typical convolutional neural network is generally stacked by convolutional layers, pooling layers and fully connected layers. In image processing, the role of the convolutional layer is to extract the features of the local region of the image. The pooling layer is added after the convolutional layer, and through feature selection, the size of the image features can be effectively reduced, and the number of model parameters can be reduced. In addition, the pooling layer uses the overall statistical features of the adjacent region at a certain position to replace the specific content at that position, so that the convolutional neural network can remain unchanged for small local shape changes, while having a larger visual receptive field. The features after convolution and pooling operations will be input into the fully connected layer to realize image classification and other tasks. The current typical deep convolutional neural network structure includes LeNet-5, Inception, ResNet, etc.

[0098] 1.2. Characteristics of different pooling operations

[0099] During the back propagation of error, different pooling functions have different number of weight updates of the last layer of convolution kernel. Convolutional neural network calculates the gradient of parameters by back propagation of error term of each layer, and realizes parameter update. Assuming that the (l+1)th layer is a pooling layer, the lth layer is a convolution layer, and the size of the convolution kernel is UxV, since the pooling layer adopts down-sampling operation, the error term δ(l+1,p) of the pth neuron of the (l+1)th layer corresponds to a region of the corresponding feature mapping of the lth layer. If the maximum pooling is adopted, the value of the error term δ(l+1,p) is directly transmitted to the neuron corresponding to the maximum value in the corresponding region of the last layer, and the error terms of other neurons in the region are all set to 0, at this time, the number of weight updates of the convolution kernel is 1. If the average pooling is adopted, each value of the error term δ(l+1,p) is evenly distributed to all neurons in the corresponding region of the last layer, at this time, the number of weight updates of the convolution kernel is UxV. The approximate median pooling takes the average value close to multiple feature mappings as the pooling result, so the value of the error term δ(l+1,p) is evenly distributed to the corresponding part of neurons in the last layer, at this time, the number of weight updates of the convolution kernel is the same as the number of feature mappings close to the median of the feature in the pooling operation. The advantage of the maximum pooling is that the number of weight updates of the convolution kernel is small, but since most of the convolution kernels do not participate in the update, the prediction accuracy of the network will be reduced; since all the weight of the convolution kernel needs to participate in the update, although the average pooling can improve the prediction accuracy of the whole network, it also increases the scale of the network parameter update; the approximate median pooling only needs to update part of the weight of the convolution kernel, but it does not distinguish the importance of multiple feature mappings close to the median of the adjacent pooling in image recognition. In view of the above problems, the importance of the feature mapping is distinguished, and an anti-adversarial sample attack pooling method for deep neural network image classification task is designed.

[0100] 2 Hybrid weighted pooling method of adjacent feature mapping median

[0101] 2.1 Statistical principle

[0102] An image is composed of a number of pixels, each of which is independent of the others, and a single pixel has little influence on the recognition of the image content. After feature extraction via a small-size convolution kernel, the number of features obtained is still large and independent of each other, and a single feature has little influence on the recognition of the image. Therefore, we can consider that the feature mapping obtained after the image is extracted by the convolution layer is approximately subject to a normal distribution in statistics. In statistics, a random variable X subject to a normal distribution satisfies P(|X-μ|<xσ)=2Φ(x)-1, where μ is the mathematical expectation, σ is the standard deviation, and Φ(x) is the distribution function of the standard normal distribution. When σ is 1, 2, and 3, the corresponding probability values are 0.6826, 0.9544, and 0.9974, respectively. These probability values show that the random variable subject to a normal distribution has nearly 70% of the values concentrated within the σ range from the center μ, and it is almost impossible to deviate from the center μ by more than 3σ. This characteristic of the normal distribution is referred to as the “3σ” principle in statistics. Considering that the median value of the feature mapping is used as the approximate mean μ in the pooling layer, the weighted pooling operation based on the “3σ” principle is performed on the part of the feature mapping adjacent to the median value, so that the convolution network can reduce the update scale of the model parameters during the training process, the update strategy has certain interpretability, and the prediction accuracy is maintained.

[0103] 2.2 Weighted strategy of median value of adjacent feature mapping

[0104] Considering that the importance of the selected feature mapping is related to the size of the distance from the median value of the feature mapping, a weighted strategy of the median value of the adjacent feature mapping is proposed based on the “3σ” principle of the normal distribution. The strategy includes two steps of effective feature selection and feature weight calculation. The effective feature selection is used to determine which features participate in the pooling operation, and the feature weight calculation calculates the weight values of these features corresponding to the model training process. The following assumes that the l+1 layer is a pooling layer, the l layer is a convolution layer, and the size of the pooling kernel is UxU.

[0105] 2.2.1 Effective feature selection

[0106] The value of the pth neuron of the l+1 layer, i.e., the current pooling layer, corresponds to a corresponding feature mapping region of the l layer with a size of UxU obtained after convolution extraction. For the UxU feature mappings, the “minimum contrast sum” method is used to quickly find the median value median p and expand to both sides with the median value as the center, select the features adjacent to the median value for weighted pooling operation, and the pooling weight of the features not selected is considered to be 0.

[0107] For the above UxU feature mapping region, the median value median pInside and near the median p The n (n≤U×U) feature maps are weighted pooled, such as Figure 3 As shown, these n feature maps are in the form of median p Arranged in ascending order of value around the center, the leftmost feature map has the smallest value, and the rightmost feature map has the largest value. These n feature maps will be divided into three parts: the blue squares represent the parts including the median. p Within this range, the value closest to the median is taken. p The feature maps have a quantity of m1; orange squares represent the next nearest median. p The feature maps, numbered m2; the remaining green squares represent feature maps with values ​​at both ends, numbered m3. The distribution of the three parts follows the... The calculation is performed using the formula m3 = n - (m1 + m2). The corresponding median value is determined based on the parity of the pooling kernel size. p It can be 1 (U×U takes an odd number) or 2 (U×U takes an even number), in order to make the median p The number of feature maps participating in pooling is the same on both sides. In combination with the parity of the selected number of feature maps n, the number of m1, m2, and m3 will be slightly adjusted during the actual allocation process.

[0108] 2.2.2 Feature Weight Calculation

[0109] Following the selection method for effective feature maps in 2.2.1, the n feature maps that need to be weighted are selected according to the median distance from the median feature map. p The size is divided into three parts, closer to the median p The greater the contribution of the feature maps in the middle part to the network function, the greater the weighting effect. Therefore, the weights of these three parts are assigned according to the "3σ" principle of normal distribution. Since the contribution of each neuron in each part is not significantly different, an average weighting strategy is adopted. Specifically, the pooling weight corresponding to each feature map in the middle part is...

[0110]

[0111] The pooling weights corresponding to each feature map in the second part are:

[0112]

[0113] The pooling weights corresponding to each feature map in the two outermost parts are:

[0114]

[0115] In the model training process, the error δ(l+1, p) needs to be back propagated, at this time, the pooling weight calculated in (1)-(3) is distributed to the n neurons in the corresponding region of the previous layer, and the corresponding gradient is calculated, so as to realize the update of the convolution kernel.

[0116] 2.3 Mixed weighted pooling of maximum features

[0117] Considering that the maximum feature map contains the texture information in the picture, in addition to selecting the n feature maps of the median of the adjacent feature maps, the maximum feature map is also integrated to participate in the pooling operation. In the pooling process, the weight of the maximum feature map is w max , at this time, the total weight of the remaining n adjacent median p feature maps is 1-w max , and the weights of the remaining three parts are calculated according to (4)-(6). This pooling operation is called mixed weighted pooling of the median of adjacent feature maps.

[0118]

[0119] In the back propagation process of the error δ(l+1, p), according to the pooling weights calculated in (4)-(6) and w max are distributed to the n+1 neurons in the corresponding region of the previous layer, and the corresponding gradient is calculated, so as to realize the update of the corresponding convolution kernel.

[0120] The weighted pooling proposed in the application operates on the n feature maps of the median of the adjacent feature maps selected in the corresponding region, so in the parameter learning process of the convolutional neural network, each value of the back propagated error δ(l+1, p) will be distributed to the n neurons in the corresponding region of the previous layer according to the weight of the pooling kernel, and the convolution kernel at the corresponding position is updated. When n is compared with UxU, the number of weight updates of this pooling operation can be significantly reduced compared with average pooling, which can effectively improve the efficiency of the algorithm, and is a lightweight weight update pooling method.

[0121] 3 Experiment and analysis

[0122] 3.1 Experimental setup

[0123] The experiment is carried out on the Cifar10 and Cifar100 standard data sets. Before model training, the same data enhancement operation is performed on all training data. First, 4 pixels are padded on each side of the picture, then random cropping is performed to obtain a picture with a size of 32x32, and finally the picture is randomly flipped to obtain an expanded data set.

[0124] For comparison with related literature, ResNet-20 residual network is selected. In the experiment, the size of the pooling kernel is 8x8, so the median of the corresponding feature map p There are 2. Considering the selected adjacent median p If the number of feature maps n is too small, it is not conducive to use the 3σ rule based on normal distribution. In the experiment, n=8, 16, 32 are taken respectively, and the number of feature maps of the three parts is allocated and the corresponding pooling weight is calculated according to the method in 2.2, as shown in Table 1.

[0125] Table 1 Allocation of feature map number and pooling weight

[0126]

[0127] Each group of n values is based on the running of ResNet-20 residual network for 8 times, and the prediction top-1 accuracy (Top-1 Accuracy) and top-5 accuracy (Top-5 Accuracy) are evaluated in the format of "mean±std". During the training process, the SGD optimizer is used, the initial learning rate is set to 0.1, and the learning rate is dynamically adjusted to 0.01 and 0.001 after the 100th epoch and the 175th epoch, respectively. When the epoch reaches 300, the network stops training.

[0128] For the mixed weighted pooling of the adjacent feature median proposed in the application, for each group of n values, different weight values w max are assigned to the maximum feature map The remaining are also adjusted accordingly In the experiment, w max =0.05, 0.1, 0.15 are taken respectively for comparison.

[0129] 3.2 Results and analysis

[0130] In the experiment, the pooling method proposed in the application is compared with the maximum pooling, average pooling and approximate median pooling. In order to illustrate the influence of the weighting strategy and the mixed maximum feature map strategy of the mixed weighted pooling of the adjacent feature median on the prediction accuracy, an ablation experiment is performed. Among them, "approximate median + maximum feature pooling" means adding maximum feature map in approximate median pooling and taking average weight pooling operation; "weighted pooling of adjacent feature median" means that the selected n feature maps of the adjacent feature map median are weighted without mixing the maximum feature map. The experimental results of the above six kinds of pooling are shown in Tables 2 and 3, and the numbers 8, 16, 32 in the brackets in the table represent the number of weight updates, 0.15, 0.1, 0.05 represent the weight of the maximum feature map selected when the accuracy of the pooling method is optimal, and the meaning in the subsequent table is the same.

[0131] Table 2 Prediction accuracy % of ResNet-20 with different pooling methods on Cifar100

[0132]

[0133] Note: Bold is the optimal result, underlined is the sub-optimal result, the same follows.

[0134] Table 3 Prediction accuracy % of ResNet-20 with different pooling methods on Cifar10

[0135]

[0136]

[0137] Experiments show that as n increases, the weight of the maximum feature mapping that makes the average prediction accuracy of the network reach the optimal maximum feature mapping is decreasing, which shows that when n is small, the maximum feature mapping plays a more obvious role, and with the addition of more feature information, the role of the maximum feature mapping is gradually weakened.

[0138] In order to intuitively analyze the influence of the pooling method and the number of weight updates n on the prediction accuracy of the network, the average prediction accuracy in Tables 2 and 3 is classified in the manner as shown in Figure 4 and Figure 7 The results of the six pooling methods running on the Cifar100 and Cifar10 data sets for 8 times are sorted from small to large, and the results are shown in Figure 5 and Figure 6 .

[0139] From Figure 4 it can be seen that for the Cifar100 data set, the first accuracy and the top 5 accuracy of the same kind of pooling have similar rules. For the approximate median pooling, the accuracy is improved with the increase of n. According to the approximate median + maximum feature pooling method, when n = 8, the network performance is most obviously improved compared with the approximate median pooling. For the weighted pooling of adjacent feature median, with the increase of n, the network performance is obviously improved gradually.

[0140] Figure 4 The experimental results of the approximate median + maximum feature pooling and the weighted pooling of adjacent feature median are similar to Figure 5The case is consistent, when n is small, the weighting based on the statistical principle is deviated from the assumption of normal distribution due to the small number of feature maps, and the addition of the texture information of the picture contained in the maximum feature map plays an important role in picture recognition; with the increase of n, the statistical distribution of the feature map is closer to the assumption of normal distribution, at this time the weighting based on the statistical principle plays an obvious role, on the other hand, since the average weight is used in the approximate median + maximum feature pooling, when the number of n increases, the weight allocated to the maximum feature value also decreases, and the role is also weakened.

[0141] Figure 5 The results show that for the Cifar100 data set, the prediction accuracy of the weighted pooling of the adjacent feature median and the mixed weighted pooling of the adjacent feature median is better than that of the approximate median pooling and significantly better than that of the maximum pooling, which shows that in the case of not changing or only increasing 1 weight update number, the addition of the weighting strategy based on the statistical principle and the mixed weighting strategy of the maximum feature map helps to improve the prediction performance of the network. When n = 8, the mixed weighted pooling of the adjacent feature median is closest to the average pooling, and at this time the number of updates of the convolution kernel in the back propagation process of the network is only 9, which is significantly reduced compared with the weight update number 64 of the average pooling, indicating that the weighting based on the statistical principle and the mixed maximum feature value pooling strategy have a positive effect on improving the network performance.

[0142] On the Cifar10 data set, since the other five kinds of pooling methods have a very obvious advantage over the maximum pooling in the prediction accuracy, in order to facilitate observation, Figure 6 The prediction results of the five kinds of pooling except the maximum pooling are given. Different from the test results of Cifar100, the weighted pooling of the adjacent feature median has the optimal prediction accuracy when n = 16, and for the mixed weighted pooling of the adjacent feature median with the maximum feature map, the accuracy is highest when n = 32, but the difference with n = 8 and n = 16 is not large, which may be related to the relatively small number of classifications of Cifar10.

[0143] From Figure 7 It can be found that the prediction accuracy of the four kinds of pooling types when n = 8 is relatively close, and the effect is almost the second best of this kind of pooling, and at this time the weight update number of the network convolution kernel is significantly reduced compared with the average pooling. Therefore, for the mixed weighted pooling of the adjacent feature median, n = 8 is still a good choice.

[0144] Table 4 Top 5 accuracy % of different poolings of ResNet-20 under FGSM, DI-FGSM, NI-FGSM attacks on Cifar100

[0145]

[0146] Table 5 Top-1 accuracy % of ResNet-20 with different pooling methods on Cifar10 under FGSM, DI-FGSM, NI-FGSM attack

[0147]

[0148]

[0149] The robustness of the above six kinds of pooling networks was compared by using FGSM, IN-FGSM and DI-FGSM adversarial sample attack methods. On Cifar100 and Cifar10, the number of weight updates n = 8 was a good choice, so n = 8 was selected for prediction, and the results are shown in Tables 4 and 5. The results show that the prediction accuracy of the pooling method proposed in the present application is the best or the second best under the three kinds of adversarial sample attack methods, and the stability is good, which shows that the pooling method proposed in the present application not only has good prediction performance, but also can enhance the robustness of the network.

[0150] Table 6 Prediction accuracy % of LeNet-5 with different pooling methods on Cifar10

[0151]

[0152] Table 7 Prediction accuracy % of ResNet-50 with different pooling methods on Cifar100

[0153]

[0154] Table 8 Top-1 accuracy % of LeNet-5 with different pooling methods on Cifar10 under adversarial sample attack

[0155]

[0156] Table 9 Top-5 accuracy % of ResNet-50 with different pooling methods on Cifar100 under adversarial sample attack

[0157]

[0158] To verify the universality of the pooling method proposed in the application in other convolutional neural networks, from the aspects of simplifying the model structure and deepening the model depth, the mixed weighted pooling of the median of adjacent features with n=8, the maximum pooling, the average pooling and the approximate median pooling based on LeNet-5 network and ResNet-50 network are compared on Cifar10 and Cifar100. The prediction accuracy experimental results are shown in Tables 6 and 7, and the robustness experimental results of the adversarial sample attack are shown in Tables 8 and 9. The results show that on LeNet-5 and ResNet-50, the mixed weighted pooling of the median of adjacent features shows the same performance as on ResNet-20, the prediction accuracy is better than that of the maximum pooling and the approximate median pooling, and the robustness under the adversarial sample attack is optimal or suboptimal. On the LeNet-5 network, the prediction accuracy and the adversarial robustness of the average pooling are obviously better than those of the mixed weighted pooling of the median of adjacent features, which is suboptimal. This may be related to the simple structure of the LeNet-5 network and the relatively small number of parameters. Therefore, the pooling method proposed in the application can obtain good experimental results on different convolutional neural networks, and the effect is more obvious on the network with complex structure and large number of parameters.

[0159] The application proposes a light-weight mixed weighted pooling method of convolutional neural network based on statistical principle. The method distinguishes the importance of multiple feature mappings of the median of adjacent features based on statistical principle, and mixes the maximum feature mapping for weighting, so that the convolutional network not only reduces the cost of weight update in the process of error back propagation, but also has good interpretability. The application further analyzes the role of the pooling operation in the robustness of the adversarial sample attack. The experimental results show that in the image classification task, the pooling method proposed in the application can effectively improve the robustness against adversarial sample attack on the premise of ensuring the network prediction effect on Cifar100 and Cifar10 data sets.

[0160] It should be noted that embodiments of the present application can be realized by hardware, software, or a combination of software and hardware. The hardware portion can be realized by a special logic; the software portion can be stored in a memory and executed by a proper instruction execution system, such as a microprocessor or a specially designed hardware. A person of ordinary skill in the art can understand that the above-mentioned apparatus and method can be realized by computer executable instructions and / or included in processor control codes, such as a carrier medium, such as a magnetic disk, CD or DVD-ROM, a programmable memory, such as a read-only memory (firmware), or a data carrier, such as an optical or electronic signal carrier. The apparatus of the present application and its modules can be realized by a hardware circuit, such as a very large scale integrated circuit or a gate array, a semiconductor, such as a logic chip, a transistor, or a programmable hardware device, such as a field programmable gate array, a programmable logic device, or the like, by software executed by various types of processors, or by a combination of the above-mentioned hardware circuit and software, such as firmware.

[0161] The above description is merely a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any modification, equivalent replacement, and improvement within the technical range disclosed by the present application, and within the spirit and principle of the present application, should be included in the protection scope of the present application.

Claims

1. An adversarial sample attack pooling method for a deep neural network image classification task, characterized in that, The method comprises the following steps: S101, a weighting strategy for a neighboring feature mapping median value, the step comprising: performing a neighboring feature mapping on an original signal, examining a plurality of feature values in a neighboring region of each data point in the signal, using a median value as a measurement index, and setting a weighting coefficient according to a statistical distribution to reduce the influence of noise and outliers, thereby ensuring that the extracted features are more representative and stable in a local range; S102, effective feature selection, the step comprising: performing statistical analysis on the mapping features obtained in the S101 step, using data distribution, correlation analysis and variance to filter out redundant and noisy data, and only retaining effective features that play a key role in overall signal identification and classification; S103, feature weight calculation, the step comprising: quantitatively evaluating the effective features selected in the S102 step according to statistical principles, calculating the weights of the features, and converting the contribution of each feature into a numerical value through normalization processing and weighted summation; S104, mixed maximum feature weighted pooling, the step comprising: using a mixed weighted pooling strategy based on the weights calculated in the S103 step, combining the maximum feature in a local region with the overall weighted summation result, thereby extracting the most significant local feature information and retaining overall statistical information, achieving efficient fusion of features and reducing data dimensions; The mixed maximum feature weighted pooling comprises: Considering that the maximum feature map contains the texture information in the image, the maximum feature map is also involved in the pooling operation in addition to the n feature maps selected from the median of the adjacent feature maps; in the pooling process, the weight of the maximum feature map is w max , and the total weight of the remaining n adjacent median p feature maps is 1-w max , and the weights of the remaining three parts are calculated according to formulas (4)-(6). This pooling operation is called mixed weighted pooling of the median of the adjacent feature maps. (4) (5) (6) In the back propagation process of error δ (l+1,p) , the pooling weights calculated in (4)~(6) are distributed to n+1 neurons in the corresponding region of the previous layer, and the corresponding gradient is calculated to update the corresponding convolution kernel. max ​​ 2. The adversarial sample attack pooling method for deep neural network image classification tasks of claim 1, wherein, The weighting strategy for a neighboring feature mapping median value comprises: The method comprises two steps of effective feature selection and feature weight calculation; The effective feature selection is used to determine which features participate in the pooling operation, and the feature weight calculation calculates the weight values of the features corresponding to the model training process.

3. The adversarial sample attack pooling method for deep neural network image classification tasks of claim 2, wherein, The effective feature selection comprises: The value of the pth neuron of the l+1th layer, i.e., the current pooling layer, corresponds to a corresponding feature mapping region of the lth layer with a size of UxU obtained after convolution extraction; For this UxU feature maps, use "min contrast sum" method to quickly find the median of feature maps p And expand to both sides with the median as the center, select the part of the features adjacent to the median to perform weighted pooling operation, and the pooling weight of the unselected features is considered as 0.

4. The adversarial sample attack pooling method for deep neural network image classification tasks of claim 3, wherein, The UxU feature map region is selected to include median p n feature maps adjacent to median p are weighted and pooled, where The n feature maps are arranged in order of value size with median p at the center, with the leftmost feature map being the smallest and the rightmost feature map being the largest. The n feature maps are divided into three parts: the blue squares represent feature maps including median p , with the number of feature maps closest to median p being m1; the orange squares represent feature maps next closest to median p , with the number of feature maps being m2; and the remaining green squares represent feature maps at both ends, with the number of feature maps being m3. The number distribution of the three parts is calculated according to , , According to the parity of the size of the pooling kernel, the corresponding median p is 1 when UxU takes an odd number, and 2 when UxU takes an even number. In order to make the number of feature maps participating in pooling on both sides of median p the same, and in combination with the parity of the number of selected feature maps n, the number of m1, m2, and m3 will be slightly adjusted in the actual distribution process.

5. The adversarial sample attack pooling method for deep neural network image classification tasks of claim 4, wherein, The feature weight calculation comprises: The selection method of effective feature mapping is to divide the n feature mappings that need to be weighted into three parts according to the size of the distance from the median value median of the feature mapping p The closer the feature mapping is to the median value median p , the greater the role it plays in the network function, so the weight of the three parts of the features is allocated according to the "3σ" principle of normal distribution; the role of each neuron in each part is not much different, so the strategy of equally allocating weights is adopted; specifically, the pooling weight corresponding to each feature mapping in the middle part is (1) The pooling weight corresponding to each feature mapping of the second part is (2) The pooling weight corresponding to each feature mapping of the most two sides is (3) In the model training process, the error δ (l+1,p) Back propagation is performed, at which time the pooling weights calculated in equations (1) to (3) are distributed to the n neurons in the corresponding region of the previous layer, and the corresponding gradient is calculated to update the convolution kernel.

6. A lightweight hybrid weighted pooling system based on statistical principles implementing the adversarial sample attack pooling method for deep neural network image classification tasks according to any one of claims 1-5, characterized in that, The lightweight mixed weighted pooling system based on statistical principles comprises: A weighting strategy module for a weighting strategy for a neighboring feature mapping median value; A feature selection module for effective feature selection; A calculation module for feature weight calculation; A weighted pooling module for mixed maximum feature weighted pooling.

7. A computer device, comprising: The computer device comprises a memory and a processor, the memory stores a computer program, and the computer program is executed by the processor to make the processor execute the steps of the anti-adversarial sample attack pooling method for deep neural network image classification tasks according to any one of claims 1-5.

8. A computer readable storage medium storing a computer program, the computer program being executed by a processor to make the processor execute the steps of the anti-adversarial sample attack pooling method for deep neural network image classification tasks according to any one of claims 1-5.

9. An information data processing terminal, characterized by The information data processing terminal is used to implement the lightweight mixed weighted pooling system based on statistical principles according to claim 6.

Images