Anti-adversarial sample attack pooling method and system for deep neural network image classification task

Abstract

The invention belongs to the technical field of deep neural networks, and discloses an anti-adversarial sample attack pooling method for a deep neural network image classification task, which is characterized in that the importance of a plurality of feature mappings of adjacent feature median values is distinguished based on a statistical principle, and the maximum feature mapping is mixed for weighting; the weight updating cost of the convolutional network is reduced in the back propagation process of the error, and meanwhile the method has good interpretability. The invention further deeply analyzes the effect of the pooling operation on the aspect of resisting sample attack robustness. Experimental results show that in an image classification task, the pooling method provided by the invention can effectively improve the robustness for resisting sample attacks on Cifar100 and Cifar10 data sets on the premise of ensuring a network prediction effect.

Description

Technical Field

[0001] The present invention belongs to the technical field of deep neural networks, and particularly relates to an anti-adversarial sample attack pooling method and system for deep neural network image classification tasks. Background Art

[0002] Deep neural networks can automatically extract effective features of targets through a layer-by-layer abstraction method, and have been widely applied to research fields such as computer vision, speech recognition, and natural language processing, achieving remarkable results. Convolutional neural networks are a special structure of deep neural networks and are often used in related tasks such as target recognition and image classification. The pooling operation in convolutional neural networks is a key step in reducing the dimensionality of features. Through the pooling operation, the number of parameter or weight updates can be reduced, and the computational cost can be lowered. According to the way of updating convolutional weights by the pooling operation, the pooling operation can be divided into three categories: (1) updating all weight parameters, such as average pooling (AveragePooling), rank-based average pooling (Rank-BasedAverage Pooling), weighted pooling (WeightedPooling); (2) updating a single weight parameter, such as max pooling (MaxPooling), row-wise max pooling (Row-wiseMaxPooling), global max pooling (GlobalMaxPooling), stochastic pooling (StochasticPooling); (3) updating some weight parameters, such as mixed pooling (MixedPooling), approximate median pooling (ApproximateMedianPooling). Among them, the two most commonly used pooling methods are max pooling and average pooling.

[0003] Although pooling operations are simple and effective in aggregating local features of images, in previous studies, the role of pooling operations in the weight update of convolutional networks and the robustness of the network has lacked sufficient analysis. The approximate median pooling proposed in the literature is inspired by human vision when extracting image features, that is, color blocks in the boundary transition region are more easily perceived. Therefore, the average value of multiple feature maps close to the pooling median is used as the result of pooling, and the role of pooling operations in weight update is discussed. However, the approximate median pooling does not distinguish the importance of multiple feature maps adjacent to the pooling median in image recognition. Considering the important role of the maximum feature map in image recognition, based on the literature, the present invention proposes a lightweight pooling method based on statistical principles, called Adjacent Median Mixed Weighted Pooling (AMMW Pooling). This pooling method enables the convolutional neural network to reduce the number of weights to be updated during the learning process, the update strategy has good interpretability, and at the same time enhances the robustness of the convolutional neural network under adversarial sample attacks.

[0004] Through the above analysis, the problems and defects existing in the prior art are as follows:

[0005] Convolutional neural networks are often used in various image processing tasks, and pooling operations are one of the important components of this network. Although pooling operations are simple and effective in aggregating local features of images, the impact of pooling operations on the number of weight updates during network training and the robustness of the model when facing adversarial sample attacks has not been fully studied. Summary of the Invention

[0006] Aiming at the problems existing in the prior art, the present invention provides an anti-adversarial sample attack pooling method for deep neural network image classification tasks.

[0007] The present invention is implemented as follows. An anti-adversarial sample attack pooling method for deep neural network image classification tasks includes:

[0008] Step S101, the weighted strategy of adjacent feature map median, the steps include: performing adjacent feature mapping on the original signal, examining multiple feature values in the adjacent region of each data point in the signal, using the median as a measurement index, and setting a weighted coefficient according to the statistical distribution to reduce the influence of noise and outliers, so as to ensure that the extracted features are more representative and stable within the local range;

[0009] Step S102, effective feature selection, which includes: performing statistical analysis on the mapped features obtained in Step S101, screening out redundant and noisy data using indicators such as data distribution, correlation analysis, and variance, and only retaining the effective features that play a key role in overall signal recognition and classification.

[0010] Step S103, feature weight calculation, which includes: quantitatively evaluating the effective features selected in Step S102 according to statistical principles, calculating the weights of each feature, and converting the contribution degree of each feature into a numerical representation through normalization processing and weighted summation methods.

[0011] Step S104, weighted pooling of the mixed maximum features, which includes: adopting a mixed weighted pooling strategy based on the weights calculated in Step S103, combining the maximum features in the local area with the overall weighted summation result, so as to extract the most significant local feature information and retain the overall statistical information, realizing the efficient fusion of features and reducing the data dimension.

[0012] Furthermore, the weighted strategy for the median of adjacent feature mappings:

[0013] It includes two steps: effective feature selection and feature weight calculation; effective feature selection is used to determine which features participate in the pooling operation, and feature weight calculation will calculate the corresponding weight values of these features during model training.

[0014] Furthermore, the effective feature selection:

[0015] The value of the p-th neuron in the (l + 1)-th layer, i.e., the current pooling layer, corresponds to a corresponding feature mapping region of size U×U in the l-th layer after convolution extraction; for these U×U feature mappings, the "minimum contrast sum" method is used to quickly find the median median p , and expand from this median to both sides, select some features adjacent to the median for weighted pooling operation, and the pooling weights of the unselected features are regarded as 0.

[0016] Furthermore, for the U×U feature mapping region, the selection includes median p and n (n ≤ U×U) feature mappings adjacent to median p for weighted pooling. These n feature mappings are arranged in order of value size with median p as the center. The leftmost feature mapping is the smallest, and the rightmost feature mapping is the largest; these n feature mappings will be divided into three parts: the blue square represents the feature mapping including median p and having the value closest to medianp, with the number being m1; the orange square represents the feature mapping second closest to median pThe feature maps, with a quantity of m2; the remaining green squares represent feature maps with values at both ends, with a quantity of m3; the quantity allocation of the three parts is calculated according to m3 = n - (m1 + m2); according to the parity of the pooling kernel size, the corresponding median p may be 1 (U×U is odd) or 2 (U×U is even). In order to make the number of feature maps participating in pooling on both sides of the median p the same and combined with the parity of the selected number of feature maps n, the quantities of m1, m2, and m3 will be slightly adjusted during the actual allocation process.

[0017] Furthermore, the calculation of the feature weights:

[0018] For the method of selecting valid feature maps, the n feature maps to be weighted are divided into three parts according to the distance from the median p of the feature maps. The feature maps closer to the median p play a greater role in the network function. Therefore, according to the "3σ" principle of the normal distribution, weight allocation is performed on the features of these three parts; the roles of each neuron in each part are not very different, so the strategy of evenly distributing weights is adopted; specifically, the pooling weight corresponding to each feature map in the middle part is

[0019]

[0020] The pooling weight corresponding to each feature map in the second part is

[0021]

[0022] For the two outermost parts, the pooling weight corresponding to each feature map is

[0023]

[0024] During the model training process, it is necessary to backpropagate the error δ(l + 1, p). At this time, the pooling weights calculated in equations (1) to (3) are assigned to the n neurons in the corresponding area of the previous layer, and the corresponding gradients are calculated for the update of the convolution kernel.

[0025] Furthermore, the weighted pooling of the hybrid maximum features:

[0026] Considering that the maximum feature map contains the texture information in the picture, in addition to selecting the n feature maps adjacent to the median of the feature maps, the maximum feature map is also incorporated to participate in the pooling operation; during the pooling process, the weight of the maximum feature map is denoted as w max , and at this time the other n feature maps adjacent to the median pThe total full weight of the feature mapping is 1 - w max , and the weights of the other three parts are calculated according to formulas (4) to (6). This pooling operation is called the hybrid weighted pooling of the median of adjacent feature mappings;

[0027]

[0028] During the backpropagation of the error δ(l + 1, p), according to the pooling weights calculated in formulas (4) to (6) and w max , they are distributed to the n + 1 neurons in the corresponding area of the previous layer, and the corresponding gradients are calculated to update the corresponding convolution kernels.

[0029] Another object of the present invention is to provide a lightweight hybrid weighted pooling system based on statistical principles, including:

[0030] A weighted strategy module for the weighted strategy of the median of adjacent feature mappings;

[0031] A feature selection module for effective feature selection;

[0032] A calculation module for feature weight calculation;

[0033] A weighted pooling module for the weighted pooling of the hybrid maximum feature.

[0034] Another object of the present invention is to provide a computer device, which includes a memory and a processor. When the computer program stored in the memory is executed by the processor, the processor executes the steps of the pooling method for anti-adversarial sample attack for deep neural network image classification tasks.

[0035] Another object of the present invention is to provide a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, the processor executes the steps of the pooling method for anti-adversarial sample attack for deep neural network image classification tasks.

[0036] Another object of the present invention is to provide an information data processing terminal for implementing the lightweight hybrid weighted pooling system based on statistical principles.

[0037] Combined with the above technical solutions and the technical problems solved, the advantages and positive effects of the technical solution to be protected by the present invention are:

[0038] First, the present invention uses a pooling method against adversarial sample attacks for deep neural network image classification tasks, effectively solving the problems of serious signal data noise interference and inaccurate feature extraction in the prior art, and improving the stability and reliability of data processing in industrial applications.

[0039] Aiming at the problem that the traditional method fails to effectively remove noise and outliers in the data preprocessing stage, resulting in inaccurate subsequent feature extraction, the present invention realizes the smoothing process of local features through the median weighted strategy of adjacent feature mapping, reduces the influence of noise on signal analysis, and thus improves the accuracy of overall signal recognition.

[0040] In industrial real-time monitoring and large-scale data processing scenarios, traditional technologies are difficult to meet the requirements due to slow processing speed and high consumption of computing resources. The present invention simplifies the data dimension through effective feature selection and feature weight calculation, and adopts a hybrid weighted pooling strategy, significantly reducing the system operation complexity and improving the data processing efficiency, providing technical support for real-time applications.

[0041] In addition, the present invention realizes the efficient integration of information through the hybrid pooling method that combines local maximum features and the overall weighted summation result, enabling the rapid and accurate capture of key features in actual industrial applications, effectively supporting the intelligent judgment and response of the decision-making system in complex scenarios, and thus solving the problems such as slow response, resource waste and misjudgment risk existing in the prior art in industrial applications.

[0042] The method of the present invention distinguishes the importance of multiple feature mappings of the median of adjacent features based on statistical principles, and mixes and weights the maximum feature mapping, so that in the process of backpropagation of errors in the convolutional network, not only the cost of weight update is reduced, but also it has good interpretability. The present invention also deeply analyzes the role played by this pooling operation in the robustness against adversarial sample attacks. Experimental results show that in the image classification task, the pooling method proposed by the present invention can effectively improve the robustness against adversarial sample attacks on the Cifar100 and Cifar10 datasets on the premise of ensuring the network prediction effect.

[0043] The present invention overcomes the deficiency of the traditional pooling method that does not utilize the statistical regularity between features. By dynamically adjusting the feature mappings participating in pooling, the parameters updated by the network are significantly reduced, improving the training efficiency of the network, enhancing the prediction accuracy of the network in ordinary scenarios, and the network robustness in the adversarial sample attack scenario.

[0044] Second, in view of the problems of low signal data processing efficiency and inaccurate feature extraction in the prior art, the technical solution of the present invention proposes an anti-adversarial sample attack pooling method for deep neural network image classification tasks. Through steps such as median weighting of adjacent feature mappings, selection of effective features, and calculation of feature weights, the influence of noise is effectively suppressed, ensuring that the key information in the signal is fully retained, thereby solving the problems of feature redundancy and noise interference existing in traditional methods.

[0045] In industrial applications, especially for large-scale, real-time signal processing scenarios, data processing speed and accuracy are crucial. Through the weighted pooling strategy of hybrid maximum features, while extracting local significant features, the overall statistical information is also integrated, realizing data dimensionality reduction and efficient fusion. This method not only greatly improves the signal processing speed, but also makes remarkable progress in data accuracy and stability, meeting the high-demand industrial application scenarios.

[0046] Compared with the prior art, the solution of the present invention significantly reduces the consumption of hardware resources and the computational complexity, has the advantage of being lightweight, and is particularly suitable for the real-time signal processing requirements in embedded devices and mobile terminals. By reducing data redundancy and noise interference, the system can respond and process input signals faster, greatly improving the overall performance and service life of the device, and providing a cost-effective solution for industrial applications.

[0047] In summary, the technical solution of the present invention not only effectively solves the deficiencies existing in the prior art in aspects such as feature extraction, data denoising, processing efficiency, and hardware resource utilization rate, but also realizes a double improvement in data processing quality and real-time response ability in industrial applications. This anti-adversarial sample attack pooling method for deep neural network image classification tasks has brought significant technological progress to the field of signal data processing, promoting the development of related industries in the direction of intelligence and precision. Brief Description of the Drawings

[0048] Figure 1 is a flowchart of the anti-adversarial sample attack pooling method for deep neural network image classification tasks provided by an embodiment of the present invention.

[0049] Figure 2 is a structural block diagram of a lightweight hybrid weighted pooling system based on statistical principles provided by an embodiment of the present invention.

[0050] Figure 3 is a schematic diagram of the selection and quantity allocation of effective features provided by an embodiment of the present invention.

[0051] Figure 4 is a graph of the average prediction accuracy of ResNet-20 on Cifar100 for the same pooling varying with the number n of weight updates provided by an embodiment of the present invention.

[0052] Figure 5 It is a prediction accuracy rate graph of ResNet-20 on Cifar100 for different poolings provided by an embodiment of the present invention.

[0053] Figure 6 It is a prediction accuracy rate graph of ResNet-20 on Cifar10 for different poolings provided by an embodiment of the present invention.

[0054] Figure 7 It is a graph of the average value of the prediction accuracy rate of ResNet-20 on Cifar10 for the same pooling varying with the number n of weight updates provided by an embodiment of the present invention. Detailed implementation manners

[0055] In order to make the objectives, technical solutions and advantages of the present invention more clear and understandable, the present invention will be further described in detail below in conjunction with embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0056] As Figure 1 shown, a pooling method for resisting adversarial sample attacks for deep neural network image classification tasks provided by an embodiment of the present invention includes the following steps:

[0057] S101, a weighted strategy for the median of adjacent feature maps;

[0058] S102, effective feature selection;

[0059] S103, feature weight calculation;

[0060] S104, weighted pooling of mixed maximum features.

[0061] (S101, a weighted strategy for the median of adjacent feature maps):

[0062] In the initial stage of signal data processing, first perform adjacent feature mapping on the original signal and adopt a median weighting strategy. Specifically, for each data point in the signal, the system will examine multiple feature values in its adjacent area, use the median as a measurement index, and set a weighting coefficient according to the statistical distribution to reduce the influence of noise and outliers, so as to ensure that the extracted features are more representative and stable within the local range.

[0063] (S102, effective feature selection):

[0064] After the neighboring weighting process, the system preliminarily screens all the mapped features and evaluates the features through statistical analysis methods. Using indicators such as data distribution, correlation analysis, and variance, redundant and noisy data are screened out, and only those effective features that play a key role in the overall signal recognition and classification are retained. This step lays a solid foundation for the subsequent calculation of feature weights, ensuring that the main information is highlighted during the processing.

[0065] (S103 Feature Weight Calculation):

[0066] After selecting the effective features, the system quantitatively evaluates the importance of each feature according to statistical principles and calculates the weights of each feature. Through normalization processing and weighted summation methods, the contribution degree of each feature is converted into a numerical representation, so that the feature can affect the final result according to its weight during subsequent hybrid pooling. This weight allocation mechanism based on statistical data effectively reflects the relative importance of each feature in the overall signal and enhances the discriminant ability of the model.

[0067] (S104 Weighted Pooling of Hybrid Maximum Features):

[0068] Finally, the system adopts a hybrid weighted pooling strategy based on the calculated weights, combining the maximum features in the local area with the overall weighted summation result. This step can not only extract the most significant local feature information but also retain the overall statistical information, realizing the efficient fusion of features. Through this hybrid method, the system not only improves the response ability to key features but also reduces the data dimension, ensuring that the entire signal processing process is both lightweight and highly accurate.

[0069] The weighted strategy for the median of neighboring feature mappings provided by the embodiments of the present invention:

[0070] It includes two steps: selecting effective features and calculating feature weights; selecting effective features is used to determine which features participate in the pooling operation, and calculating feature weights will calculate the corresponding weight values of these features during model training.

[0071] The effective feature selection provided by the embodiments of the present invention:

[0072] The value of the p-th neuron in the (l + 1)-th layer, that is, the current pooling layer, corresponds to a corresponding feature mapping region of size U×U in the l-th layer after convolution extraction; for these U×U feature mappings, the "minimum contrast sum" method is used to quickly find the median median of the feature mapping p , and expand from the median to both sides, select some features adjacent to the median for weighted pooling operation, and the pooling weights of the unselected features are regarded as 0.

[0073] The U×U feature mapping region provided by the embodiment of the present invention selects and includes median p and the neighboring median p to perform weighted pooling on n (n≤U×U) feature mappings. These n feature mappings are arranged in order of their values with median p as the center. The leftmost feature mapping is the smallest, and the rightmost feature mapping is the largest; these n feature mappings will be divided into three parts: the blue squares represent the feature mappings that include median p and have values closest to median p , with the number being m1; the orange squares represent the feature mappings that are the second closest to median p , with the number being m2; the remaining green squares represent the feature mappings with values at both ends, with the number being m3; the quantity allocation of the three parts is calculated in the manner of m3 = n - (m1 + m2); according to the parity of the pooling kernel size, the corresponding median p may be 1 (when U×U is odd) or 2 (when U×U is even). To make the number of feature mappings participating in pooling on both sides of median p the same and combined with the parity of the selected number of feature mappings n, the quantities of m1, m2, and m3 will be slightly adjusted during the actual allocation process.

[0074] Feature weight calculation provided by the embodiment of the present invention:

[0075] For the method of selecting valid feature mappings, the n feature mappings to be weighted are divided into three parts according to their distances from the median p of the feature mappings. The closer the feature mapping is to median p , the greater the role it plays in the network function, so the weights of these three parts of features are allocated according to the "3σ" principle of the normal distribution; the role of each neuron in each part is not very different, so the strategy of evenly distributing weights is adopted; specifically, the pooling weight corresponding to each feature mapping in the middle part is

[0076]

[0077] The pooling weight corresponding to each feature mapping in the second part is

[0078]

[0079] For the two outermost parts, the pooling weight corresponding to each feature mapping is

[0080]

[0081] During the model training process, it is necessary to backpropagate the error δ(l+1,p). At this time, the pooling weights calculated in equations (1)-(3) are assigned to n neurons in the corresponding area of the previous layer, and the corresponding gradients are calculated to update the convolutional kernel.

[0082] The weighted pooling of hybrid maximum features provided by the embodiments of the present invention:

[0083] Considering that the maximum feature map contains the texture information in the picture, in addition to selecting n feature maps with the median value of the adjacent feature maps, the maximum feature map is also incorporated to participate in the pooling operation together; during the pooling process, the weight of the maximum feature map is denoted as w max , at this time the total full weight of the remaining n adjacent median p feature maps is 1 - w max , and the weights of the other three parts are calculated according to equations (4)-(6). This pooling operation is called the hybrid weighted pooling of the median of adjacent feature maps;

[0084]

[0085] During the backpropagation process of the error δ(l+1,p), according to the pooling weights calculated in equations (4)-(6) and w max are assigned to n + 1 neurons in the corresponding area of the previous layer, and the corresponding gradients are calculated to realize the update of the corresponding convolutional kernel.

[0086] As Figure 2 shown, a lightweight hybrid weighted pooling system based on statistical principles provided by the embodiments of the present invention includes:

[0087] A weighted strategy module for the weighted strategy of the median of adjacent feature maps;

[0088] A feature selection module for effective feature selection;

[0089] A calculation module for calculating feature weights;

[0090] A weighted pooling module for the weighted pooling of hybrid maximum features.

[0091] Another object of the present invention is to provide a computer device, the computer device includes a memory and a processor, the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of the anti-adversarial sample attack pooling method for deep neural network image classification tasks.

[0092] Another object of the present invention is to provide a computer-readable storage medium storing a computer program, which, when executed by a processor, causes the processor to execute the steps of the anti-adversarial sample attack pooling method for deep neural network image classification tasks.

[0093] Another object of the present invention is to provide an information data processing terminal for implementing the lightweight hybrid weighted pooling system based on statistical principles.

[0094] Specific implementation of the present invention:

[0095] 1 Pooling operations of convolutional neural networks

[0096] 1.1 Convolutional neural network structure

[0097] A convolutional neural network is a deep feedforward neural network. A typical convolutional neural network is generally stacked by convolutional layers, pooling layers and fully connected layers. In image processing, the role of the convolutional layer is to extract the features of local regions of the image. After the convolutional layer, a pooling layer is added. Through feature selection, the scale of image features can be effectively reduced, and the number of model parameters can be reduced. In addition, the pooling layer uses the overall statistical features of adjacent regions at a certain position to replace the specific content at that position, so that the convolutional neural network can remain unchanged for small local morphological changes and has a larger visual receptive field. The features after convolution and pooling operations will be input into the fully connected layer to implement tasks such as image classification. Currently, typical deep convolutional neural network structures include LeNet-5, Inception, ResNet, etc.

[0098] 1.2 Characteristics of different pooling operations

[0099] During the backpropagation of errors, different pooling functions have different numbers of weight updates for the convolutional kernels in the previous layer. The convolutional neural network calculates the gradients of the parameters by backpropagating the error terms of each layer and realizes parameter updates. Assume that the (l + 1)-th layer is the pooling layer and the l-th layer is the convolutional layer, and the size of the convolutional kernel is U×V. Since the pooling layer uses downsampling operation, the error term δ(l + 1, p) of the p-th neuron in the (l + 1)-th layer corresponds to a region in the corresponding feature map of the l-th layer. If max pooling is adopted, the value of the error term δ(l + 1, p) will be directly passed to the neuron corresponding to the maximum value in the corresponding region of the previous layer, and the error terms of other neurons in this region are set to 0. At this time, the number of weight updates of the convolutional kernel is 1. If average pooling is adopted, each value of the error term δ(l + 1, p) will be evenly distributed to all neurons in the corresponding region of the previous layer. At this time, the number of weight updates of the convolutional kernel is U×V. The approximate median pooling takes the average value close to multiple feature maps as the pooling result. Therefore, the value of the error term δ(l + 1, p) will be evenly distributed to some neurons in the corresponding region of the previous layer. At this time, the number of weight updates of the convolutional kernel is the same as the number of feature maps close to the median value of the features taken in the pooling operation. The advantage of max pooling is that the number of convolutional kernel weight updates is small. However, since the vast majority of convolutional kernels do not participate in the update, the prediction accuracy of the network will decrease. Since all convolutional kernel weights need to participate in the update in average pooling, although it can improve the prediction accuracy of the entire network, it also increases the scale of network parameter updates. The approximate median pooling only needs to update the weights of some convolutional kernels, but does not distinguish the importance of multiple feature maps adjacent to the pooling median in image recognition. To address the above problems, the present invention distinguishes the importance of feature maps and designs an anti-adversarial sample attack pooling method for deep neural network image classification tasks.

[0100] 2 Hybrid weighted pooling method for neighboring feature map medians

[0101] 2.1 Statistical principles

[0102] An image is composed of a number of pixels, and each pixel is independent of the others. The influence of a single pixel on the recognition of the image content is very small. After the image is subjected to feature extraction by a small-sized convolutional kernel, the number of resulting features is still relatively large and independent of each other, and the influence of a single feature on the recognition of the picture is not significant. Therefore, we can consider that the feature map obtained after the image passes through the convolutional layer for feature extraction approximately follows a normal distribution statistically. In statistics, a random variable X that follows a normal distribution satisfies P(|X - μ| < xσ) = 2Φ(x) - 1, where μ is the mathematical expectation, σ is the standard deviation, and Φ(x) is the distribution function of the standard normal distribution. When σ takes 1, 2, and 3 respectively, the corresponding probability values are 0.6826, 0.9544, and 0.9974. These probability values indicate that approximately 70% of the values of a random variable that follows a normal distribution are concentrated within the range of σ from the center μ, and it is almost impossible to deviate from the center μ by a distance exceeding 3σ. Statistically, this characteristic of the normal distribution is called the "3σ" principle. Considering using the median value of the feature map as the approximate mean μ in the pooling layer, a weighted pooling operation based on the "3σ" principle is performed on some feature maps adjacent to the median value of the feature map, so that the convolutional network can reduce the update scale of model parameters during the training process, the update strategy has a certain interpretability, and at the same time maintain a good prediction accuracy.

[0103] 2.2 Weighting Strategy for Feature Maps Adjacent to the Median

[0104] Considering that the importance of the selected feature maps is related to their distance from the median value of the feature map, a weighting strategy for feature maps adjacent to the median value is proposed based on the "3σ" principle of the normal distribution for the selected multiple feature maps. This strategy includes two steps: effective feature selection and feature weight calculation. Effective feature selection is used to determine which features participate in the pooling operation, and feature weight calculation will calculate the corresponding weight values of these features during the model training process. The following all assume that the (l + 1)-th layer is the pooling layer, the l-th layer is the convolutional layer, and the size of the pooling kernel is U×U.

[0105] 2.2.1 Effective Feature Selection

[0106] The value of the p-th neuron in the (l + 1)-th layer, i.e., the current pooling layer, corresponds to a corresponding feature map region of size U×U in the l-th layer obtained after convolutional extraction. For these U×U feature maps, the "minimum contrast sum" method is used to quickly find the median value median p , and expand from this median value to both sides, select some features adjacent to the median value for weighted pooling operation, and the pooling weights of the unselected features are regarded as 0.

[0107] For the above U×U feature map region, select including median pinside and adjacent to median p Perform weighted pooling on n (n ≤ U×U) feature maps, as Figure 3 shown. These n feature maps are arranged in order of their values with median p as the center. The leftmost feature map has the smallest value, and the rightmost feature map has the largest value. These n feature maps will be divided into three parts: The blue squares represent the feature maps that include median p and have values closest to median p , with a quantity of m1; the orange squares represent the feature maps that are the next closest to median p , with a quantity of m2; the remaining green squares represent the feature maps with values at both ends, with a quantity of m3. The quantity distribution of the three parts is calculated in the manner of m3 = n - (m1 + m2). According to the parity of the pooling kernel size, the corresponding median p may be 1 (when U×U is odd) or 2 (when U×U is even). In order to make the number of feature maps participating in pooling distributed equally on both sides of median p and combined with the parity of the selected number of feature maps n, the quantities of m1, m2, and m3 will be slightly adjusted during the actual distribution process.

[0108] 2.2.2 Feature Weight Calculation

[0109] According to the method of selecting valid feature maps in 2.2.1, the n feature maps to be weighted are divided into three parts according to their distances from the median p of the feature maps. The closer the feature map is to median p , the greater the role it plays in the network function. Therefore, according to the "3σ" principle of the normal distribution, weights are assigned to the features of these three parts. The roles of each neuron in each part are not very different, so the strategy of evenly distributing weights is adopted. Specifically, the pooling weight corresponding to each feature map in the middle part is

[0110]

[0111] The pooling weight corresponding to each feature map in the second part is

[0112]

[0113] For the two outermost parts, the pooling weight corresponding to each feature map is

[0114]

[0115] During the model training process, it is necessary to perform backpropagation on the error δ(l+1,p). At this time, the pooling weights calculated in equations (1) to (3) are assigned to n neurons in the corresponding area of the previous layer, and the corresponding gradients are calculated to update the convolutional kernel.

[0116] 2.3 Weighted Pooling of Mixed Maximum Features

[0117] Considering that the maximum feature map contains the texture information in the picture, in addition to selecting n feature maps with the median value of the neighboring feature maps, the maximum feature map is also incorporated to participate in the pooling operation. During the pooling process, the weight of the maximum feature map is denoted as w max , at this time the total full weight of the remaining n neighboring median p feature maps is 1 - w max , and the weights of the other three parts are calculated according to equations (4) to (6). This pooling operation is called the mixed weighted pooling of the median of neighboring feature maps.

[0118]

[0119] During the backpropagation process of the error δ(l+1,p), according to the pooling weights calculated in equations (4) to (6) and w max are assigned to n + 1 neurons in the corresponding area of the previous layer, and the corresponding gradients are calculated to update the corresponding convolutional kernel.

[0120] The weighted pooling proposed by the present invention operates on n feature maps with the median of neighboring feature maps selected in the corresponding area. Therefore, during the parameter learning process of the convolutional neural network, each value of the backpropagation error δ(l+1,p) will be assigned to n neurons in the corresponding area of the previous layer according to the weights of the pooling kernel, and the convolutional kernel at the corresponding position will be updated accordingly. When n is relatively small compared to U×U, this pooling operation can significantly reduce the number of weight updates compared to average pooling, and can effectively improve the efficiency of the algorithm. It is a pooling method for lightweight weight update.

[0121] 3 Experiments and Analysis

[0122] 3.1 Experimental Settings

[0123] The experiments were carried out on the Cifar10 and Cifar100 standard datasets. Before model training, the same data augmentation operation was performed on all training data. First, 4 pixels were padded on each side of the picture, then randomly cropped to obtain pictures with a size of 32×32, and finally the pictures were randomly flipped to obtain the augmented dataset.

[0124] For comparison with relevant literature, the ResNet-20 residual network was selected. In the experiment, a pooling kernel of size 8×8 was used, so the median of the corresponding feature map was p There were 2. Considering that if the number n of the selected neighboring medians p of the feature maps was too small, it was not conducive to using the 3σ rule based on the normal distribution. In the experiment, n = 8, 16, 32 were taken respectively, and the number of feature maps in the three parts was allocated according to the method in 2.2 and the corresponding pooling weights were calculated, as shown in Table 1.

[0125] Table 1 Allocation of the number of feature maps and pooling weights

[0126]

[0127] For each n value in the group, the ResNet-20 residual network was run 8 times, and the prediction top-1 accuracy (Top-1 Accuracy) and top-5 accuracy (Top-5 Accuracy) were evaluated in the format of "mean±std". During the training process, the SGD optimizer was used, the initial learning rate was set to 0.1, and the learning rate was dynamically adjusted to 0.01 and 0.001 after the 100th epoch and the 175th epoch respectively. When the epoch reached 300, the network stopped training.

[0128] For the hybrid weighted pooling of neighboring feature medians proposed in the present invention, for each group of n values, different weight values w max were separately assigned to the maximum feature map, and the remaining were also adjusted accordingly to In the experiment, w max = 0.05, 0.1, 0.15 were taken respectively for comparison.

[0129] 3.2 Results and analysis

[0130] In the experiment, the pooling method proposed in the present invention was compared with max pooling, average pooling, and approximate median pooling. To illustrate the influence of the weighting strategy and the hybrid maximum feature map strategy of the proposed hybrid weighted pooling of neighboring feature medians on the prediction accuracy, ablation experiments were carried out. Among them, "approximate median + max feature pooling" means a pooling operation that adds the maximum feature map to the approximate median pooling and takes the average weight; "weighted pooling of neighboring feature medians" means a pooling operation that weights the n feature maps of the selected neighboring feature map medians without mixing and adding the maximum feature map. The experimental results of the above six kinds of pooling are shown in Table 2 and Table 3. The 8, 16, 32 in the brackets in the table represent the number of weight updates, and 0.15, 0.1, 0.05 represent the weights of the maximum feature map selected when the accuracy of this pooling method is the best. The meanings in the subsequent tables are the same.

[0131] Table 2 Prediction accuracy percentages of ResNet-20 with different pooling methods on Cifar100

[0132]

[0133] Note: Bold indicates the optimal result, and underlined indicates the sub-optimal result. The same applies hereinafter.

[0134] Table 3 Prediction accuracy percentages of ResNet-20 with different pooling methods on Cifar10

[0135]

[0136]

[0137] Experiments show that as n increases, the weight of the maximum feature map that enables the average network prediction accuracy to reach the optimal value decreases. This indicates that when the value of n is small, the role of the maximum feature map is more obvious, and with the addition of more feature information, the role of the maximum feature map gradually weakens.

[0138] To intuitively analyze the influence of the pooling method and the number of weight updates n on the network prediction accuracy, the average prediction accuracies in Table 2 and Table 3 are classified as shown in Figure 4 and Figure 7 The results of running the six pooling methods 8 times each on the Cifar100 and Cifar10 datasets are sorted from smallest to largest, and the results are as shown in Figure 5 and Figure 6 shown.

[0139] From Figure 4 it can be seen that for the Cifar100 dataset, the top-1 accuracy and top-5 accuracy of the same pooling method have similar patterns. For approximate median pooling, the accuracy increases as n increases. According to the pooling method of approximate median + maximum feature, when n = 8, the network performance is improved most significantly compared with approximate median pooling. For the weighted pooling of neighboring feature medians, the network performance is significantly improved step by step as n increases.

[0140] Figure 4 In Figure 5In the same situation, when n is small, due to the small number of feature mappings based on statistical principles, the actual distribution will deviate from the assumption of the normal distribution. At this time, the addition of the picture texture information contained in the maximum feature mapping will play an important role in picture recognition. As n increases, the statistical distribution of feature mappings will be closer to the assumption of the normal distribution. At this time, the weighting based on statistical principles will play an obvious role. On the other hand, since the approximate median + maximum feature pooling uses an average weight, when the number of n increases, the weight assigned to the maximum eigenvalue will also decrease, and the role played will also weaken.

[0141] Figure 5 The results show that for the Cifar100 dataset, the prediction accuracies of the weighted pooling of adjacent feature medians and the hybrid weighted pooling of adjacent feature medians are better than the approximate median pooling and significantly better than the maximum pooling. This shows that without changing or only increasing the number of weight updates by 1, adding the weighting strategy based on statistical principles and the hybrid weighting strategy adding the maximum feature mapping helps to improve the prediction performance of the network. When n = 8, the result of the hybrid weighted pooling of adjacent feature medians is closest to that of the average pooling. At this time, the number of updates to the convolution kernel during the backpropagation process of the network is only 9, and the parameters are significantly reduced compared with the 64 weight update numbers of the average pooling, indicating that the two aspects of the weighting based on statistical principles and the pooling strategy of the hybrid maximum eigenvalue have a positive effect on improving the network performance.

[0142] On the Cifar10 dataset, since the other five pooling methods have a very obvious advantage in prediction accuracy over the maximum pooling, for the convenience of observation, Figure 6 the prediction results of the five poolings other than the maximum pooling are given. Different from the test results of Cifar100, for the weighted pooling of adjacent feature medians, when n = 16, the prediction accuracy of the network is optimal. For the hybrid weighted pooling of adjacent feature medians with the addition of the maximum feature mapping, the accuracy is the highest when n = 32, but the gap with n = 8 and n = 16 is not large, which may be related to the relatively small number of classification categories of Cifar10.

[0143] From Figure 7 it can be found that the prediction accuracies of the four pooling types are relatively close when n = 8, and the effects are almost all sub-optimal for this type of pooling. At this time, the number of weight updates of the network convolution kernel is significantly reduced compared with the average pooling. Therefore, for the hybrid weighted pooling of adjacent feature medians, taking n = 8 is still a good choice.

[0144] Table 4 Top-5 accuracies (%) of different poolings of ResNet-20 on Cifar100 under FGSM, DI-FGSM, NI-FGSM attacks

[0145]

[0146] Table 5 Top-1 accuracy % of different poolings of ResNet-20 under FGSM, DI-FGSM, and NI-FGSM attacks on Cifar10

[0147]

[0148]

[0149] The experiments used FGSM, IN-FGSM, and DI-FGSM adversarial sample attack methods to compare the network robustness of the above six poolings. On Cifar100 and Cifar10, the number of weight updates n = 8 was a better choice, so n = 8 was selected for prediction. The results are shown in Table 4 and Table 5. The results show that the pooling method proposed by the present invention has the optimal or sub-optimal prediction accuracy under the three adversarial sample attack methods and has good stability, indicating that the pooling method proposed by the present invention not only has good prediction performance but also can enhance the network robustness.

[0150] Table 6 Prediction accuracy % of different poolings of LeNet-5 on Cifar10

[0151]

[0152] Table 7 Prediction accuracy % of different poolings of ResNet-50 on Cifar100

[0153]

[0154] Table 8 Top-1 accuracy % of different poolings of LeNet-5 under adversarial sample attacks on Cifar10

[0155]

[0156] Table 9 Top-5 accuracy % of different poolings of ResNet-50 under adversarial sample attacks on Cifar100

[0157]

[0158] To test the generality of the proposed pooling method in other convolutional neural networks, from the perspectives of simplifying the model structure and deepening the model depth, experiments were conducted on the Cifar10 and Cifar100 datasets using the hybrid weighted pooling of the median of adjacent features, max pooling, average pooling, and approximate median pooling with n = 8 based on the LeNet-5 network and the ResNet-50 network respectively. The experimental results of the prediction accuracy are shown in Tables 6 and 7, and the experimental results of the robustness against adversarial sample attacks are shown in Tables 8 and 9. The results show that on LeNet-5 and ResNet-50, the performance of the hybrid weighted pooling of the median of adjacent features is basically the same as that on ResNet-20. The prediction accuracy is better than both max pooling and approximate median pooling, and the robustness against adversarial sample attacks is either optimal or sub-optimal. On the LeNet-5 network, the prediction accuracy and adversarial robustness of average pooling are significantly better than those of the sub-optimal hybrid weighted pooling of the median of adjacent features, which may be related to the simple structure and relatively small number of parameters of the LeNet-5 network. Therefore, the proposed pooling method of the present invention can achieve good experimental results on different convolutional neural networks, and the effect is more obvious on networks with complex structures and large numbers of parameters.

[0159] The present invention proposes a lightweight hybrid weighted pooling method for convolutional neural networks based on statistical principles. This method distinguishes the importance of multiple feature maps of the median of adjacent features based on statistical principles and mixes and weights the maximum feature maps, enabling the convolutional network to not only reduce the cost of weight update during the backpropagation of errors but also have good interpretability. The present invention also deeply analyzes the role of this pooling operation in the robustness against adversarial sample attacks. The experimental results show that in the image classification task, the proposed pooling method of the present invention can effectively improve the robustness against adversarial sample attacks on the Cifar100 and Cifar10 datasets while ensuring the network prediction effect.

[0160] It should be noted that the embodiments of the present invention can be implemented by hardware, software, or a combination of software and hardware. The hardware part can be implemented using dedicated logic; the software part can be stored in a memory and executed by a suitable instruction execution system, such as a microprocessor or dedicated designed hardware. Those of ordinary skill in the art can understand that the above devices and methods can be implemented using computer-executable instructions and / or included in processor control code, such as provided on a carrier medium such as a disk, CD, or DVD-ROM, a programmable memory such as a read-only memory (firmware), or a data carrier such as an optical or electronic signal carrier. The devices and their modules of the present invention can be implemented by hardware circuits of programmable hardware devices such as very large scale integrated circuits or gate arrays, semiconductors such as logic chips, transistors, etc., or field programmable gate arrays, programmable logic devices, etc., can also be implemented by software executed by various types of processors, or can be implemented by a combination of the above hardware circuits and software such as firmware.

[0161] As described above, the above are only specific embodiments of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present invention, any modifications, equivalent replacements, and improvements made within the spirit and principle of the present invention should be covered by the protection scope of the present invention.

Claims

1. An anti-adversarial sample attack pooling method for deep neural network image classification tasks, characterized in that It includes the following steps: Step S101, the weighted strategy of the median of adjacent feature mapping. The steps include: performing adjacent feature mapping on the original signal, examining multiple eigenvalues within the adjacent region of each data point in the signal, using the median as a measurement index, and setting a weighted coefficient according to the statistical distribution to reduce the influence of noise and outliers, so as to ensure that the extracted features are more representative and stable within the local range; Step S102, selection of effective features. The steps include: performing statistical analysis on the mapped features obtained in Step S101, and screening out redundant and noisy data by using data distribution, correlation analysis and variance, and only retaining the effective features that play a key role in the overall signal recognition and classification; Step S103, calculation of feature weights. The steps include: quantitatively evaluating the effective features selected in Step S102 according to statistical principles, calculating the weights of each feature, and converting the contribution degree of each feature into a numerical representation through normalization processing and weighted summation method; and Step S104, weighted pooling of mixed maximum features. The steps include: adopting a mixed weighted pooling strategy based on the weights calculated in Step S103, combining the maximum features in the local region with the overall weighted summation result, so as to extract the most significant local feature information and retain the overall statistical information, realize the efficient fusion of features and reduce the data dimension.

2. The anti-adversarial sample attack pooling method for deep neural network image classification tasks according to claim 1, wherein, The weighted strategy of the median of adjacent feature mapping: It includes two steps: selection of effective features and calculation of feature weights; The selection of effective features is used to determine which features participate in the pooling operation, and the calculation of feature weights will calculate the corresponding weight values of these features during the model training process.

3. The anti-adversarial sample attack pooling method for deep neural network image classification tasks according to claim 1, characterized in that, The selection of effective features: The value of the p-th neuron in the (l + 1)-th layer, that is, the current pooling layer, corresponds to a corresponding feature mapping region of size U×U in the l-th layer obtained after convolution extraction; For these U×U feature maps, the "minimum contrast sum" method is used to quickly find the median value in the feature maps p , and expand from the median to both sides. Select some features adjacent to the median for weighted pooling operation, and the pooling weights of the unselected features are regarded as 0.

4. The anti-adversarial sample attack pooling method for deep neural network image classification tasks according to claim 3, wherein For the U×U feature mapping region, select those including median p and adjacent to median p to perform weighted pooling on n (n≤U×U) feature mappings. These n feature mappings are arranged in ascending order of values with median p as the center. The leftmost feature mapping has the smallest value, and the rightmost feature mapping has the largest value. These n feature mappings will be divided into three parts: The blue squares represent the feature mappings including median p and having values closest to median p , with the number being m1; the orange squares represent the feature mappings second closest to median p , with the number being m2; the remaining green squares represent the feature mappings with values at both ends, with the number being m3. The quantity distribution of the three parts is calculated in the manner of m3 = n - (m1 + m2). According to the parity of the pooling kernel size, the corresponding median p may be 1 (when U×U is odd) or 2 (when U×U is even). To make the number of feature mappings participating in pooling on both sides of median p the same and considering the parity of the number of selected feature mappings n, the quantities of m1, m2, and m3 will be slightly adjusted during the actual distribution process.

5. The anti-adversarial sample attack pooling method for deep neural network image classification tasks according to claim 1, wherein The calculation of feature weights: For the method of selecting effective feature maps, the n feature maps to be weighted are divided into three parts according to the distance from the median of the feature maps. The closer the feature map is to the median, the greater the role it plays in the network function. Therefore, according to the "3σ" principle of the normal distribution, weight distribution is performed on the features of these three parts; the roles of each neuron in each part are not very different, so an equal weight distribution strategy is adopted; specifically, the pooling weight corresponding to each feature map in the middle part is p of the size, the closer the feature map is to the median p , the greater the role it plays in the network function. Therefore, according to the "3σ" principle of the normal distribution, weight distribution is performed on the features of these three parts; the roles of each neuron in each part are not very different, so an equal weight distribution strategy is adopted; specifically, the pooling weight corresponding to each feature map in the middle part is The pooling weight corresponding to each feature mapping in the second part is For the outermost two parts, the pooling weight corresponding to each feature mapping is During the model training process, it is necessary to backpropagate the error δ(l + 1, p). At this time, the pooling weights calculated in equations (1) to (3) are assigned to n neurons in the corresponding region of the previous layer, and the corresponding gradients are calculated for the update of the convolution kernel.

6. The anti-adversarial sample attack pooling method for deep neural network image classification tasks according to claim 1, wherein The weighted pooling of mixed maximum features: Considering that the maximum feature map contains the texture information in the image, in addition to selecting n feature maps that are the median of the neighboring feature maps, the maximum feature map is also incorporated to participate in the pooling operation together; during the pooling process, the weight of the maximum feature map is denoted as w max , at this time, the total full weight of the remaining n neighboring median p feature maps is 1 - w max , the weights of the other three parts are calculated according to formulas (4) to (6) respectively, and this pooling operation is called the hybrid weighted pooling of the median of neighboring feature maps; During the backpropagation of the error δ(l+1,p), the pooling weights calculated according to equations (4) to (6) and w max are distributed to n+1 neurons in the corresponding area of the previous layer, and the corresponding gradients are calculated to update the corresponding convolutional kernel.

7. A lightweight hybrid weighted pooling system based on statistical principles for implementing the anti-adversarial sample attack pooling method for deep neural network image classification tasks as described in any one of claims 1-6, characterized in that, The lightweight mixed weighted pooling system based on statistical principles includes: A weighted strategy module for the weighted strategy of the median of adjacent feature mapping; A feature selection module for the selection of effective features; A calculation module for the calculation of feature weights; A weighted pooling module for the weighted pooling of mixed maximum features.

8. A computer device, characterized in that, The computer device includes a memory and a processor. When the computer program stored in the memory is executed by the processor, the processor executes the steps of the anti-adversarial sample attack pooling method for deep neural network image classification tasks as described in any one of claims 1-6.

9. A computer-readable storage medium stores a computer program. When the computer program is executed by a processor, the processor is caused to execute the steps of the anti-adversarial sample attack pooling method for deep neural network image classification tasks according to any one of claims 1-6.

10. An information data processing terminal, characterized in that, The information data processing terminal is used to implement the lightweight hybrid weighted pooling system based on statistical principles according to claim 7.

Images