An image convolutional neural network model security evaluation method based on principal mode neuron coverage

Abstract

The present application relates to a kind of based on main mode neuron coverage including image classification, target detection and image segmentation in the security evaluation method of image recognition field convolutional neural network, comprising step one: parsing image recognition field convolutional neural network model structure;Step two: setting model level plug-in point;Step three: using the test set for model security evaluation to carry out model security test;Step four: coverage driven model security test sample generation;Step five: after expansion, model security evaluation test set carries out model security evaluation, obtains more sufficient model security evaluation result.The coverage index granularity finer presented in the present application, index understanding and calculation are simple, more easily realize.Using main mode neuron coverage as the measurement mode of model security evaluation sufficiency, the security problem of model when facing the disturbance attack in reality situation is measured.Coverage driven model test sample generation technology can generate sample using multiple attack means, and it is strong in flexibility.

Description

Technical Field

[0001] This invention belongs to the field of artificial intelligence model testing and evaluation, and relates to a method for evaluating the security of image convolutional neural network models. Background Technology

[0002] DeepXplore (Pei K, Cao Y, Yang J, et al. DEEPXPLORE: Automated Whitebox Testing of Deep Learning Systems[J]. Mobile Computing and Communications Review, 2018, 22(3):36-38) is the first white-box testing framework for classic artificial neural networks in the field of artificial intelligence model testing and evaluation, and it first proposed the concept of neuron coverage. DeepXplore defines neuron coverage as the proportion of activated neurons among all neurons in the model. A neuron is considered activated if its output exceeds a certain threshold after passing through the activation function. It is generally believed that neuron coverage is positively correlated with test adequacy. DeepXplore uses multiple similar DNNs for cross-comparison to assist in generating test cases, maximizing neuron coverage in the process to improve the adequacy of model evaluation and obtain more reliable model evaluation results.

[0003] However, DeepXplore relies on multiple similar DNN models, which is difficult to satisfy, and its criteria for determining coverage / activation are too coarse. Nearly 100% coverage can be achieved using simple adversarial examples, thus requiring further research into finer-grained coverage criteria. Furthermore, DeepXplore targets fully connected neural networks for image classification, rather than the convolutional neural networks most commonly used in image recognition fields such as image classification, object detection, and image segmentation. Therefore, its application in current AI model testing within the image recognition field is limited.

[0004] DeepGauge (Multi-granularity testing criteria for deep learning systems) proposes multi-granularity testing and evaluation criteria for artificial neural networks, including neuron-level coverage criteria and layer-level coverage criteria. The layer-level coverage criterion measures how many neurons in each layer are the most active, defining coverage as the percentage of the most active neurons in each layer out of the total number of neurons in the neural network. This layer-level coverage criterion is called the Top-k criterion in DeepGauge. The specific definition of the Top-k criterion is as follows:

[0005] Given an input x and two different neurons n1 and n2 in the same layer, if out(n1,x) > out(n2,x), it means that neuron n1 is more active. (out(n,x) represents the output value of neuron n after x is input into the neural network model.) Using topology... k (x,i) represents the k most active neurons in the i-th layer after x is input into the model. The Top-k neuron coverage represents the proportion of the k most active neurons in each layer in the total number of neurons in the network model under the activation of the input set T, as shown in the following formula:

[0006]

[0007] Where l represents the total number of layers in the neural network model, and |N| represents the total number of neurons in the neural network model.

[0008] However, DeepGauge's Top-k coverage calculation only considers cases where k = 1, 2, and 3. Convolutional neural networks typically have millions, or even more, neurons. Achieving a high coverage level requires a massive amount of data, which is difficult to meet in practice. Furthermore, the neuron-level coverage criterion proposed in DeepGauge relies on pre-setting the boundaries of the neurons' main output ranges. Obtaining these boundaries is challenging and prone to error, thus limiting its practicality.

[0009] Currently, security evaluation of image recognition convolutional neural network models, including those for image classification, object detection, and image segmentation, faces the problems of "insufficient and inaccurate" assessments. These assessments neglect the vulnerability of models to various attacks and interferences in real-world applications, leading to significant discrepancies between actual model performance and experimental data. This limits the application of models in security-critical scenarios and hinders the continued growth of the artificial intelligence market. This invention aims to incorporate consideration of test adequacy into model security evaluation, using neuron coverage as a measure of test adequacy. This improves convolutional neural network security evaluation technology, helps to accurately grasp model performance, provides important references for model development and application, and addresses the security issues of artificial intelligence models in real-world environments. Summary of the Invention

[0010] To address the aforementioned technical issues, this invention proposes a method for calculating the main modality neuron coverage of image convolutional neural network models, including image classification, object detection, and image segmentation. This method is used to measure the adequacy of model testing, conduct more comprehensive and complete security assessments of image convolutional neural network models, and obtain model security assessment results that are more consistent with actual application scenarios.

[0011] The present invention specifically adopts the following technical solution:

[0012] A security evaluation method for image convolutional neural network models based on principal modality neuron coverage includes the following steps:

[0013] Step 1: Analyze the structure of the image convolutional neural network model; during the initial run of the model,

[0014] The depth-first traversal technique is used to traverse each layer of the model from top to bottom, saving the name and type of each layer, as well as the size of the output feature map of the convolutional layer, the activation function and the parameter information of the max pooling layer;

[0015] Step 2: Set instrumentation points for model layers; During the process of traversing the detection model layers using depth-first search, instrumentation points are set for convolutional layers and max pooling layers using hook functions. Code is added to the hook functions to save the input / output feature map data of the convolutional layers and max pooling layers.

[0016] Step 3: Perform model safety testing using the test set used for model safety evaluation; input the test set used for model safety evaluation into the model batch by batch, and obtain the model output results. During this process, the input / output feature map data of the convolutional layer and max pooling layer after the current batch of image data is input into the model is obtained batch by batch through the instrumentation points set in Step 2. This allows for the calculation of the input / output feature map data of the current batch of data into the model.

[0017] The coverage rate of the main modality neurons in the model after modeling. This is obtained after the original test set is input into the model in batches.

[0018] The coverage of the main modality neurons of the model under the original test set input;

[0019] Step 4: Generation of Coverage-Driven Model Security Test Samples; Using images from the original test set for model security evaluation as seed data, perform perturbation attacks or other attacks on the images to generate images that meet the requirements.

[0020] To meet the requirements for diversity and to expand the model safety evaluation test set with samples that have higher coverage of main modality neurons;

[0021] Step 5: Conduct model security assessments using the expanded model security test set to obtain more comprehensive model security assessment results.

[0022] Preferably, in step one, the activation function and max pooling layer parameter information specifically refer to the pooling kernel size, pooling kernel stride, number of zeros padded to each edge of the input feature map of the max pooling layer, pooling kernel element spacing, and activation function type after each convolutional layer.

[0023] Preferably, the specific implementation of step two is as follows: when the model is first run, starting from the model entry point, a depth-first traversal technique is used to sequentially traverse downwards to detect the model layer type and name, and assign numbers; if the layer is identified as a container type layer, that is, a container type layer is an ordered / unordered container composed of multiple layers, then the layer traversal detection is performed inside the container until there is no next layer; otherwise, the layer traversal detection continues downwards; during the traversal process, instrumentation is performed to bind hook functions to each layer, which are used to obtain the input / output feature map data of each layer after the model input data.

[0024] Preferably, in step three, the specific method is to obtain the image test set used for the security evaluation of the image convolutional neural network model, input N images into the model in each batch, and obtain the model output results; during the process, the hook function bound to each layer will save the input / output feature map data of each layer of the current batch of the model to a temporary variable area for the calculation of the main modality neuron coverage; the specific steps for calculating the main modality neuron coverage are:

[0025] Step 3.1: Obtain the newly covered main modality neurons of the convolutional layer after inputting the current batch of data into the model; for the convolutional layer with layer number i, its output feature map has a size of Four-dimensional tensor FeaturesOut i Where N refers to the number of images in each batch. The number of channels in the output feature map of the convolutional layer with index i. The height of the output feature map of the convolutional layer with index i. This refers to the width of the output feature map of the convolutional layer with index i. Each data point in this output feature map represents a neuron, and there are a total of N×... One neuron;

[0026] If the information obtained in step one indicates that an activation function exists subsequently in convolutional layer i, use the symbol σ. i (·) indicates that FeatureOut is needed to obtain the covered neurons in the convolutional layer. i After the activation function σ i (·) The processed feature map data, i.e., FeatureOut i =σ i (FeaturesOut i );

[0027] For FeaturesOut i Summing the first dimension yields a size of The three-dimensional tensor, which is about to The neurons are merged into one through a summation operation. One neuron; continue adjusting the tensor shape to make it of size [size missing]. The two-dimensional tensor still has the number of neurons. Obtain the two-dimensional tensor channel The sum of neuron data i,j Next, obtain the top k largest neurons in channel j of the current batch of input data, which satisfy the following formula:

[0028]

[0029]

[0030] In the formula, This represents the sum of data from the top k largest neurons in channel j of convolutional layer i; ratio is a percentage coefficient, which can be taken from 0% to 100% of the data, excluding 0%. The top k largest neurons in channel j are then calculated as the "covered neurons," which play a dominant modality role. The remaining neurons that interfere with the model's classification, detection, and segmentation tasks are discarded, forming the CovNeuronSet, the set of covered neurons in channel j of convolutional layer i. i,j The set stores the sequence numbers of the covered neurons.

[0031] Step 3.2: Obtain the new main modality neurons covered by the max pooling layer after inputting the current batch of data into the model; based on the information of max pooling layer i obtained in Step 1, modify the input feature map of max pooling layer i obtained in Step 2. i Perform max pooling again to obtain the neuron indices selected by the max pooling kernel in each channel of max pooling layer i, forming the covered neuron set CovNeuronSet. i,j These selected neurons are the main modality neurons, while the unselected neurons interfere with the model's classification, detection, and segmentation tasks and are not included in the set of covered neurons.

[0032] Step 3.3: Merge the current batch of covered neurons into the global set of covered neurons. The set of covered neurons obtained in steps 3.1-3.2 for the current batch of input data is merged into the global set of covered neurons after all previous batches of data have been input into the model. Right now Duplicate neuron indices are not retained in the set; when the first batch of data has not yet been input into the model, the model is globally covered by the set of neurons. Empty;

[0033] Step 3.4: Obtain the latest principal modality neuron coverage calculation results for each convolutional layer and max pooling layer;

[0034] The coverage rate of the main modality neurons in channel j of convolutional layer i after inputting the current batch of data into the model is calculated using the formula shown below:

[0035]

[0036] In the formula, len(·) is a function to obtain the length of the set;

[0037] Based on this, the coverage rate of the main modality neurons in each channel of the convolutional layer is averaged to obtain the average coverage rate of the main modality neurons in convolutional layer i, as shown in the following formula:

[0038]

[0039] The coverage of the dominant modality neurons in channel j of the max-pooling layer is calculated using the formula shown below:

[0040]

[0041] In the formula, The height of the input feature map for max pooling layer i. is the width of the input feature map for max pooling layer i.

[0042] Based on this, the average coverage rate of the main modality neurons in each channel of the max pooling layer is calculated to obtain the average coverage rate of the main modality neurons in max pooling layer i, as shown below:

[0043]

[0044] In the formula, The number of channels in the input feature map for max pooling layer i;

[0045] Step 3.5: Obtain the overall principal modality neuron coverage of the convolutional neural network model; average the principal modality neuron coverage of all convolutional layers and max-pooling layers of the convolutional neural network to obtain the overall principal modality neuron coverage Cov of the model, i.e. Where n is the total number of convolutional layers and max pooling layers in the model.

[0046] Preferably, step four involves the following steps:

[0047] Step 4.1: Use all batches of image data from the original test set used in Step 3 above as seed data to form a seed data sequence S;

[0048] Step 4.2: Pop the last data of the seed data sequence and use it as the original seed data x for this sample generation operation;

[0049] Step 4.3: Randomly select a perturbation attack method or other attack methods to attack x and generate a new sample x′;

[0050] Step 4.4: Calculate the average L2 distance between x′ and the pixels of image x, and the coverage of the main modality neurons of the model after x′ is input into the model. If the average L2 distance exceeds the set maximum value or the coverage does not increase, it means that the new batch of samples is invalid, and the number of attempts is incremented by 1. If the maximum number of attempts Max_iter is not exceeded, return to step 4.3 to continue using x for perturbation mutation; otherwise, return to step 4.2. If the average L2 distance is less than the set maximum value and the coverage increases, it means that the new sample x′ is a valid sample, add it to the end of sequence S, and return to step 4.2.

[0051] Step 4.5 When sequence S is empty, save the valid samples generated in step 4.4, and combine them with the original test set to form the expanded security evaluation test set for model security evaluation.

[0052] This invention has the following advantages and positive effects:

[0053] 1. To facilitate the calculation of the main modality neuron coverage of an image convolutional neural network model, this invention proposes a method for obtaining model hierarchical information and dynamic instrumentation based on depth-first traversal and Hook functions. This method does not require modification of the original hierarchical structure and code of the model, nor does it require human intervention, making it more suitable for large-scale model convolutional neural network testing tasks.

[0054] 2. Based on the analysis of different hierarchical structures of convolutional neural network models, this invention proposes a criterion and method for calculating the coverage rate of the main modality neurons in image convolutional neural networks, suitable for image classification, object detection, and image segmentation. The coverage metric proposed in this invention has a finer granularity than the neuron coverage in DeepXplore, unlike DeepXplore which can easily achieve 100% coverage with just a few batches of samples. It is simpler to understand and calculate than the metric in DeepCover and DeepCT, such as symbol-symbol coverage, distance-symbol coverage, symbol-value coverage, distance-value coverage, t-way combined sparse coverage, t-way combined dense coverage, and (p,t)-integrity coverage, and is easier to implement.

[0055] 3. Using the coverage rate of the main modality neurons as a measure of the adequacy of model security assessment, a more comprehensive and complete model security assessment method and process can be established, which can measure the security issues of the model when it is subjected to perturbation attacks in real-world usage scenarios.

[0056] 4. The coverage-driven model test sample generation technology proposed in this invention can flexibly use a variety of attack methods to generate samples, without limiting the specific attack methods, and is highly flexible. Attached Figure Description

[0057] The present invention includes the following figures:

[0058] Figure 1 This is a flowchart illustrating the method described in this invention;

[0059] Figure 2 Flowchart for calculating the coverage of the main modality neurons in a convolutional neural network model;

[0060] Figure 3 Flowchart for generating test samples for coverage-driven models;

[0061] Figure 4 The coverage of the main modality neurons in the model after the original test set and newly generated samples are input into the model;

[0062] Figure 5 This represents the average L2 distance between the pixels of the newly generated sample and the original seed image.

[0063] Figure 6 The model metrics of the SSD300 model are presented under the original security evaluation test set, the newly generated samples, and the expanded test set.

[0064] Figure 7 This section describes the perturbation attack methods and parameters used in the experiment.

[0065] Figure 8 The parameters for the perturbation attack in the experiment are set.

[0066] Figure 9 This refers to the settings of other parameters used in the experiment;

[0067] Figure 10 This is a comparison table of several AI model testing methods. Detailed Implementation

[0068] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them, and do not constitute a limitation on the present invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the protection scope of the present invention.

[0069] This invention proposes a security evaluation method for image convolutional neural networks based on the coverage of the main modality neurons. The specific steps of this method are as follows:

[0070] Step 1: Analyze the structure of the image convolutional neural network model. During the initial run of the model, use a depth-first traversal technique to traverse each layer of the model from top to bottom, saving the name and type of each layer, as well as the size of the output feature map of the convolutional layer, the activation function, and the parameter information of the max pooling layer.

[0071] Step 2: Set instrumentation points for model layers. During the process of traversing the detection model layers using depth-first search, instrumentation points are set for convolutional layers and max-pooling layers using hook functions. Code is added to these hook functions to store the input / output feature map data of the convolutional and max-pooling layers.

[0072] Step 3: Perform model safety testing using the test set used for model safety evaluation. Input the test set for model safety evaluation into the model batch by batch, and obtain the model output results. During this process, the input / output feature map data of the convolutional and max-pooling layers after the current batch of image data is input into the model is obtained batch by batch through the instrumentation points set in Step 2. This allows for the calculation of the model's main modality neuron coverage after the current batch of data is input into the model. Once all batches of the original test set have been input into the model, the model's main modality neuron coverage under the original test set input is obtained.

[0073] Step 4: Generation of Coverage-Driven Model Security Test Samples. Using images from the original test set for model security evaluation as seed data, perturbation attacks or other attacks are performed on the images to generate samples that meet the image diversity requirements and have higher coverage of main modality neurons, thus expanding the model security evaluation test set.

[0074] Step 5: Conduct model security assessments using the expanded model security test set to obtain more comprehensive model security assessment results.

[0075] Through the above steps, using the original model security evaluation test set as seed data, a series of image perturbation attack methods were employed to generate a test set with higher coverage of main modality neurons, thereby conducting a more comprehensive security evaluation of the image recognition convolutional neural network model.

[0076] In step one above, "activation function and max pooling layer parameter information" specifically refers to the size of the max pooling kernel, the stride of the pooling kernel, the number of zeros padded to each edge of the input feature map of the pooling kernel, the element spacing of the pooling kernel, and the type of activation function after each convolutional layer.

[0077] Step two involves the following steps during the initial model run: starting from the model entry point, a depth-first traversal technique is used to sequentially detect and number the model's layer types and names. If a layer is identified as a container type (i.e., an ordered / unordered container composed of multiple layers), the layer traversal continues within that container until no further layers are found; otherwise, the traversal continues downwards. During this traversal, instrumentation is performed to bind hook functions to each layer, enabling the acquisition of input / output feature map data for each layer after the model receives input data.

[0078] In step three, the specific approach involves obtaining the image test set used for the security evaluation of the image convolutional neural network model. N images are input into the model in batches, and the model output is obtained. During this process, the hook functions bound to each layer save the input / output feature map data of each layer of the current batch model to a temporary variable region, which is used to calculate the coverage rate of the main modality neurons.

[0079] The specific steps for calculating the coverage of the dominant modality neurons are as follows: (see attached) Figure 2 )

[0080] Step 3.1 Obtain the newly covered main modality neurons in the convolutional layers after inputting the current batch of data into the model. For the convolutional layer with layer number i, its output feature map has a size of... Four-dimensional tensor FeaturesOut i Where N refers to the number of images in each batch. The number of channels in the output feature map of the convolutional layer with index i. The height of the output feature map of the convolutional layer with index i. This refers to the width of the output feature map of the convolutional layer with index i. Each data point in this output feature map represents a neuron, and there are a total of [number missing]. One neuron.

[0081] If the information obtained in step one indicates that an activation function exists after convolutional layer i, use the symbol σ. i (·) indicates that FeatureOut is needed to obtain the covered neurons in the convolutional layer. i After the activation function σ i (·) The processed feature map data, i.e., FeaturesOut i =σ i (FeaturesOut i ),

[0082] For FeaturesOut i Summing the first dimension yields a size of The three-dimensional tensor, which is about to The neurons are merged into one through a summation operation. One neuron. Continue adjusting the tensor shape to make it of size [size missing]. The two-dimensional tensor still has the number of neurons. Obtain the two-dimensional tensor channel The sum of neuron data i,j Next, the top k neurons in channel j of the current batch of input data are obtained, which satisfy the following formula:

[0083]

[0084]

[0085] In the formula, This represents the sum of data from the top k largest neurons in channel j of convolutional layer i. The ratio is a percentage coefficient, which can be taken from 0% to 100% (excluding 0%). The top k largest neurons in channel j are then calculated as the "covered neurons," which play a dominant modality role. The remaining neurons that interfere with the model's classification, detection, and segmentation tasks are discarded, forming the CovNeuronSet, the set of covered neurons in channel j of convolutional layer i. i,j The set stores the sequence number of the covered neurons.

[0086] Step 3.2: Obtain the new main modality neurons covered by the max pooling layer after inputting the current batch of data into the model. Based on the information of max pooling layer i obtained in Step 1, modify the input feature map of max pooling layer i obtained in Step 2. i Perform max pooling again to obtain the neuron indices selected by the max pooling kernel in each channel of max pooling layer i, forming the covered neuron set CovNeuronSet. i,j These selected neurons are the main modality neurons. Unselected neurons interfere with the model's classification, detection, and segmentation tasks and are not included in the set of covered neurons.

[0087] Step 3.3: Merge the current batch of covered neurons into the global set of covered neurons. The set of covered neurons obtained in steps 3.1-3.2 for the current batch of input data is merged into the global set of covered neurons after all previous batches of data have been input into the model. Right now Duplicate neuron indices are not retained in the set. Note that when the first batch of data has not yet been input into the model, the model is globally covered by the set of neurons. Empty.

[0088] Step 3.4: Obtain the latest main modality neuron coverage calculation results for each convolutional layer and max pooling layer.

[0089] The coverage rate of the main modality neurons in channel j of convolutional layer i after inputting the current batch of data into the model is calculated using the formula shown below:

[0090]

[0091] In the formula, len(·) is a function to get the length of the set.

[0092] Based on this, the coverage rate of the main modality neurons in each channel of the convolutional layer is averaged to obtain the average coverage rate of the main modality neurons in convolutional layer i, as shown in the following formula:

[0093]

[0094] The coverage of the dominant modality neurons in channel j of the max-pooling layer is calculated using the formula shown below:

[0095]

[0096] In the formula, The height of the input feature map for max pooling layer i. is the width of the input feature map for max pooling layer i.

[0097] Based on this, the average coverage rate of the main modality neurons in each channel of the max pooling layer is calculated to obtain the average coverage rate of the main modality neurons in max pooling layer i, as shown below:

[0098]

[0099] In the formula, The number of channels of the input feature map for max pooling layer i

[0100] Step 3.5: Obtain the overall principal modality neuron coverage of the convolutional neural network model. Average the principal modality neuron coverage of all convolutional layers and max-pooling layers in the convolutional neural network to obtain the overall principal modality neuron coverage (Cov) of the model. Where n is the total number of convolutional layers and max pooling layers in the model.

[0101] The specific steps for step four are as follows: (see attached) Figure 3 )

[0102] Step 4.1 Use all batches of image data from the original test set used in Step 3 above as seed data to form a batch image sequence as seed data sequence S.

[0103] Step 4.2 Pop the last data of the seed data sequence as the original seed data x for this sample generation operation.

[0104] Step 4.3 Randomly select a perturbation attack method or other attack methods to attack x and generate a new sample x′.

[0105] After step 4.4, calculate the average L2 distance between x′ and the pixels of image x, as well as the coverage of the main modality neurons in the model after x′ is input into the model. If the average L2 distance exceeds the set maximum value or the coverage does not increase, it indicates that the new batch of samples is invalid, and the number of attempts is incremented by 1. If the maximum number of attempts, Max_iter, is not exceeded, return to step 4.3 to continue using x for perturbation and mutation; otherwise, return to step 4.2. If the average L2 distance is less than the set maximum value and the coverage increases, it indicates that the new sample x′ is a valid sample, and it is added to the end of sequence S, returning to step 4.2.

[0106] Step 4.5 When sequence S is empty, save the valid samples generated in step 4.4, and combine them with the original test set to form an expanded security evaluation test set for model security evaluation, so as to achieve a more comprehensive and complete security evaluation.

[0107] The practical effects of this invention are illustrated using the SSD300 model and the VOC2007 dataset as examples.

[0108] Figure 7 The perturbation attack method used in the experiment and the parameter descriptions are given.

[0109] Figure 8 The parameter settings for the perturbation attack in the experiment are given.

[0110] Figure 9 Provide the settings for other parameters used in the experiment.

[0111] Figure 10 A comparison table of several AI model testing methods is provided.

[0112] Appendix Figure 4 Given the original test set and newly generated samples as inputs into the model, the coverage of the main modality neurons in the model is analyzed.

[0113] Appendix Figure 5 The average L2 distance between the newly generated samples and the original seed image pixels is given. It can be seen that the average pixel L2 distance is only 0.017, a very small difference that is difficult for the human eye to distinguish and will not affect image recognition.

[0114] Appendix Figure 6This paper presents the model performance metrics of the SSD300 model under the original security evaluation test set, the newly generated samples, and the expanded test set, to measure the model's security performance under perturbation attacks. It can be seen that the accuracy of the SSD300 model decreases significantly on the expanded set, and the mAP value of the SSD300 model decreases significantly by more than 4%, indicating that the model's security against perturbation attacks is weak.

[0115] To facilitate the calculation of the main modality neuron coverage of image convolutional neural network models, this invention proposes a method for obtaining model hierarchical information and dynamic instrumentation based on depth-first traversal and Hook functions. This method does not require modification of the original hierarchical structure and code of the model, minimizes human intervention, and is more suitable for large-scale model convolutional neural network testing tasks.

[0116] This invention, starting from the structural characteristics and hierarchical functions of convolutional neural networks, formulates a criterion for calculating the coverage rate of principal modal neurons in convolutional neural networks specifically applicable to image classification, object detection, and image segmentation. The proposed method for calculating the coverage rate of principal modal neurons in convolutional layers, based on neuron output and proportion, is proposed after comprehensively considering the significance of the convolutional layer's role and the disadvantages of artificial neural network models such as DeepGauge and DeepCon. It is closer to how humans process image recognition tasks such as image classification, object detection, and image segmentation, and is more suitable for testing image convolutional neural network models.

[0117] In calculating the coverage of master mode neurons in the max pooling layer, considering the "selection" effect of the max pooling kernel on the input feature map data (neurons), the selected neurons are regarded as the covered master mode neurons. No similar technical solutions have been seen in other neuron coverage studies.

[0118] This invention is based on the perspective of test adequacy, using the coverage rate of the main modality neurons as a adequacy metric to generate a test sample set with higher coverage for security evaluation, thereby obtaining more practical results. The overall approach is particularly crucial.

[0119] The above description is merely a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims. The information disclosed in the background section is intended only to enhance the understanding of the overall background technology of the present invention and should not be construed as an admission or implication in any way that such information constitutes prior art known to those skilled in the art.

Claims

1. A method for security evaluation of image convolutional neural network models based on principal modality neuron coverage, characterized in that, Includes the following steps: Step 1: Analyze the structure of the image convolutional neural network model; during the first run of the model, use the depth-first traversal technique to traverse each layer of the model from top to bottom, and save the name and type of each layer, as well as the size of the output feature map of the convolutional layer, the activation function and the parameter information of the max pooling layer; Step 2: Set instrumentation points for model layers; During the process of traversing the detection model layers using depth-first search, instrumentation points are set for convolutional layers and max pooling layers using hook functions. Code is added to the hook functions to save the input / output feature map data of the convolutional layers and max pooling layers. Step 3: Perform model safety testing using the test set used for model safety evaluation; input the test set used for model safety evaluation into the model batch by batch, and obtain the model output results. During this process, the input / output feature map data of the convolutional layer and max pooling layer after the current batch of image data is input into the model is obtained batch by batch through the instrumentation points set in Step 2. The coverage rate of the main modality neurons of the model after the current batch of data is input into the model is calculated. After all the original test sets are input into the model in batches, the coverage rate of the main modality neurons of the model under the input of the original test sets is obtained. Step 4: Generation of Coverage-Driven Model Security Test Samples; Using images from the original test set of the model security evaluation as seed data, perturbation attacks are performed on the images to generate samples that meet the image difference requirements and have higher coverage of main modality neurons, thereby expanding the model security evaluation test set. Step 5: Conduct model security assessments using the expanded model security test set to obtain more comprehensive model security assessment results.

2. The security evaluation method for an image convolutional neural network model based on principal modality neuron coverage as described in claim 1, characterized in that, In step one, the activation function and max pooling layer parameter information specifically refer to the pooling kernel size, pooling kernel stride, number of zeros padded on each edge of the input feature map of the max pooling layer, pooling kernel element spacing, and activation function type after each convolutional layer.

3. The security evaluation method for an image convolutional neural network model based on principal modality neuron coverage as described in claim 1, characterized in that, The specific steps of step two are as follows: When the model is run for the first time, starting from the model entry point, a depth-first traversal technique is used to sequentially traverse downwards to detect the model's layer type and name, and assign them numbers; if a layer is identified as a container type layer, that is, a container type layer is an ordered / unordered container composed of multiple layers, then the layer traversal detection is performed inside the container until there is no next layer; otherwise, the layer traversal detection continues downwards; during the traversal process, instrumentation is performed to bind hook functions to each layer, which are used to obtain the input / output feature map data of each layer after the model inputs data.

4. The security evaluation method for an image convolutional neural network model based on principal modality neuron coverage as described in claim 1, characterized in that, In step three, the specific method is to obtain the image test set used for the security evaluation of the image convolutional neural network model, input N images into the model in each batch, and obtain the model output results. During the process, the hook function bound to each layer saves the input / output feature map data of each layer of the current batch model to a temporary variable region for calculating the neuron coverage rate; the specific steps for calculating the main modality neuron coverage rate are as follows: Step 3.1: Obtain the newly covered main modality neurons of the convolutional layer after inputting the current batch of data into the model; for the convolutional layer with layer number i, its output feature map has a size of Four-dimensional tensor FeaturesOut i Where N refers to the number of images in each batch. The number of channels in the output feature map of the convolutional layer with index i. The height of the output feature map of the convolutional layer with index i. This refers to the width of the output feature map of the convolutional layer with index i. Each data point in this output feature map represents a neuron, and there are a total of [number missing]. One neuron; If the information obtained in step one indicates that an activation function exists subsequently in convolutional layer i, use the symbol σ. i (·) indicates that FeatureOut is needed to obtain the covered neurons in the convolutional layer. i After the activation function σ i (·) The processed feature map data, i.e., FeaturesOut i =σ i (FeaturesOut i ); For FeaturesOut i Summing the first dimension yields a size of The three-dimensional tensor, which is about to The neurons are merged into one through a summation operation. One neuron; Continue adjusting the tensor shape to make it have a size of The two-dimensional tensor still has the number of neurons. Obtain the two-dimensional tensor channel j The sum of neuron data i,j Next, obtain the top k largest neurons in channel j of the current batch of input data, which satisfy the following formula: In the formula, The sum of data from the top k largest neurons in channel j of convolutional layer i is represented by the expression "ratio". The "ratio" is a percentage coefficient, taking data within the range of 0% to 100% (excluding 0%). The calculated top k largest neurons in channel j are then considered "covered neurons," which function as the dominant modality. The remaining neurons that interfere with the model's classification, detection, and segmentation tasks are discarded, forming the "CovNeuronSet" of covered neurons in channel j of convolutional layer i. i,j The set stores the sequence numbers of the covered neurons. Step 3.2: Obtain the new dominant modality neurons covered by the max pooling layer after the current batch of data is input into the model; Based on the information of the max pooling layer i obtained in step one, the input feature map of the max pooling layer i obtained in step two is... i Perform max pooling again to obtain the neuron indices selected by the max pooling kernel in each channel of max pooling layer i, forming the covered neuron set CovNeuronSet. i,j These selected neurons are the main modality neurons, while the unselected neurons interfere with the model's classification, detection, and segmentation tasks and are not included in the set of covered neurons. Step 3.3: Merge the current batch of covered neurons into the global set of covered neurons. The set of covered neurons obtained in steps 3.1 to 3.2 for the current batch of input data is merged into the global set of covered neurons after all previous batches of data have been input into the model. Right now Duplicate neuron indices are not retained in the set; when the first batch of data has not yet been input into the model, the model is globally covered by the set of neurons. Empty; Step 3.4: Obtain the latest principal modality neuron coverage calculation results for each convolutional layer and max pooling layer; The coverage rate of the main modality neurons in channel j of convolutional layer i after inputting the current batch of data into the model is calculated using the formula shown below: In the formula, len(·) is a function to obtain the length of the set; Based on this, the coverage rate of the main modality neurons in each channel of the convolutional layer is averaged to obtain the average coverage rate of the main modality neurons in convolutional layer i, as shown in the following formula: The coverage of the dominant modality neurons in channel j of the max-pooling layer is calculated using the formula shown below: In the formula, The height of the input feature map for max pooling layer i. The width of the input feature map for max-pooling layer i; Based on this, the average coverage rate of the main modality neurons in each channel of the max pooling layer is calculated to obtain the average coverage rate of the main modality neurons in max pooling layer i, as shown below: In the formula, The number of channels in the input feature map for max pooling layer i; Step 3.5: Obtain the overall principal modality neuron coverage of the convolutional neural network model; average the principal modality neuron coverage of all convolutional layers and max-pooling layers of the convolutional neural network to obtain the overall principal modality neuron coverage Cov of the model, i.e. Where n is the total number of convolutional layers and max pooling layers in the model. Other types of layers in the model are not considered in the calculation of the coverage of the main modality neurons.

5. The security evaluation method for an image convolutional neural network model based on principal modality neuron coverage as described in claim 1, characterized in that, Step four involves the following steps: Step 4.1: Use all batches of image data from the original test set used in Step 3 above as seed data to form a seed data sequence S; Step 4.2: Pop the last data of the seed data sequence and use it as the original seed image data x for this sample generation operation; Step 4.3: Randomly select a perturbation attack method to attack x and generate a new sample x. ′ ; Step 4.4: Calculate x ′ The average L2 distance between pixels in image x and x and in x ′ The coverage rate of the main modality neurons after inputting into the model is calculated. If the average L2 distance exceeds the set maximum value or the coverage rate does not increase, it indicates that the new batch of samples is invalid, and the number of attempts is incremented by 1. If the number of attempts does not exceed the maximum number of attempts Max_iter, return to step 4.3 to continue using x for perturbation mutation; otherwise, return to step 4.

2. If the average L2 distance is less than the set maximum value and the coverage rate increases, it indicates that the new sample x′ is a valid sample, and it is added to the end of sequence S, returning to step 4.

2. Step 4.5 When sequence S is empty, save the valid samples generated in step 4.4, and combine them with the original test set to form the expanded security evaluation test set for model security evaluation.

Images