Deep learning neural network robustness adaptive verification method and platform

Abstract

The invention discloses a neural network robustness adaptive verification method and platform for deep learning. According to the method, a sample attack algorithm, an approximation algorithm and a precision algorithm are combined, and an optimal algorithm combination and verification timeout duration are generated by constructing a self-adaptive verification strategy generation model. And analyzing the features of the neural network model as the input of the model to ensure the balance between the verification efficiency and the verification result. According to the method, for a time sequence task, a data set suitable for a multi-layer perceptron and a convolutional neural network model is constructed by using methods of recombining a data set structure, introducing a Grubrum angle field and the like. Meanwhile, the invention provides a method for converting a long short-term memory neural network model into an equivalent LSTMCELL model so as to support robustness verification of the LSTM. And the robustness verification efficiency and applicability of the neural network model are remarkably improved. The method is suitable for time sequence tasks such as financial market risk prediction and other safety key fields needing to ensure the robustness of the neural network.

Description

Technical Field

[0001] The present invention belongs to the field of artificial intelligence, and in particular relates to a method and platform for adaptively verifying the robustness of a neural network using deep learning. Background Art

[0002] With the rapid development and widespread application of artificial intelligence technology, the performance of neural networks in various tasks such as image recognition, speech recognition, and natural language processing has been significantly improved. In the field of financial technology, especially in complex time series tasks such as stock market risk prediction, neural networks have gradually become an important tool. Through deep learning models to analyze and process large amounts of historical data, neural networks can capture implicit patterns and trends in financial markets and effectively predict stock prices and assess risks. However, despite the powerful learning and prediction capabilities of neural networks, their robustness issues have attracted widespread attention in application scenarios involving property and personal safety.

[0003] Current research on robustness verification of neural networks mainly focuses on verification algorithms based on boundary approximation, which have made important progress in the robustness verification of large neural networks. Common methods include the use of convex optimization, symbolic interval propagation, Lagrange multiplier constraint solving, and branch-and-bound techniques, which approximate the robustness boundary of neural networks in different ways. However, these methods also have some limitations: although the computational complexity is lower than that of the exact method, the verification ability under Luban is poor. On the contrary, exact algorithms often require a lot of computing resources, especially when dealing with large deep neural networks with millions of parameters, the computational cost will increase significantly. Therefore, in small neural networks, the efficiency of approximate methods is usually not as efficient as that of exact methods, but in large neural networks, approximate algorithms can save a lot of verification time. A single algorithm cannot meet the needs of real-time applications. Combining approximate and exact algorithms can reduce verification time while ensuring verification performance.

[0004] On the other hand, most of the existing robustness verification research focuses on feedforward neural networks (FNN) and convolutional neural networks (CNN). These network structures perform well in processing static images and non-time series data, but have limitations when dealing with time series tasks (such as stock market prediction, sensor data processing, speech signal processing, etc.). The data in time series tasks usually have complex time dependencies and high noise characteristics, which puts higher requirements on the robustness of neural network models. There is relatively insufficient research on robustness verification of neural networks for these tasks (such as long short-term memory networks LSTM), resulting in many challenges in robustness verification in these fields. Summary of the invention

[0005] The present invention proposes an adaptive verification method, which combines sample attack algorithms, approximate algorithms and exact algorithms to construct an adaptive verification model AGTN (Algorithm Group-Timeout Networks) to automatically select the optimal verification algorithm combination and timeout duration. This method improves the verification efficiency and the effect of robustness verification, and is applicable to neural network models of various types and sizes.

[0006] The specific technical solution for achieving the purpose of the present invention is:

[0007] A deep learning neural network robustness adaptive verification method includes the following steps:

[0008] Step 1: Build an adaptive verification strategy generation model based on deep learning;

[0009] Step 1.1: Train multiple MLP and CNN models with different numbers of neurons and layers, and change the hyperparameters to obtain models with different accuracy.

[0010] Step 1.2, generate model training data by performing robustness verification and analytical acquisition adaptive verification strategy on the trained MLP and CNN models;

[0011] Step 1.2.1, obtain the number of neurons in the fully connected layer, the number of layers in the fully connected layer, the number of neurons in the convolutional layer, the number of layers in the convolutional layer and the accuracy of the neural network model as feature values;

[0012] Step 1.2.2: Under the set timeout period, use different verification algorithm combinations to perform robustness verification on the trained model, and record the algorithm combination with the largest number of successful verification samples. If multiple algorithm combinations obtain the same number of successful verification samples, record the algorithm combination with the shortest total verification time; at the same time, record the longest verification time when a single sample is successfully verified under the algorithm as the timeout period;

[0013] Step 1.2.3, the input feature values ​​of the adaptive verification strategy generation model are the number of neurons in the fully connected layer, the number of layers in the fully connected layer, the number of neurons in the convolutional layer, the number of layers in the convolutional layer, and the accuracy rate; the target classification label is the algorithm combination and the timeout duration;

[0014] Step 1.3: construct an adaptive verification strategy generation model, which is composed of an algorithm combination and a timeout duration strategy generation model;

[0015] Step 1.3.1, construct an algorithm combination strategy generation model based on a deep learning feedforward neural network. The model input feature values ​​are the number of neurons in the fully connected layer, the number of layers in the fully connected layer, the number of neurons in the convolutional layer, the number of layers in the convolutional layer and the accuracy rate. The output is the algorithm combination classification.

[0016] Step 1.3.2, construct a timeout duration strategy generation model based on a deep learning feedforward neural network, the model input feature values ​​are the number of neurons in the fully connected layer, the number of layers in the fully connected layer, the number of neurons in the convolutional layer, the number of layers in the convolutional layer, the accuracy rate, and the algorithm combination classification output by the algorithm combination strategy generation model; the output is the timeout duration;

[0017] Step 1.4: Train the adaptive verification strategy generation model based on the collected data set, and provide it for subsequent generation of adaptive strategies;

[0018] Step 2: Perform analytical evaluation on any neural network model to be verified for robustness, and determine the input feature value of the adaptive verification strategy generation model based on the analytical evaluation result;

[0019] Step 2.1, according to the architecture of the neural network model to be verified, obtain the number of neurons in the fully connected layer, the number of layers in the fully connected layer, the number of neurons in the convolutional layer, and the number of layers in the convolutional layer;

[0020] Step 2.2: Use the test sample to evaluate the accuracy of the neural network model to be verified;

[0021] Step 2.3, combining the number of neurons in the fully connected layer, the number of layers in the fully connected layer, the number of neurons in the convolutional layer, the number of layers in the convolutional layer, and the precision rate, to construct the input of the adaptive verification strategy generation model;

[0022] Step 3: Based on the constructed input, call the adaptive verification strategy generation model to generate an adaptive verification strategy, which includes a verification algorithm combination and a timeout duration;

[0023] Step 3.1, if the model to be verified is LSTM, generate a robustness verification strategy for LSTM, which is a fixed algorithm combination and timeout duration that supports robustness verification of the LSTMCELL model;

[0024] Step 3.2: If it is an MLP or CNN model, call the adaptive verification strategy generation model to generate an adaptive verification strategy, which includes a verification algorithm combination and a timeout duration;

[0025] Step 4: Perform robustness verification on the neural network model to be verified according to the generated robustness verification strategy;

[0026] Step 4.1, determine whether it is an LSTM model, if so, convert the LSTM to an equivalent LSTMCELL model;

[0027] Step 4.1.1, read the LSTM python file and generate the corresponding abstract syntax tree AST object;

[0028] Step 4.1.2, locate the LSTM layer code in the syntax tree, replace it with the equivalent LSTMCELL code according to the LSTM parameters, and save it as a temporary LSTMCELL python file;

[0029] Step 4.1.3, load the LSTM and LSTMCELL models, and assign the weight parameters of the LSTM model to the LSTMCELL model;

[0030] Step 4.2, perform robustness verification of multiple algorithm combinations on the neural network model to be verified according to the generated robustness verification strategy;

[0031] Step 4.3: Output the verification results, including the number of safe samples, the number of unsafe samples, and the number of unknown samples.

[0032] Furthermore, the robustness verification of the multi-algorithm combination includes the following steps:

[0033] Step 1: Load the neural network model to be verified;

[0034] Step 2: Load the test sample of the neural network model to be verified;

[0035] Step 3. The verification algorithm combination includes a sample adversarial verification algorithm (such as projected gradient descent PGD), an approximate verification algorithm (such as α-CROWN), and an exact verification algorithm (such as mixed integer programming MIP); the combination includes a single algorithm or two or three algorithms at the same time; the combination order is adversarial verification algorithm-approximate verification algorithm-exact verification algorithm; during verification, they are called from left to right. If the verification is successful, the result is returned in advance, or if the set timeout period is exceeded, the verification of the sample is abandoned and the next test sample is verified.

[0036] A neural network robustness adaptive verification platform based on the above method comprises:

[0037] The neural network model import management module reads the onnx or pytorch model files saved by the user in the platform specified path and generates a neural network model selection list; the onnx model files are converted into pytorch model objects for subsequent verification, and the pytorch model files provide the platform required methods in the code to call and return the pytorch model objects;

[0038] The data set import management module reads the user's custom verification data set saved in the platform's specified path and generates a custom Data set selection list; custom data set code gives the system specified class to generate the data set, the required graph sheet or csv Put it in the same folder;

[0039] Algorithm combination - timeout duration generation module, parses and evaluates the neural network model to be verified, obtains the input feature value of the adaptive verification strategy generation model, uses the adaptive verification strategy generation model to generate a robust adaptive verification strategy, and updates the corresponding configuration items of the platform;

[0040] The robustness verification module generates a configuration file based on the configuration items, and then calls the corresponding algorithm combination based on the configuration file. It cyclically verifies all verification samples under the set timeout duration and disturbance value, and returns the verification results, including the number of safe samples, the number of unsafe samples, and the number of unknown samples.

[0041] Adversarial training module improves the robustness of the neural network model through PGD adversarial training.

[0042] The method of the present invention predicts and adaptively verifies robustness of time series tasks under MLP and CNN, including:

[0043] The dataset of the MLP model is spliced ​​into a one-dimensional tensor with the help of reorganization;

[0044] The CNN model uses the Gramian Angular Field (GAF) method to construct a dataset of image shapes;

[0045] A neural network robustness adaptive verification method based on deep learning is used to generate adaptive verification strategies and perform robustness verification.

[0046] This paper proposes an adaptive neural network robustness verification method, which combines the advantages of sample attack algorithm approximation algorithm and exact algorithm, and constructs an adaptive verification strategy generation model AGTN (Algorithm Group-Timeout Networks), which automatically selects the optimal verification algorithm combination and timeout duration according to the different characteristics of the neural network to balance the computational cost and verification effect. This method is particularly suitable for time series tasks such as stock market risk prediction, and can effectively improve the robustness verification efficiency and accuracy of neural networks, providing strong technical support for applications in financial technology and other security-critical fields.

[0047] The present invention realizes adaptive robustness verification of neural network models through adaptive verification strategy generation model, combines approximate and precise algorithms, and improves the efficiency and accuracy of robustness verification. At the same time, the present invention enhances the applicability of verification through the conversion from LSTM to LSTMCELL, solves the problem that traditional verification methods are not effective in time series tasks, and is particularly suitable for safety-critical fields such as stock market risk prediction. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 is a flow chart of the adaptive neural network robustness verification method of the present invention;

[0049] Figure 2 Schematic diagram of the adaptive verification strategy generation model architecture;

[0050] Figure 3 Schematic diagram of the equivalent conversion process from LSTM to LSTMCELL;

[0051] Figure 4 Schematic diagram of the LSTM model robustness verification process;

[0052] Figure 5 Overall architecture diagram of the neural network robustness adaptive verification platform for deep learning. DETAILED DESCRIPTION

[0053] The specific implementation modes of the present invention are described in detail below with reference to the accompanying drawings.

[0054] Example

[0055] like Figure 1 As shown, the method of the present invention may include the following 4 steps:

[0056] Step S101, constructing and training an adaptive verification strategy generation model AGTN used in an adaptive neural network robustness verification method, and providing subsequent calls;

[0057] Step S102, training the stock market prediction neural network to be verified. Analyze and evaluate the neural network model to be verified, and obtain input feature values ​​for building an adaptive verification strategy generation model;

[0058] Step S103, using the adaptive verification strategy generation model to generate an adaptive robustness verification strategy;

[0059] Step S104, performing robustness verification according to the algorithm combination and timeout duration of the robustness verification strategy;

[0060] S101 builds and trains the AGTN (Algorithm Group-Timeout Networks) model. The specific steps are as follows:

[0061] Step S101-1, the AGTN model architecture is as follows Figure 2As shown in the figure, the model architecture constructs a multi-layer perceptron (MLP) neural network for the algorithm combination and the timeout period. Each MLP contains an input layer, an output layer, and a hidden fully connected layer (with 20 parameters). In order to prevent the model from overfitting on a small data set, a random dropout layer (Dropout) is added. The input of the MLP model for the timeout period consists of the output of the MLP model of the algorithm combination and the feature values ​​of the neural network model to be verified. The input feature values ​​of the AGTN model include: the number of neurons in the fully connected layer, the number of layers in the fully connected layer, the number of neurons in the convolutional layer, the number of layers in the convolutional layer, and the accuracy. There are 4 types of output algorithm combinations: PGD-α-CROWN, PGD-αβ-CROWN, PGD-α-CROWN-MIP, and MIP. There are 5 categories for the timeout period: 0, 30, 60, 90, and 120 seconds.

[0062] Step S101-2, the data set of the training model can be verified by using 4 types of algorithm combinations for MLP models with different numbers of neurons, numbers of layers, and precision trained in MINST, and recording the number of successful verification samples and the longest duration of a single successful verification sample. The algorithm combination with the largest number of verification samples is taken as the optimal algorithm combination. If there are multiple algorithm combinations with the same number of successful verifications, the algorithm with the shortest total verification time is taken as the optimal algorithm combination. The longest duration of a single successful verification sample is rounded up to a timeout duration of 5 categories, and the rounded value is the optimal timeout duration. For CNN models, the training model can be used to generate training data sets under CIFAR-10.

[0063] Step S101-3, preprocess the data set, remove noise data and normalize it, and then use K-fold cross validation to train the algorithm combination strategy generation model and the timeout generation strategy generation model.

[0064] S102 is to train the neural network for stock market prediction to be verified. The specific steps for analyzing and evaluating the neural network model to be verified and obtaining the input feature values ​​for constructing the adaptive verification strategy generation model are as follows:

[0065] Step S102-1, construct a training data set for the stock market prediction neural network model. Use the AKShare library to obtain historical data of the A-share market, including daily opening price, closing price, highest price, lowest price, trading volume, transaction amount, amplitude, increase or decrease, increase or decrease amount, and turnover rate, etc. after removing feature values ​​irrelevant to the task. Set a classification label for the task. When the increase or decrease amount is negative or zero, it means that the stock price has fallen, and the label is set to 0; when the increase or decrease amount is positive, it means that the stock price has risen, and the label is set to 1. Clean the data, handle missing values ​​and outliers, and perform normalization to ensure the standardization of data input.

[0066] Step S102-2, construct an MLP data set, assuming x1, x2...x n is n recorded historical data, and the input of the neural network is p. The first sample is x1, x2...x p Reorganized one-dimensional tensor, labeled x p+1 The second sample is x2, x3...x p+1 , labelled x p+2 , and so on.

[0067] Step S102-3, construct the CNN data set. Through three steps: normalization, polar coordinate conversion, and construction of the GAF matrix

[0068] (1) Normalization: The 10 eigenvalues ​​of AKShare are normalized to [-1, 1] respectively to facilitate the polar coordinate conversion in the next step.

[0069]

[0070] Where t is the timestamp, ranging from [1,32], with a total of 32 records. k is the number of eigenvalues, ranging from [1,10], with a total of 10 eigenvalues.

[0071] (2) Polar coordinate conversion: After normalizing the data to the interval [-1, 1], each data point can be mapped to a polar coordinate angle φ, which is calculated using the arccosine function:

[0072]

[0073] here, Indicates the angle, is a normalized value in the interval [-1,1]. The purpose of angle mapping is to convert each point of the time series into polar coordinates, providing a nonlinear representation method.

[0074] (3) Construct the Gram angular field matrix: The GAF matrix is ​​constructed by combining the cosine values ​​of the polar coordinate angles.

[0075]

[0076] Where i, j represent In the position of the Gram angular field matrix, i is the row coordinate and j is the column coordinate. k is Position of the channels in the final dataset tensor.

[0077] Step 102-3, training the MLP, CNN, and LSTM models for stock market prediction.

[0078] Step S102-4, load the model and determine the model type. If it is MLP and CNN, continue to call the AGTN model.

[0079] Step S102-5, by injecting hook and forward propagation, the number of neurons in the fully connected layer, the number of neurons in the convolutional layer, the number of fully connected layers, and the number of convolutional layers are calculated.

[0080] Step S102-6, using the test data set to evaluate the neural network model to be verified to obtain the model accuracy.

[0081] Step S102-7, using the number of fully connected layer neurons, the number of convolutional layer neurons, the number of fully connected layers and the number of convolutional layers and the accuracy obtained in the above steps, generate the AGTN model input tensor.

[0082] In S104, the specific steps of performing neural network robustness verification according to the adaptive algorithm combination and timeout duration output by AGTN are as follows:

[0083] Step S104-1, generate a verification configuration file according to the robustness verification strategy, the configuration file includes the algorithm combination, timeout duration, whether it is LSTM, disturbance value, neural network model path to be verified and verification number sample path.

[0084] Step S104-2, read the configuration file, and load the corresponding model to be verified and the verification sample.

[0085] Step S104-3, determine whether the neural network model to be verified is LSTM. LSTM is only supported by the PGD-α-CROWN method in the method, so it is necessary to add a check in the method to determine whether it is an LSTM model. If it is, directly use the PGD-α-CROWN algorithm combination, and set the timeout duration to 0. If not, use the AGTN model. The process is as follows Figure 4 .

[0086] Step S104-3-1, the conversion steps of LSTM are as follows Figure 3 As shown in the figure, this intermediate module parses the Python file of the LSTM model through AST to generate the corresponding syntax tree. After that, it traverses the nodes of the syntax tree, locates the definition and call of the nn.LSTM layer, and replaces it with the equivalent implementation of LSTMCELL. In this process, the processing method of the input data needs to be adjusted so that the output is calculated step by step, and the hidden state and cell state are manually updated. Finally, by further accessing the original LSTM model weight dictionary, the LSTM layer weight parameters are extracted and mapped to the LSTMCELL layer to maintain the functional consistency of the model.

[0087] Step S104-3-2, determine whether the model to be verified is MLP or CNN. If yes, use the algorithm combination and timeout duration obtained by AGTN to perform robustness verification.

[0088] Step S104-4, loop through the sample verification, call the algorithm combination verification for each sample individually, and record the verification results.

[0089] Step S104-5, first use the first algorithm in the algorithm combination to start verification, such as PGD-α-CROWN-MIP uses the first algorithm PGD for verification.

[0090] Step S104-6: If the first algorithm is successfully verified, the subsequent verification is skipped and the robustness verification of the next verification sample is started. If the first algorithm fails to be verified and the sample verification status is unknown, the second algorithm is used for verification (α-CROWN in PGD-α-CROWN-MIP). All algorithms in the algorithm combination are called in sequence until all are called. If the verification is still unsuccessful or the timeout period is exceeded, the sample is recorded as unknown and the next sample is verified.

[0091] Step 104-7, after all samples are verified, a summary is output, showing the number of safe, unsafe, and unknown samples.

[0092] The robust adaptive verification platform architecture design for deep learning is as follows Figure 5 As shown, it includes 5 functional modules and related basic services:

[0093] Neural network model import management module 1: The user places the neural network model file to be verified into the specified folder. The platform reads all neural network models to generate a list for subsequent selection, and supports importing neural network models in multiple formats (such as ONNX and PTH formats);

[0094] Dataset import management module 2: The user places the validation dataset file into the specified folder. The platform reads all validation datasets to generate a list for subsequent selection. It supports importing datasets in multiple formats (such as CSV, image files) and user-defined datasets;

[0095] Algorithm combination-timeout duration generation module 3: Use the AGTN model to parse the selected neural network model to be verified to obtain the AGTN model input feature value. Generate the verification strategy for the optimal verification algorithm combination and verification timeout duration;

[0096] Robustness verification module 4: executes the verification algorithm combination in the configuration file, performs robustness verification on the selected neural network model to be verified, and cyclically verifies the samples in the verification data set;

[0097] Adversarial training module 5: used to perform PGD adversarial training on the neural network model to enhance the robustness of the neural network model.

[0098] Basic services include the computing libraries auto_LiRPA and Gurobi required for verification, the onnx conversion service that provides onnx model file reading and loading functions, and the logger service that provides logging. All services and modules run on the conda environment.

Claims

1. A deep learning neural network robustness adaptive verification method, characterized in that: The following steps are involved: Step 1: Build an adaptive verification strategy generation model for deep learning; Step 1.1: Train multiple MLP and CNN models with different numbers of neurons and layers, and change the hyperparameters to obtain models with different accuracy. Step 1.2, generate model training data by performing robustness verification and analytical acquisition adaptive verification strategy on the trained MLP and CNN models; Step 1.2.1, obtain the number of neurons in the fully connected layer, the number of layers in the fully connected layer, the number of neurons in the convolutional layer, the number of layers in the convolutional layer and the accuracy of the neural network model as feature values; Step 1.2.2: Under the set timeout period, use different verification algorithm combinations to perform robustness verification on the trained model, and record the algorithm combination with the largest number of successful verification samples. If multiple algorithm combinations obtain the same number of successful verification samples, record the algorithm combination with the shortest total verification time; at the same time, record the longest verification time when a single sample is successfully verified under the algorithm as the timeout period; Step 1.2.3, the input feature values ​​of the adaptive verification strategy generation model are the number of neurons in the fully connected layer, the number of layers in the fully connected layer, the number of neurons in the convolutional layer, the number of layers in the convolutional layer, and the accuracy rate; the target classification label is the algorithm combination and the timeout duration; Step 1.3: construct an adaptive verification strategy generation model, which is composed of an algorithm combination and a timeout duration strategy generation model; Step 1.3.1, construct an algorithm combination strategy generation model based on a deep learning feedforward neural network. The model input feature values ​​are the number of neurons in the fully connected layer, the number of layers in the fully connected layer, the number of neurons in the convolutional layer, the number of layers in the convolutional layer and the accuracy rate. The output is the algorithm combination classification. Step 1.3.2, construct a timeout duration strategy generation model based on a deep learning feedforward neural network, the model input feature values ​​are the number of neurons in the fully connected layer, the number of layers in the fully connected layer, the number of neurons in the convolutional layer, the number of layers in the convolutional layer, the accuracy rate, and the algorithm combination classification output by the algorithm combination strategy generation model; the output is the timeout duration; Step 1.4: Train the adaptive verification strategy generation model based on the collected data set, and provide it for subsequent generation of adaptive strategies; Step 2: Perform analytical evaluation on any neural network model to be verified for robustness, and determine the input feature value of the adaptive verification strategy generation model based on the analytical evaluation result; Step 2.1, according to the architecture of the neural network model to be verified, obtain the number of neurons in the fully connected layer, the number of layers in the fully connected layer, the number of neurons in the convolutional layer, and the number of layers in the convolutional layer; Step 2.2: Use the test sample to evaluate the accuracy of the neural network model to be verified; Step 2.3, combining the number of neurons in the fully connected layer, the number of layers in the fully connected layer, the number of neurons in the convolutional layer, the number of layers in the convolutional layer, and the precision rate, to construct the input of the adaptive verification strategy generation model; Step 3: Based on the constructed input, call the adaptive verification strategy generation model to generate an adaptive verification strategy, which includes a verification algorithm combination and a timeout duration; Step 3.1, if the model to be verified is LSTM, generate a robustness verification strategy for LSTM, which is a fixed algorithm combination and timeout duration that supports robustness verification of the LSTMCELL model; Step 3.2: If it is an MLP or CNN model, call the adaptive verification strategy generation model to generate an adaptive verification strategy, which includes a verification algorithm combination and a timeout duration; Step 4: Perform robustness verification on the neural network model to be verified according to the generated robustness verification strategy; Step 4.1, determine whether it is an LSTM model, if so, convert the LSTM to an equivalent LSTMCELL model; Step 4.1.1, read the LSTM python file and generate the corresponding abstract syntax tree AST object; Step 4.1.2, locate the LSTM layer code in the syntax tree, replace it with the equivalent LSTMCELL code according to the LSTM parameters, and save it as a temporary LSTMCELL python file; Step 4.1.3, load the LSTM and LSTMCELL models, and assign the weight parameters of the LSTM model to the LSTMCELL model; Step 4.2, perform robustness verification of multiple algorithm combinations on the neural network model to be verified according to the generated robustness verification strategy; Step 4.3: Output the verification results, including the number of safe samples, the number of unsafe samples, and the number of unknown samples.

2. The neural network robustness adaptive verification method according to claim 1, characterized in that: The robustness verification of the multi-algorithm combination includes the following steps: Step 1: Load the neural network model to be verified; Step 2: Load the test sample of the neural network model to be verified; Step 3. The verification algorithm combination includes a sample adversarial verification algorithm, an approximate verification algorithm, and an exact verification algorithm; the combination includes a single algorithm or two or three algorithms at the same time; the combination order is adversarial verification algorithm-approximate verification algorithm-exact verification algorithm; during verification, they are called from left to right. If the verification is successful, the result is returned in advance, or if the set timeout period is exceeded, the verification of the sample is abandoned and the next test sample is verified.

3. A neural network robustness adaptive verification platform based on the method of claim 1, characterized in that: include: The neural network model import management module reads the onnx or pytorch model files saved by the user in the platform specified path and generates a neural network model selection list; the onnx model files are converted into pytorch model objects for subsequent verification, and the pytorch model files provide the platform required methods in the code to call and return the pytorch model objects; The data set import management module reads the user-defined verification data set saved in the platform-specified path and generates a custom data set selection list; the custom data set code gives the system-specified class to call and generate the data set, and the required images or csv are placed in the same-level folder; Algorithm combination - timeout duration generation module, parses and evaluates the neural network model to be verified, obtains the input feature value of the adaptive verification strategy generation model, uses the adaptive verification strategy generation model to generate a robust adaptive verification strategy, and updates the corresponding configuration items of the platform; The robustness verification module generates a configuration file based on the configuration items, and then calls the corresponding algorithm combination based on the configuration file. It cyclically verifies all verification samples under the set timeout duration and disturbance value, and returns the verification results, including the number of safe samples, the number of unsafe samples, and the number of unknown samples. Adversarial training module improves the robustness of the neural network model through PGD adversarial training.

Images