A reliability prediction method for image classification neural network model

Abstract

The present invention proposes a reliability prediction method for an image classification neural network model, including training a reliability model, and then using the trained reliability model to perform reliability prediction on a neural network model to be predicted for image classification, wherein, according to the training and testing of the neural network model, the input features of the reliability model include a model training factor and a model testing factor, the model training factor characterizes the data and model factors that affect the reliability of the neural network, and the model testing factor is used to characterize the adequacy of the neural network test; the output of the reliability model is the reliability prediction result of the neural network model. According to the characteristics of the neural network model, the present invention decomposes and analyzes the influencing factors and key parameters of its reliability, and constructs a neural network model reliability prediction method based on support vector regression, which can effectively improve the accuracy, generalization and fault tolerance of the neural network model reliability prediction.

Description

Technical Field

[0001] The invention belongs to the technical field of computer software testing, and in particular relates to a reliability prediction method of a neural network model. Background Art

[0002] As a leading machine learning technology, neural networks play a vital role in contemporary science and technology and industrial applications. From image processing to identity authentication, autonomous driving, medical diagnosis, financial analysis and many other fields, neural network technology continues to expand its boundaries and solve a series of complex problems, showing its unique capabilities and wide applicability, and has gradually become the core technology of current intelligent software.

[0003] At the same time, challenges in the safety and reliability of neural networks are becoming increasingly prominent. For example, in self-driving cars, a tiny algorithm error or data deviation may lead to serious safety accidents; in medical diagnosis, an inaccurate prediction of the model may have irreversible effects on the patient's health. Therefore, improving the reliability of neural networks is not only a requirement to improve system performance, but also a necessary condition to ensure safety and protect lives. Reliability modeling of neural network models is crucial to ensuring the robustness and reliability of intelligent software systems in the real world.

[0004] At present, research on the reliability of intelligent software is insufficient, and research on the reliability of traditional software is mainly based on statistics and probability theory. Classic models include the non-homogeneous Poisson process model (NHPP) and other models based on time or fault data, which focus on predicting the future performance of software through historical fault data. These models usually assume that the statistical characteristics of software behavior are deterministic and consistent, but have limitations when dealing with complex and dynamic system environments. In addition, as the core of intelligent software, the programming paradigm of neural networks is different from that of traditional software, and their structure and operation mechanism are far more complex than traditional software. The behavior of neural network models is affected by many factors such as training data and model structure. The reliability measurement model established only with time or fault data is too simple and cannot meet the reliability measurement requirements of neural network models in terms of prediction accuracy and adaptability.

[0005] Reliability modeling of neural networks faces many difficulties, partly due to the complexity of their internal structure. A typical deep learning model may contain millions of parameters and multiple layers of nonlinear transformations, which makes it extremely difficult to analyze its behavior. In addition, the training process of neural networks itself is highly dynamic and unpredictable, relying on initial conditions and randomness. These characteristics make it difficult to apply traditional reliability modeling methods to neural networks, because these methods usually assume that the system behavior has a certain degree of statistical regularity or repeatability and can be predicted based on time. However, for neural networks, each prediction is independent and does not interfere with each other, and there will be no problems such as accumulation of faults over time, resulting in traditional time-based prediction techniques no longer being applicable. Summary of the invention

[0006] In view of this, the present invention analyzes the factors affecting the reliability of neural networks and their modeling methods, combines the chaos theory of traditional software reliability and the machine learning technology that can extract implicit features, and proposes a reliability prediction method for image classification neural network models.

[0007] The specific technical solutions are as follows:

[0008] A reliability prediction method for an image classification neural network model includes training a reliability model, and then using the trained reliability model to perform reliability prediction on a neural network model to be predicted for image classification, wherein, according to the training and testing of the neural network model, the input features of the reliability model include a model training factor and a model testing factor, the model training factor characterizes data and model factors that affect the reliability of the neural network, and the model testing factor is used to characterize the adequacy of the neural network test; the output of the reliability model is a reliability prediction result of the neural network model.

[0009] Furthermore, the model training factors include training data quality factors and model performance factors, wherein the training data quality factors include the size of the training data set and the distinguishability of the training data, wherein the size of the training data set reflects the adequacy of the training data, and the distinguishability of the training data reflects the complexity of the target task; the model performance factors include the amount of model parameters, model training loss, and the number of training iterations, wherein the amount of model parameters reflects the complexity of the model structure, and the model training loss reflects the model training effect; the number of training iterations determines the depth and degree of the model's learning from the data.

[0010] Furthermore, LiRPA calculates the distinguishability of training data based on linear relaxation perturbation analysis.

[0011] Furthermore, the process of calculating the distinguishability of training data includes,

[0012] Each sample in the training data set corresponds to a correct class, and LiRPA is used to calculate the confidence interval of each sample classified into each possible class under perturbation;

[0013] Calculate the probability of each sample being misclassified into each other possible class, where the probability of any sample being misclassified from the correct class a to the class b is equal to the intersection and union of the confidence interval corresponding to the correct class a and the confidence interval corresponding to the wrong class b, divided by 2;

[0014] Calculate the probability that all samples corresponding to the correct class a are misclassified into each other possible class and average them, which is recorded as the confusion degree of class a data;

[0015] The confusion degree of all classes of data is averaged to obtain the confusion degree of the entire training data set, which is the distinguishability of the training data.

[0016] Furthermore, the model training loss is the average of the losses of each sample in the training set.

[0017] Furthermore, the model testing factor is characterized by the test degree factor, which specifically includes the number of test iterations, the size of the test data set, and the model structure coverage; the number of test iterations and the size of the test data set directly reflect the depth of the model testing and the test cost; the model structure coverage is a test adequacy characterization indicator based on the neuron activation status.

[0018] Furthermore, neuron coverage is adopted as the model structure coverage indicator.

[0019] Furthermore, the reliability model adopts support vector regression SVR.

[0020] Furthermore, training the reliability model includes preparing a training data set, a test data set, and a neural network model set. All models in the neural network model set are trained on the training data set to obtain model training factors; all models in the neural network model set are tested on the test data set to obtain model testing factors; eight reliability influencing factor index values ​​formed by the model training factors and the model testing factors constitute the explanatory variables of the reliability model, and the dependent variable is the accuracy of the neural network model on the corresponding data set. Training the reliability model is used to establish the relationship between the explanatory variables and the dependent variables.

[0021] Furthermore, the trained reliability model is used to perform reliability prediction on the neural network model to be predicted for image classification, including preparing: a model training factor of the neural network model to be predicted, and a model testing factor.

[0022] Compared with the existing software reliability measurement and prediction technology, the beneficial effects of the present invention are:

[0023] 1. The present invention solves the problems of the lack of current neural network model reliability prediction methods, the difficulty in modeling reliability influencing factors, and the difficulty in measuring safety reliability levels. According to the characteristics of the neural network model, the present invention decomposes and analyzes the influencing factors and key parameters of its reliability, and constructs a neural network model reliability prediction method based on support vector regression, which can effectively improve the accuracy, generalization and fault tolerance of neural network model reliability prediction.

[0024] 2. The modeling factors of the reliability model include model performance, data quality, and testing degree. Using the operating data of the neural network model as the training data of the support vector regression model can obtain the end-to-end neural network model reliability prediction results in a more dynamic and intuitive way.

[0025] 3. Model training factors characterize the data and model factors that affect the reliability of the neural network, and model testing factors are used to characterize the adequacy of the test. These two types of factors can more comprehensively characterize the reliability of the model.

[0026] 4. The impact of data set size on neural network training is extremely significant, including model performance, generalization ability, training speed, and final application scope. Understanding how data volume affects training results is the key to optimizing deep learning projects. When the data set is large enough, the samples are more widely distributed, and the model has more opportunities to learn the nuances and complex structures in the data, so that it can predict and classify more accurately. The distinguishability of the data set greatly determines the performance and reliability of the neural network.

[0027] 5. By combining the dual detection methods of test iteration number and model structure coverage, the reliability of the neural network can be modeled more finely and systematically. The number of test iterations and the size of the test data set directly reflect the depth of the model being tested and the test cost. Generally, the higher the degree of testing, the higher the reliability of the tested model is considered.

[0028] 6. The present invention can effectively support the development of intelligent software. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 is a neural network model reliability prediction flow chart in an embodiment;

[0030] Figure 2 Schematic diagram of factors affecting the reliability of a neural network model in an embodiment;

[0031] Figure 3 4 is a framework diagram of the neural network model reliability prediction in the embodiment. DETAILED DESCRIPTION

[0032] The following is a detailed description of an embodiment of the present invention in conjunction with the accompanying drawings: This embodiment is implemented on the premise of the technical solution of the present invention, and a detailed implementation method and a specific operation process are given, but the protection scope of the present invention is not limited to the following embodiment.

[0033] This embodiment starts from the factors affecting the reliability of neural networks and establishes a reliability measurement model suitable for neural networks. Figure 2 As shown in FIG, the influencing factors are divided into model training factors and model testing factors according to the two aspects of training and testing of the neural network model. The model training factors include the training data quality factors and the model performance factors, and the model testing factors include the testing degree factors.

[0034] in,

[0035] The training data quality factors include the size of the training data set and the distinguishability of the training data, which reflect the adequacy of the training data and the complexity of the target task, respectively;

[0036] Model performance factors include the number of model parameters, model training loss, and the number of training iterations. The first two reflect the complexity of the model structure and the training effect, respectively. The number of model parameters is a direct indicator of the model's potential. Models with more parameters usually have stronger data fitting capabilities and can theoretically capture more complex data features, but they may also bring the risk of overfitting. Model training loss is an important indicator for evaluating training results, which is directly related to the reliability and accuracy of the model in practical applications. The lower the training loss, the better the model performs on the training set, but too low a loss may mean overfitting. The basic purpose of the loss function is to quantify the difference between the model output and the actual label. During the training process, by continuously optimizing this difference, the model gradually learns how to accurately predict or classify;

[0037] Test intensity factors include the number of test iterations, the size of the test data set, and the model structure coverage.

[0038] In reliability model training, you can directly set: training data size, training iteration number, model parameter quantity (determined by model structure), test data set size, and test iteration number;

[0039] During reliability model training, you need to provide: a neural network model and a data set that meets the set conditions;

[0040] In the reliability model training, the following are obtained based on the neural network model process and output data: training data distinguishability, model training loss, and model structure coverage;

[0041] In reliability prediction, the party being tested needs to provide: training data scale, number of training iterations, number of test iterations, tested model, and test data set;

[0042] In reliability prediction, the following can be directly obtained from the tested model and test data set: model parameter quantity and test data set size;

[0043] In reliability prediction, the following are obtained based on the neural network model process and output data: training data distinguishability, model training loss, and model structure coverage.

[0044] The number of model training iterations has an important impact on model reliability. The number of training iterations determines the depth and degree to which the model learns from the data. When the number of training iterations is too small, the model may not fully learn the features in the data, resulting in underfitting, which is manifested as poor performance on both the training set and the test set, and inability to accurately predict new data. On the contrary, too many training iterations may lead to overfitting, that is, the model over-learns the noise and details in the training data, and performs unstable on the test set or new data, affecting generalization ability. Therefore, it is crucial to choose an appropriate number of training iterations, which balances the model's learning depth in the training data and its ability to predict new data, ensuring model reliability.

[0045] As a direct indicator, the number of test iterations can reflect the depth of model testing and the testing cost, and can be used to preliminarily model the test adequacy.

[0046] Training data distinguishability calculation: Dataset distinguishability calculation is based on Linear Relaxation Perturbation Analysis (LiRPA). Linear Relaxation Perturbation Analysis is an advanced technique designed to evaluate and enhance the ability of neural networks to distinguish between different categories of input. The distinguishability of a dataset is evaluated by the ability of the network to maintain correct output when subjected to small changes in input. By gradually adding perturbation strength and observing the changes in the confidence interval predicted by the model at this time, we can determine which categories a data may be misled into and analyze the similarity between two categories. By analyzing the overlapping intervals of two categories, we can obtain the distance relationship between any two categories, and finally obtain the total dataset distinguishability on average.

[0047] In a neural network, the output of each neuron is a nonlinear function of the input. The core of the LiRPA method is to approximate the behavior of these nonlinear functions with linear upper and lower bounds, thereby calculating the response limits of the output neurons within a given range of input perturbations. This calculation is done by constructing a computational graph that includes all network layers, and the linear response of each layer is derived based on the output of the previous layer. To achieve this, LiRPA uses a series of algorithms, such as automatic differentiation and interval propagation, to automate this process, thereby avoiding the tedious process of manually deriving the response of each layer.

[0048] The computation graph is defined as a directed acyclic graph, represented as .in, is the set of neurons in the graph, and is a set of neuron pairs, indicating that the neuron It's a neuron To simplify the description, the neuron The in-degree of , and its input neuron set is recorded as , where each pair All belong to ,and .

[0049] Each neuron There are several key attributes: is the corresponding calculation function, and It is made by neurons The generated vector. Although in practical applications It may be a tensor, but in this invention, to simplify the discussion, it is assumed to have been flattened into a vector.

[0050] Each neuron Either it is an independent neuron, with in-degree , represents the input neuron of the graph (such as network parameters or model input), or is a dependent neuron, indicating that some calculation is performed (such as ReLU).

[0051] For individual neurons, Is an identity function. For neural networks, Indicates that Represents the input data. For dependent neurons, define For all A collection of such that each neuron The output can be expressed as The function of , without explicitly referencing Without loss of generality, assume that the computational graph has a single output neuron .

[0052] To perform perturbation analysis, for a neural network, From its input space If Undisturbed, ,in is a constant vector. For neural networks, we define for space, where Each part of In their input space Disturbed inside.

[0053] In Linear Relaxation Perturbation Analysis (LiRPA), the ultimate goal is to calculate the output neuron Provable lower and upper bounds on the value of , i.e., element-wise lower bounds and upper bound exist When the internal disturbance: , for all In LiRPA, for neural networks, the calculation of To find tight lower and upper bounds, use linear bounds on :

[0054] , quilt Linear functions and parameters of Definition.

[0055] For neural networks, existing LiRPA methods are categorized into two types: forward mode perturbation analysis and backward mode perturbation analysis. These two methods aim to obtain bounds in different ways.

[0056] Forward mode: In forward mode, LiRPA calculates the linear bounds of each neuron with respect to all independent neurons, that is, The linear limit of , propagates to its successor neurons, all the way to the output neuron o.

[0057] Backward mode: In backward mode, LiRPA propagates the linear bounds of the output neuron o with respect to the dependent neurons to the preceding predecessor neurons until all independent neurons are reached.

[0058] Forward mode LiRPA on a general computational graph, for each neuron in the graph , for neural networks, calculation Linear bounds on all independent neurons: .

[0059] For a neural network, start with an independent neuron. , for neural networks, we have , so obviously for neural networks, we can get the bound For a dependent neuron , for the neural network, there is a forward LiRPA prediction function , which accepts each of as input and for the neuron Generate new linear bounds, assuming that all The neurons have been defined, then .

[0060] Backward mode LiRPA on a general computation graph:

[0061] For each neuron, two properties are maintained and , these two attributes represent Relative to The coefficients in the linear limits of : .

[0062] and is the bias term in the neural network algorithm. Assume that the neuron The output dimension is , the output neuron The dimension is , then the matrix and The shape is , then:

[0063] This makes the previous formula valid.

[0064] When neurons is a dependent neuron, there is a backward LiRPA prediction function , which aims to calculate The lower and upper bounds of and use its precursor neurons A linear function of represents these bounds:

[0065] , for neural networks, replace the above new bounds in the formula terms, so that these terms are back-propagated and are represented by Related item replacements.

[0066] Ultimately, all of these terms are propagated to individual neurons, will be bounded only by linear functions of independent neurons.

[0067] Through the above derivation, LiRPA can solve the confidence interval of a sample (original label a) classified into each possible class (a, b, c, d) under a certain disturbance; the intersection and union ratio of the confidence intervals of the correct class a and the wrong class b divided by 2 is the probability of misclassification into class b, and the probability of misclassification into c and d can be calculated similarly; the misclassification probability of all samples with the original label a is calculated, and then averaged respectively, to obtain the overall probability of a being misclassified into b, c, d; the average of the probabilities of a being misclassified into b, c, d is recorded as the confusion degree of class a data; similarly, the confusion degree of class b, c, d data can be obtained; the confusion degree of class a, b, c, d data is averaged to obtain the confusion degree of the entire test data set; the smaller the confusion value, the more distinguishable the data set.

[0068] Specifically, LiRPA can obtain the confidence intervals of each data item predicted to each category under a specific disturbance. For example, a CIFAR10 image may originally be classified as a dog. Under certain disturbances, its predicted confidence intervals will fluctuate. The intersection of the confidence intervals of the dog and the confidence intervals of the cat is 0.4 to 0.6, and the union is 0.2 to 0.9. At this time, the intersection-to-union ratio is 2 / 7, indicating the probability of the confidence falling on the intersection when classified as a cat or dog. It is believed that the probability of correct and incorrect classification is 1 / 2 at this time. Therefore, the probability of an image originally labeled as a dog being misclassified as a cat is 2 / 7*1 / 2=1 / 7. By combining all images and all categories, the overall misclassification probability of the data set can be obtained. The closer the value is to 0, the greater the distinguishability.

[0069] The model training loss can estimate the degree of inconsistency between the model's predicted value and the true value. It is related to the loss function set by the model and can be directly output by the model. The training set used to obtain the training loss can extract a part of the data set of the initial training model. In terms of calculation, the training loss is calculated by calculating the average of the loss of each sample in the training set.

[0070] Model structure coverage is a test adequacy characterization indicator based on neuron activation. The paper proposes indicators such as neuron coverage, neuron boundary coverage, and neuron density coverage. The case of the present invention uses neuron coverage as a model structure coverage indicator. The activation threshold is set to 0.75. If the neuron activation value is greater than the threshold, the neuron is considered to be covered. The ratio of coverage to the total number of neurons is calculated as the model structure coverage indicator.

[0071] The reliability model uses the support vector regression SVR method to implement chaotic modeling of the reliability of the neural network model. Among them, the input of the reliability model is the aforementioned model training factor and model test factor, and the output is the task accuracy of the neural network model, that is, the reliability of the neural network model. In the complex environment of actual development, the interaction of multiple factors often presents a complex nonlinear relationship. By introducing the kernel function to achieve linear separability after high-dimensional mapping, the problem of nonlinear regression analysis is solved. By setting the regularization parameter, the balance between model complexity and training error is achieved, and the accuracy, generalization and fault tolerance of the neural network model reliability prediction are improved. The reliability prediction framework is shown in the figure below. Figure 3 As shown, the following is a common-sense description of support vector regression.

[0072] Support vector regression, which can also be seen as function estimation, aims to find an optimal function from a given sample data set that can accurately reflect the relationship between sample data. Consider a training set of a data set ,in is a vector composed of 8 explanatory variable values ​​(influencing factor index values), is the corresponding dependent variable value. The goal of this study is to find an optimal functional relationship to fit these data points. In the support vector machine model, the function f(x) is defined as a linear form, that is: , where w represents the weight vector and b is the bias term. Support vector regression assumes that there is at most The error is only when the absolute value of the difference between f(x) and y is greater than The loss is calculated only when . Therefore, the objective of SVR is as follows:

[0073] ,

[0074] Where C is the regularization constant, As the loss function, in order to enhance the prediction ability of the model in the case of small samples, we introduce Insensitive loss function adjusts the model's fit to the data. This loss function is defined by the following formula:

[0075] ,

[0076] At this point, the maximum fitting error of all data points in the entire training set does not exceed , which means that the maximum distance from all input data points to the decision plane defined by the model in the high-dimensional feature space is .

[0077] Solve the dual optimal problem and obtain the optimal regression estimation function as follows:

[0078] ,

[0079] ,

[0080] in:

[0081] ,

[0082] Considering the nonlinearity of the neural network model, the kernel function is introduced and the sample space x , x i Using the mapped image , Instead, let , then the regression function is changed to:

[0083] ,

[0084] in:

[0085] ,

[0086] In the above formula The number of standard support vectors.

[0087] Reliability Model Input x For the aforementioned model training factor and model testing factor, output y It is the accuracy of the model task under test, which is the recognition accuracy in image recognition, and serves as the reliability of the neural network model.

[0088] pass( x , y ) obtains the parameters of the reliability model and outputs the reliability prediction results. In the new neural network model test, the reliability model can also be continuously revised to make it closer to the actual situation of the neural network model reliability.

[0089] In general, the present invention first trains the reliability model and then performs reliability prediction on the neural network model to be predicted. Figure 1 The specific method flow is as follows:

[0090] Part I Reliability Model Training

[0091] Step 1. Preparation of neural network model set (15 types): Use commonly used open source neural network models, including ResNet50, MobileNetV2, GoogLeNet, InceptionNet V2, DenseNet, SqueezeNet, and simple double hidden layer fully connected network (the number of hidden layer neurons is [20, 50, 100], [20, 50, 100] in pairs).

[0092] Step 2. Dataset (6 types * 10 sizes) preparation: On the training set of the public dataset, the training dataset size is set to 10%, 20%, 30%...100% of the original training set, and the dataset that meets the size requirement is randomly sampled. Among them, the public datasets include CIFAR-10, MNIST, ImageNet, SVHN (Street View House Numbers), CIFAR-100, and Fashion-MNIST.

[0093] Step 3: Neural network model training: Set the random number of training iterations, fluctuating between 40 and 300, to build models with different training effects.

[0094] Step 4: Neural network model testing: Set the number of random test iterations and the size of the test data set. The former is selected between 40 and 300, and the latter is selected between 60% and 100% of the original test set. Test on the corresponding test set and record the accuracy of each test as the dependent variable for SVR model training.

[0095] Step 5: Calculation of neural network influencing factor indicators: extract the training data scale, training iteration number, model parameter quantity (determined by the model structure), test data set scale, and test iteration number, calculate the training data distinguishability, model training loss, and model structure coverage, and obtain a set of 8 influencing factor indicator results as explanatory variables for SVR model training.

[0096] Step 6: Reliability model training: Use the sklearn library to train the SVR-based reliability model, where the parameter setting is The value is 0.01, the value of C is 10, and the kernel function is the rbf kernel function. The trained reliability model is obtained.

[0097] Part II Reliability Prediction

[0098] Step 7: The user provides the neural network model for reliability prediction, training data size, test data set, number of training iterations, and number of test iterations.

[0099] Step 8: Based on the neural network model and data set provided by the user, calculate the neural network influencing factor indicators, including: model parameter quantity, test data set size, training data distinguishability, model training loss, and model structure coverage.

[0100] Step 9: Input the eight influencing factor index values ​​into the trained reliability model and output the reliability prediction results.

Claims

1. A reliability prediction method for an image classification neural network model, characterized in that: The method includes training a reliability model, and then using the trained reliability model to perform reliability prediction on a neural network model to be predicted for image classification, wherein, according to the training and testing of the neural network model, the input features of the reliability model include a model training factor and a model testing factor, the model training factor characterizes the data and model factors that affect the reliability of the neural network, and the model testing factor is used to characterize the adequacy of the neural network test; the output of the reliability model is a reliability prediction result of the neural network model; The model training factors include the training data quality factor and the model performance factor, wherein the training data quality factor includes the training data set size and the training data distinguishability, wherein the training data set size reflects the adequacy of the training data, and the training data distinguishability reflects the complexity of the target task; the model performance factors include the model parameter amount, the model training loss, and the number of training iterations, wherein the model parameter amount reflects the complexity of the model structure, the model training loss reflects the model training effect, and the number of training iterations determines the depth and degree of the model's learning from the data; The model testing factor is characterized by the test degree factor, which includes the number of test iterations, the size of the test data set, and the model structure coverage. The number of test iterations directly reflects the depth of the model being tested, the size of the test data set directly reflects the test cost, and the model structure coverage is a test adequacy characterization indicator based on the neuron activation status.

2. The reliability prediction method for an image classification neural network model according to claim 1, characterized in that: LiRPA computes the distinguishability of training data based on linear relaxation perturbation analysis.

3. The reliability prediction method for an image classification neural network model according to claim 2, characterized in that: The process of calculating the distinguishability of training data includes: each sample in the training data set corresponds to a correct class, and LiRPA solves the confidence interval of each sample classified into each possible class under perturbation; Calculate the probability of each sample being misclassified into each other possible class, where the probability of any sample being misclassified from the correct class a to the class b is equal to the intersection and union of the confidence interval corresponding to the correct class a and the confidence interval corresponding to the wrong class b, divided by 2; calculate the probability of all samples corresponding to the correct class a being misclassified into each other possible class and average it, recorded as the confusion degree of class a data; average the confusion degrees of all classes of data to obtain the confusion degree of the entire training data set, which is the distinguishability of the training data.

4. The reliability prediction method for an image classification neural network model according to claim 1, characterized in that: The model training loss is the average of the losses for each sample in the training set.

5. The reliability prediction method for an image classification neural network model according to claim 4, characterized in that: Neuron coverage is used as the model structure coverage indicator.

6. The reliability prediction method for an image classification neural network model according to claim 1, characterized in that: The reliability model uses support vector regression (SVR).

7. A reliability prediction method for an image classification neural network model according to any one of claims 1 to 6, characterized in that: Training the reliability model includes preparing a training data set, a test data set, and a neural network model set. All models in the neural network model set are trained on the training data set to obtain model training factors. All models in the neural network model set are tested on the test data set to obtain the model test factors; the eight reliability influencing factor index values ​​formed by the model training factors and the model test factors constitute the explanatory variables of the reliability model, and the dependent variable is the accuracy of the neural network model on the corresponding data set. The training reliability model is used to establish the relationship between the explanatory variables and the dependent variables.

8. A reliability prediction method for an image classification neural network model according to any one of claims 1 to 6, characterized in that: The trained reliability model is used to perform reliability prediction on the neural network model to be predicted for image classification, including preparing: a model training factor of the neural network model to be predicted, and a model testing factor.

Images