An intelligent system failure mechanism analysis method based on structural risk minimization

Abstract

The present application relates to a kind of intelligent system failure mechanism analysis method based on structural risk minimization, steps include: according to the function design of intelligent system, the convolutional neural network structure model corresponding function is established;A learning rate reduction strategy is selected for model, different values are set for learning rate according to the strategy, and the learning rate decay degree is obtained;Structural risk minimization is introduced, and the risk of model is composed of empirical error of training sample and hypothesis space complexity;Empirical precision is regarded as the function of learning rate decay degree and is fitted by first-order multinomial relationship, Rademacher complexity is used to evaluate hypothesis space complexity, empirical precision replaces empirical error, and the definition of Rademacher complexity is expanded to obtain the relationship between learning rate decay degree and the risk of model.The method analyzes the influence of learning rate on intelligent model according to structural risk minimization strategy, and the model risk can be identified and evaluated by the strategy, to reduce intelligent system failure.

Description

Technical Field

[0001] This invention proposes a failure mechanism analysis method for intelligent systems based on minimizing structural risk, belonging to the field of artificial intelligence and reliability engineering. Background Technology

[0002] In the era of big data, artificial intelligence (AI) models, by simulating human intelligence, automatically process and analyze massive amounts of data, improving work efficiency and decision-making accuracy, and promoting innovation and social development. Intelligent models are the foundation for realizing intelligent functions in intelligent systems; they can contain multiple intelligent models to achieve more complex functions. While AI models have achieved unprecedented performance levels in learning to solve increasingly complex computational tasks and have had a significant impact on human society, they have also raised concerns among researchers. They pose direct or indirect threats to political, military, and social security, further threatening the system's execution capabilities. To ensure the safe and reliable operation of intelligent systems, AI models are typically intricate, with hundreds of millions of parameters. This leads to decreasing model transparency, making it difficult for people to understand how the models work and trust their decisions, thus limiting their application in real-world tasks, especially in areas where incorrect predictions are costly. Currently, AI model design still primarily focuses on performance optimization, with few algorithm researchers paying attention to the failure mechanism analysis of AI models. With the proliferation of intelligent systems, cases of internal AI model failures are becoming increasingly common. However, intelligent models differ from ordinary algorithms; when failures occur, it is difficult to pinpoint the cause. Therefore, research on failure mechanisms for intelligent models is both necessary and urgent, and presents unprecedented challenges.

[0003] The causes of intelligent system failures are diverse, making it difficult to exhaustively list every influencing factor. Considering that the learning rate is a common tuning technique in model training and has a crucial impact on the final performance of the intelligent model, analyzing the impact of the learning rate on model training performance from a mechanistic perspective is a pressing issue. Risk analysis of models is typically used to assess the likelihood of intelligent system failures. By identifying and managing high-risk factors, preventative measures can be taken to reduce the probability and impact of failures.

[0004] Therefore, it is necessary to construct a method for analyzing the failure mechanism of intelligent systems based on structural risk minimization (SRM). Summary of the Invention

[0005] The purpose of the present invention is to analyze the failure mechanism of intelligent systems, and proposes an intelligent system failure mechanism analysis method based on structural risk minimization. This method focuses on the model training and tuning process in intelligent systems, reveals the influence mechanism of learning rate on model training results, and characterizes the influence process of learning rate on intelligent model training by introducing a structural risk minimization factor, which mainly comprises the following steps:

[0006] Step 1: Establish a convolutional neural network structure model with corresponding functions according to the functional design of the intelligent system.

[0007] Step 2: Construct the learning rate attenuation degree of the intelligent model in the intelligent system;

[0008] First, set an initial learning rate value lr1 (1e -12 <lr1<1), then select a learning rate attenuation strategy to set m (m≥3) learning rate values with different magnitudes [lr1,...,lr m (lr m <...<lr1, lr1<1, lr m ≥1e -12 ), therefore the corresponding learning rate attenuation degree rd=[rd1,...,rd m , where rd i =lr1-lr i .

[0009] Step 3: Establish the relationship between the learning rate attenuation degree and the risk of the intelligent model;

[0010] First, for each learning rate attenuation degree rd i obtained in Step 2, train the convolutional neural network model respectively on the training sample D, and obtain the corresponding empirical accuracy . Taking empirical accuracy as a function of learning rate attenuation degree rd, the fitting is obtained through a first-order polynomial relational expression:

[0011] (1)

[0012] wherein x represents an unknown learning rate attenuation degree, k D represents the slope, represents the maximum empirical training accuracy expected to be obtained, , Hs is the set of input-to-output mappings, that is, the hypothesis space, h is the hypothesis function actually learned by the model, and h∈Hs.

[0013] Second, calculate the risk of the intelligent model in the intelligent system to characterize the failure rate of the intelligent model, and the formula is:

[0014] (2)

[0015] Among them, R srm Indicating the risks of intelligent models, For training samples D={(x i ,y i The empirical error of the group i=1,...,m} This represents the complexity of the model.

[0016] The above formulas can be used to correlate the complexity of the intelligent model, the sample state, and the failure consequences, thereby constructing a failure mechanism analysis model under different sample and model states.

[0017] Meanwhile, the Rademacher complexity definition for the hypothesis space Hs on training samples D is introduced to measure the complexity of the model. ,Right now:

[0018] (3)

[0019] Where δ represents the confidence level (0 < δ < 1). Indicates predicted value Compared with the true value The correlation between them This indicates that all hypotheses in Hs are considered, and the expected value E is taken for the empirical Rademacher complexity of the hypothesis space Hs on the training samples D.

[0020] Then, using empirical precision Replacement of empirical error Substituting formula (3) into formula (2), we obtain the expression for the risk of the intelligent model with respect to empirical accuracy:

[0021] (4)

[0022] Finally, substituting the first-order polynomial relationship (1) between empirical accuracy and learning rate decay into formula (4) yields the learning rate decay rd and the intelligent model risk R. srm The relationship between them:

[0023] (5)

[0024] Through the above steps, a failure mechanism analysis method for intelligent systems based on structural risk minimization is presented.

[0025] The superior effect of this invention is that, in order to analyze the failure mechanism of intelligent systems and reduce possible failures, starting from the learning rate that affects the model training results in intelligent systems, structural risk minimization is introduced to establish a relationship between the slope of the empirical accuracy function with the learning rate decay as the independent variable and the risk of the intelligent model. By deriving the relationship between the learning rate and the model risk, the probability of intelligent system failure can be reduced from the perspective of reducing model risk, providing unique insights for the failure mechanism analysis of intelligent systems. Attached Figure Description

[0026] Figure 1 The diagram shows a flowchart of a failure mechanism analysis method for intelligent systems based on structural risk minimization.

[0027] Figure 2 The image shows the training and validation results of the model under three different learning rates.

[0028] Figure 3 The training results of the model under multiple different learning rates are shown.

[0029] Figure 4 The validation results of the model under multiple different learning rates are shown.

[0030] Figure 5 As shown Resulting graph Detailed Implementation

[0031] To provide a clearer understanding of the features and advantages of the present invention, a detailed description is provided below in conjunction with the accompanying drawings and embodiments:

[0032] This experiment uses an intelligent system composed of intelligent models such as ResNet18, ResNet34, and ResNet50, along with experimental datasets from CIFAR-10 and MNIST, to illustrate the implementation steps. Taking the intelligent system composed of ResNet18 as an example, the specific implementation steps are as follows:

[0033] 1. Use ResNet18 as the training model .

[0034] 2. Set the initial learning rate lr1 to 0.100, and adjust the learning rate according to the Lambda function. Therefore, we can get lr=[0.100,0.050,0.020,0.010,0.005,0.001], and rd=[0.000,0.050,0.080,0.090,0.095,0.099].

[0035] 3. For each different learning rate decay point, the model can obtain the corresponding empirical accuracy after training:

[0036]

[0037] Secondly, the risks of computational models:

[0038]

[0039] Then, the Rademacher complexity is used for evaluation. We can obtain the following by substituting empirical accuracy for empirical error and taking the expectation of the empirical Rademacher complexity of the hypothesis space on the training samples:

[0040]

[0041]

[0042] This allows us to derive the relationship between the risk level and learning rate of the ResNet18 model on the dataset. The effectiveness of the proposed method is then verified using the MNIST dataset.

[0043] Figure 2 , Figure 3 and Figure 4 Training and validation results for the models at different learning rates are presented. Multiple comparisons show that the performance of ResNet18, ResNet34, and ResNet50 continuously improves with decreasing learning rates, meaning the model accuracy increases. Therefore, continuously decreasing the learning rate or choosing a smaller learning rate may yield better results. Furthermore, more complex models do not necessarily have superior performance. When the learning rate is lr=0.005, ResNet18 achieves the highest accuracy in both training and validation on the MNIST dataset. Therefore, model complexity may not be directly proportional to model performance.

[0044] exist Figure 5 The rd value was calculated under different learning rate decay degrees. The value of rd. The horizontal axis in the graph represents the degree of learning rate decay, and the vertical axis on the left represents the current rd value. The value on the right represents the current value. The training accuracy value is shown in the figure. It can be seen from the graph that the model validation accuracy decreases and the error increases when the gray circle is drawn, but... The trend of continuous decline indicates that the method of the present invention may have the ability to capture overfitting and the change is stable.

[0045] The above description represents the preferred embodiment of the present invention. For those skilled in the art, modifications or equivalent substitutions can be made to the specific embodiments of the present invention without departing from the overall concept of the present invention, and these modifications or substitutions should also be considered within the scope of protection of the present invention.

Claims

1. A structural risk minimization-based intelligent system failure mechanism analysis method, characterized by, The method is used to analyze the failure mechanism of convolutional neural network models in intelligent systems, so as to reduce the model risk through the mechanism. It includes the following steps: Step 1: Establish a convolutional neural network structure model corresponding to the functions of the intelligent system; Step 2: Construct the learning rate decay of the intelligent model in the intelligent system; First, an initial learning rate value lr1 (1e -12 <lr1<1) is set for the convolutional neural network model constructed in step 1, then a learning rate decay strategy is selected to set m (m≥3) learning rate values of different magnitudes [lr1,...,lr m (lr m <...<lr1, lr1<1, lr m ≥1e -12 ), thus the corresponding learning rate decay degree rd=[rd1,...,rd m , wherein rd i =lr1-lr i ; Step 3: Establish the relationship between learning rate decay and the risk of the intelligent model; First, an experimental dataset was constructed using the CIFAR-10 and MNIST datasets. For each learning rate decay rd obtained in step 2... i The convolutional neural network models were trained on the training samples D, and the corresponding empirical accuracy was obtained. ; to increase the accuracy of experience As a function of the learning rate decay rd, it is obtained by fitting a first-order polynomial relation: where x represents an unknown learning rate decay degree, k D represents the slope, represents the maximum experience training accuracy expected to be obtained, Hs is a set of input to output mappings, i.e. hypothesis space, h is a hypothesis function actually learned by the model, h ∈ Hs; Secondly, the risk of the intelligent model in the intelligent system is calculated to characterize the failure rate of the intelligent model. The formula is as follows: Among them, R srm Indicating the risks of intelligent models, For training samples D={(x i ,y i The empirical error of the group i=1,...,m} The complexity of the model; The above formulas can be used to correlate the complexity of the intelligent model, the sample state, and the failure consequences, thereby constructing a failure mechanism analysis model under different sample and model states. Meanwhile, the Rademacher complexity definition for the hypothesis space Hs on training samples D is introduced to measure the complexity of the model. ,Right now: Where δ represents the confidence level (0 < δ < 1). Indicates predicted value Compared with the true value The correlation between them This indicates that all hypotheses in Hs are considered, and the expected value E is taken for the empirical Rademacher complexity of the hypothesis space Hs on the training samples D. Then, using empirical precision Replacement of empirical error Substituting the definition of Rademacher complexity into the risk of the intelligent model, we obtain an expression for the risk of the intelligent model with respect to empirical accuracy: Finally, the first-order polynomial relationship between the empirical accuracy and the learning rate decay degree is substituted into the above formula to obtain the relationship between the learning rate decay degree rd and the intelligent model risk R srm between the learning rate decay degree rd and the intelligent model risk R This leads to the relationship between the risk of the intelligent model on the MNIST dataset and the learning rate decay.

Images