Industrial intelligent model evaluation method for aluminum smelting industry

Abstract

An industrial intelligent model evaluation method for the aluminum smelting industry relates to the technical field of industrial intelligent model evaluation in the aluminum smelting industry, and comprises the following steps: S01, obtaining a to-be-evaluated industrial intelligent model, and classifying the model according to model use characteristics; s02, confirming input and output data requirements of the to-be-evaluated model, and reserving a data interface; s03, selecting data for model evaluation; s04, preprocessing the data; s05, inputting the data into the model, and obtaining model output data; s06, constructing an evaluation data set; s07, evaluating each evaluation dimension of the model; s08, weighting is carried out for comprehensive evaluation, and a model score is obtained. According to the method, the accuracy, the robustness, the interpretability, the efficiency and the reliability of the model are evaluated respectively, and finally weighted summation is performed to obtain a model evaluation score, so that a factory is helped to accurately evaluate the advantages and disadvantages of the model, and powerful guarantee can be provided for model self-evaluation and self-updating in the intelligent construction process of the factory.

Description

Technical Field

[0001] The present invention relates to the technical field of model evaluation in the aluminum smelting industry, and particularly to an industrial intelligent model evaluation method for the aluminum smelting industry. Background Art

[0002] As one of the important basic industries, the production process of the aluminum smelting industry is complex and involves multiple technological processes, including extracting alumina from bauxite and then generating primary aluminum through the electrolysis process; with the rapid development of industrial automation and intelligent technologies, the aluminum smelting industry is gradually promoting the construction of intelligent factories, aiming to improve production efficiency and product quality through informatization and digital means.

[0003] During the construction of intelligent factories, a large number of industrial intelligent models have been developed for each link of aluminum smelting, such as models for measuring, controlling, and optimizing various process variables; however, with the continuous production operation of the factory, changes in internal and external conditions pose new challenges to the applicability of industrial intelligent models. Specifically, changes in internal factory conditions include equipment aging, process parameter adjustment, raw material composition fluctuations, etc., while changes in external conditions involve market demand changes, environmental protection policy adjustments, energy price fluctuations, etc.; these changes may cause the original industrial intelligent models to no longer be applicable to the current industrial environment.

[0004] In order to ensure that industrial intelligent models can continuously and effectively serve the production process, it is necessary to regularly correct or overall update the models to improve problems such as inaccurate model prediction, poor production efficiency, and unqualified quality and output. However, before correcting or updating the model, there must be a complete and effective model evaluation method; through a scientific evaluation method, the performance of the model in the current environment can be comprehensively evaluated, and it can be identified whether the model has deviations or failures, thereby providing a basis for correcting or updating the model.

[0005] Existing model evaluation methods often have problems such as single evaluation indicators, unsystematic evaluation processes, and difficulty in comprehensively reflecting model performance, resulting in inaccurate evaluation results and difficulty in effectively guiding model optimization and improvement. Summary of the Invention

[0006] In view of the above-mentioned disadvantages and deficiencies of the prior art, the present invention provides an industrial intelligent model evaluation method for the aluminum smelting industry, which is used to evaluate various industrial intelligent models involved in the aluminum smelting industry, and further helps the informatization and intelligent systems of the factory to realize the self-evaluation function of the model, providing guarantee for model self-update.

[0007] The "industrial intelligent models of the aluminum smelting industry" mentioned in the present invention include measurement models, prediction models, control models, optimization models, decision-making models, and simulation models for each production operation management link.

[0008] The specific feature of the industrial intelligent model is to input production process index data into the model and output the required production process index data after model calculation.

[0009] The main technical solutions adopted in the present invention include the following steps:

[0010] Step S01: Obtain the industrial intelligent model to be evaluated and classify the model according to the characteristics of the model's use;

[0011] Step S02: Confirm the input and output data requirements of the model to be evaluated and reserve data interfaces.

[0012] Step S03: Select the data for model evaluation;

[0013] Step S04: Preprocess the data;

[0014] Step S05: Input the data into the model and obtain the model output data;

[0015] Step S06: Construct an evaluation data set;

[0016] Step S07: Evaluate each evaluation dimension of the model;

[0017] Step S08: Conduct a comprehensive evaluation with weights to obtain the model score.

[0018] In the step S01, when obtaining the industrial intelligent model to be evaluated, if the industrial intelligent model is being used in an industrial system, the industrial intelligent model shall be completely exported first, then the model shall be evaluated, and the industrial intelligent models in the aluminum smelting industry are divided into: soft sensor models, prediction models, control models, optimization models, decision-making models, and industrial simulation models;

[0019] In the step S02, confirm the category of the exported industrial intelligent model, confirm the input and output data requirements of the model, including the dimension and data format of the data, and develop the data interface of the model in the software system.

[0020] In the step S04, when preprocessing the data used for model evaluation, the methods adopted include the following steps:

[0021] Step S41: Unify the data sampling interval;

[0022] Step S42: Check and process data missing;

[0023] Step S43: Detect and process data noise;

[0024] Specifically: during the evaluation process, outliers are screened out from the noise data, and for each index data, outliers are screened separately;

[0025] First, calculate the standard deviation of the sample:

[0026]

[0027] where x i is the sample value, is the sample mean, and n is the sample size;

[0028] Then, calculate the error between a single data point and the average value:

[0029] Respectively, judge the magnitude of |Δx i |. If |Δx i | > ρ·σ, then label x i as a suspected outlier. The value of ρ ranges from 2.0 to 4.0;

[0030] If the absolute values of the errors of three consecutive values x i , x i+1 , x i+2 are all greater than ρ·σ, then determine that these three values are not occasional outliers, and retain the three values x i , x i+1 , x i+2 .

[0031] Step S44: Standardize or normalize the data to unify the data dimension;

[0032] Step S45: According to the input requirements of the selected model, construct a structured data set indexed by time for model evaluation.

[0033] In the said step S07, the evaluation content of the model includes: S07 - A model accuracy evaluation; S07 - B model robustness evaluation; S07 - C model interpretability evaluation; S07 - D model efficiency evaluation; S07 - E model reliability evaluation; for different types of models, the dimensions to be evaluated are different; among them, measurement models and prediction models are evaluated for their accuracy, efficiency, and reliability; control models and optimization models are evaluated for their accuracy, robustness, efficiency, and reliability; decision models and simulation models are evaluated for their accuracy, interpretability, and efficiency.

[0034] S07 - A model accuracy evaluation: Use the evaluation data set constructed in step S06 to evaluate the accuracy of the model:

[0035] For a model with only one output variable, use the root mean square error RMSE to evaluate whether the model output is accurate:

[0036]

[0037] In the formula, y iIt is the calculation output of the model each time. It represents the model output control data.

[0038] For a model with M output variables, the root mean square relative error RMSRE is used to evaluate the accuracy of the multi-output model:

[0039]

[0040] In the formula, y ij represents the output value of the j-th run of the i-th output variable of the model, represents the control value of the j-th run of the i-th output variable of the model;

[0041] If the output value y ij contains 0, update RMSRE to

[0042]

[0043] When evaluating the overall accuracy of the model, the following formula is used to perform weighted summation on all outputs of the model:

[0044]

[0045] In the formula, θ j is the weight of the j-th output. If all outputs are equally important, then θ j = 1.

[0046] The robustness evaluation of the S07-B model is to add random noise to the data set constructed in step S03 artificially to form a sample or adversarial sample with perturbed noise, input the sample into the model, and check whether the output of the model is still accurate;

[0047] If the model has only one output, the empirical robustness score is as follows:

[0048]

[0049] In the formula, n represents the number of samples, and y i represents the model output obtained by inputting the original sample into the model, represents the model output obtained by inputting the sample with noise or adversarial sample into the model, represents a distance metric function, using the L2 distance (Euclidean distance, dimension is K):

[0050]

[0051] Or cosine similarity:

[0052]

[0053] If the model has M outputs, the formula for the empirical robustness score is as follows:

[0054]

[0055] The value of the empirical robustness score ranges from 0 to 1. The closer the value is to 1, the better the robustness of the model and the less affected it is by perturbations and adversarial samples.

[0056] The interpretability evaluation of the S07-C model uses the subjective judgment of technical experts. The interpretability of the industrial model is evaluated by technical experts to determine whether the model's explanations are reasonable, in line with the actual situation, and can clearly reveal the decision-making logic of the model. An interpretability ITP score is given manually.

[0057] The efficiency evaluation of the S07-D model includes three aspects: model training or establishment efficiency, model calculation efficiency, and model resource consumption. The efficiency evaluation formula is as follows:

[0058]

[0059] In the formula, T Train represents the time taken for model training or establishment, T Calc represents the time taken for each model calculation result, R represents the size of the computing power resources used by the model, and α, β, γ represent the weight coefficients of each efficiency, and α + β + γ = 1.

[0060] The reliability evaluation of the S07-E model is the weighted sum of the output data consistency index S CDI and the model stability analysis index S MSA :

[0061] RC = ω1S CDI + ω2S MSA ;

[0062] In the formula, ω1 and ω2 are weight coefficients, with values ranging from 0 to 1, and ω1 + ω2 = 1;

[0063] The data consistency index S CDI is used to measure the output consistency of the model at different times and in different environments. The calculation formula is as follows:

[0064]

[0065] In the formula, N is the number of tests, μ i represents the overall accuracy of the model in the i-th test, represents the average value of the overall accuracy of the model output in all tests, σ i represents the standard deviation of the output data of the model in the i-th test, S CDIThe value ranges from 0 to 1, where 1 represents perfect consistency and 0 represents no consistency;

[0066] Model stability analysis index S MSA It is used to measure the stability of the model during long-term operation and observe whether there are fluctuations or decays in the model. The calculation formula is as follows:

[0067]

[0068] In the formula, T represents the observation time period for model stability evaluation, and V t represents the output variance of the model at time t, and V init represents the output variance at the initial moment of the model, and S Msa The value ranges from 0 to 1, where 1 represents complete stability and 0 represents no stability.

[0069] In the step S08, finally, the evaluations of each dimension of the model in step S07 are weighted and comprehensively evaluated to calculate the final score (MCE) of the model. The calculation formula is as follows:

[0070] MCE = η1·ME All +η2·ERS + η3·ITP + η4·EC + η5·RC

[0071] In the formula, η1, η2, η3, η4, η5 represent the weight coefficients of the accuracy evaluation, robustness evaluation, interpretability evaluation, efficiency evaluation, and reliability evaluation of the model respectively. The values range from 0 to 1, and η1 + η2 + η3 + η4 + η5 = 1;

[0072] That is, as shown in the evaluation dimension table in step S01, the weights are specifically expressed as:

[0073] For the measurement model and the prediction model, η2 = η3 = 0, η1 + η4 + η5 = 1;

[0074] For the control model and the optimization model, η3 = 0, η1 + η2 + η4 + η5 = 1;

[0075] For the decision-making model and the simulation model, η2 = η5 = 0, η1 + η3 + η4 = 1.

[0076] The beneficial effects of the present invention are as follows: By providing a model evaluation method, the present invention provides a basis for the evaluation of industrial intelligent models in the aluminum smelting industry, helping factories scientifically judge the advantages and disadvantages of models. Compared with existing methods, this technical solution classifies the industrial intelligent models in the aluminum smelting industry, selects the evaluation dimensions required for the models by category, evaluates the accuracy, robustness, interpretability, efficiency, and reliability of the models respectively, and finally performs weighted summation to obtain the model evaluation score. The evaluation method of this technical solution is more general, more systematic and comprehensive, and can provide strong guarantee for the self-evaluation and self-update of models in the process of factory intelligent construction. BRIEF DESCRIPTION OF THE DRAWINGS

[0077] Figure 1 is the flowchart of the model evaluation steps of the present invention;

[0078] Figure 2 is the composition diagram of the model evaluation dimensions of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0079] The following further elaborates on the specific embodiments of the present invention in conjunction with the accompanying drawings, making the technical content of the present invention easier to understand. Although the accompanying drawings show exemplary embodiments of the present disclosure, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided to enable a more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art.

[0080] As Figure 1 shown, the present invention provides an industrial intelligent model evaluation method for the aluminum smelting industry, which sequentially performs the following steps to achieve model classification, data preprocessing, obtaining outputs, and selectively evaluating the accuracy, robustness, interpretability, efficiency, and reliability of the model according to the model category, and finally obtaining the comprehensive score of the model.

[0081] Step S01: Obtain the industrial intelligent model to be evaluated and classify the model according to the model usage characteristics.

[0082] Specifically, when obtaining the industrial intelligent model to be evaluated, if the industrial intelligent model is being used in an industrial system, it is necessary to completely export the industrial intelligent model and then evaluate the model. According to the usage of the model, the industrial intelligent models in the aluminum smelting industry are divided into: (soft) measurement models, prediction models, control models, optimization models, decision models, and industrial simulation models.

[0083] Step S02: Confirm the input and output data requirements of the model to be evaluated and reserve data interfaces.

[0084] Specifically, confirm the category of the exported industrial intelligence model, and confirm the input and output data requirements of the model, including the dimension and format of the data. Finally, develop the data interface of the model in the software system.

[0085] Step S03: Select the data for model evaluation.

[0086] Specifically, the data selected for model evaluation must be recent data, ensuring that the operating environment of the factory is relatively stable within a certain period of time and the internal and external environmental conditions of the factory are basically the same. And it is preferably continuous historical data.

[0087] Step S04: Preprocess the data.

[0088] Furthermore, the methods and steps for data preprocessing are as follows:

[0089] Step S41: Unify the data sampling interval.

[0090] Specifically, the methods for unifying the sampling interval of the dataset can be interval sampling, data interpolation, and interval mean methods.

[0091] Step S42: Check and handle missing data.

[0092] Specifically, the method for handling missing data is to delete the entire row; for continuous time series data, interpolation with the most likely value (average value, historical value) is used.

[0093] Step S43: Detect and handle data noise.

[0094] Specifically, there are usually no mutations in individual indicator data in the process industry data of continuous production. Therefore, during the evaluation process, outliers should be screened out from the noise data. For each indicator data, outlier screening is performed separately in the following way:

[0095] Calculate the standard deviation of the sample:

[0096]

[0097] where x i is the sample value, is the sample mean, and n is the number of samples.

[0098] Then calculate the error between a single data and the average value:

[0099] Judge the magnitude of |Δx i | respectively. If |Δx i | > ρ·σ, then mark this data as a suspected outlier. The value of ρ can be determined according to the data situation, and generally, the value taken is between 2.0 and 4.0.

[0100] If |Δx i | > ρ·σ when a certain value appears, then x i is defined as a suspected outlier. However, if the absolute values of the errors of three consecutive values x i , x i+1 , x i+2 are all greater than ρ·σ, then we consider that these three values are not accidental outliers but characteristic states with practical reference significance, and these three values need to be retained.

[0101] For discrete data, the outliers in the data can be deleted; for continuous time series data, the most likely values (average value, historical value) can be used to replace the noise values.

[0102] Step S44: Standardize or normalize the data to unify the data dimension for facilitating model evaluation. Specifically, the following formula can be used

[0103]

[0104] In the formula, x is the original data, x min is the minimum value in the original data, and x max is the maximum value in the original data.

[0105] Step S45: Construct a structured data set indexed by time according to the input requirements of the selected model for model evaluation.

[0106] Step S05: Input the data into the model to obtain the model output data.

[0107] Specifically, in step S05, the data set constructed in step S03 is fed into the model to be evaluated one by one through the data interface reserved in step S02 to obtain the model output result data.

[0108] Step S06: Construct an evaluation data set.

[0109] Specifically, in step S06, the model output result data needs to be corresponded one by one with the control data to form an evaluation data set. Among them, the control data needs to be obtained in different ways according to the model category described in step S01.

[0110] The control data of the soft measurement model comes from the real dominant variable, such as the sampling and testing results of the soft measurement object.

[0111] The control data of the prediction model, control model, and optimization model comes from the real results in the predicted stage.

[0112] The control data of the decision-making model and simulation model comes from the manual evaluation results of factory process experts and scheduling experts.

[0113] Optionally, the output error or loss function of the model can also be used to train a neural network for evaluating the model output results.

[0114] Step S07: Evaluate each evaluation dimension of the model;

[0115] Since different types of models require different evaluation dimensions, according to the model categories described in step S01, the evaluation dimensions are shown in the following table:

[0116] Model Classification Accuracy Robustness Interpretability Efficiency Reliability Measurement Model, Prediction Model √ √ √ Control Model, Optimization Model √ √ √ √ Decision Model, Simulation Model √ √ √

[0117] That is, according to the classification and evaluation dimensions in the table:

[0118] ● Measurement models and prediction models need to be evaluated for their accuracy, efficiency, and reliability;

[0119] ● Control models and optimization models need to be evaluated for their accuracy, robustness, efficiency, and reliability;

[0120] ● Decision models and simulation models need to be evaluated for their accuracy, interpretability, and efficiency.

[0121] Furthermore, as Figure 2 shown, the evaluation content in step S07 includes:

[0122] (1) S07-A Model accuracy evaluation.

[0123] Specifically, use the evaluation dataset constructed in step S06 to evaluate the accuracy of the model.

[0124] For a model with only one output variable, we evaluate whether the model output is accurate, using the root mean square error (RMSE):

[0125]

[0126] In the formula, y i is the computational output of the model each time, represents the model output control data.

[0127] For a model with M output variables, we need to consider the dimensional differences of different output values. To avoid weight changes caused by dimensional differences, the root mean square relative error (RMSRE) needs to be used to evaluate the accuracy of the multi-output model.

[0128]

[0129] In the formula, y ij represents the output value of the i-th output variable of the model for the j-th run. represents the control value of the i-th output variable of the model for the j-th run.

[0130] If the output value y ij might include 0, it will lead to an infinite calculation result. However, it is almost impossible for all outputs of the model to be zero, so RMSRE is changed to

[0131]

[0132] When evaluating the overall accuracy of the model, the weighted sum of all outputs of the model can be calculated using the following formula:

[0133]

[0134] In the formula, θ j is the weight of the j-th output. If all outputs are equally important, then θ j = 1.

[0135] (2) Robustness evaluation of the S07-B model.

[0136] Specifically, by artificially adding noise to the dataset constructed in step S03 to form samples with perturbed noise or adversarial samples, and inputting these samples into the model to check whether the output of the model is still accurate.

[0137] If the model has only one output, the empirical robustness score is as follows:

[0138]

[0139] In the formula, n represents the number of samples, y i represents the model output obtained by inputting the original sample into the model, represents the model output obtained by inputting the sample with noise or adversarial sample into the model. represents a distance metric function, and the L2 distance (Euclidean distance, dimension K) can be used:

[0140]

[0141] The cosine similarity can also be used:

[0142]

[0143] The value of the empirical robustness score is between 0 and 1. The closer the value is to 1, the better the robustness of the model and the less affected it is by perturbed and adversarial samples.

[0144] If the model has M outputs, the empirical robustness score formula is:

[0145]

[0146] The value of the empirical robustness score of the multi-output model is also between 0 and 1. The closer the value is to 1, the better the robustness of the model, and the less affected it is by perturbations and adversarial samples.

[0147] (3) Evaluation of the interpretability of the S07-C model.

[0148] Specifically, the subjective judgment of technical experts is mainly adopted. It is usually difficult to quantitatively evaluate the interpretability of industrial models, and it is necessary for technical experts to evaluate whether the interpretation of the model is reasonable, whether it conforms to the actual situation, and whether it can clearly reveal the decision-making logic of the model, and manually give the interpretability (ITP) score.

[0149] (4) Evaluation of the efficiency of the S07-D model.

[0150] Specifically, the evaluation of model efficiency includes three aspects: model training or establishment efficiency, model calculation efficiency, and model resource consumption. The efficiency evaluation formula is as follows:

[0151]

[0152] In the formula, T Train represents the time taken for model training or establishment, T Cale represents the time taken for each calculation result of the model, R represents the size of the computing power resources used by the model, and α, β, γ represent the weight coefficients of each efficiency, and α + β + γ = 1.

[0153] (5) Evaluation of the reliability of the S07-E model.

[0154] Specifically, the evaluation of model reliability mainly evaluates the consistency of the model output, and the evaluation result is the weighted sum of the output data consistency index S CDI and the model stability analysis index S MSA :

[0155] RC = ω1S CDI + ω2S MSA

[0156] In the formula, ω1 and ω2 are weight coefficients, with values between 0 and 1, and ω1 + ω2 = 1.

[0157] The data consistency index S CDI is used to measure the output consistency of the model at different times and in different environments, and the calculation formula is as follows:

[0158]

[0159] In the formula, N is the number of tests, μ i represents the overall accuracy of the model in the i-th test, represents the average value of the overall accuracy of the model output in all tests, σi Represents the standard deviation of the output data of the representative model in the i-th test. S CDI The value of S is between 0 and 1, where 1 indicates perfect consistency and 0 indicates no consistency.

[0160] Model stability analysis index S MSA It is used to measure the stability of the model during long-term operation, observing whether there are fluctuations or decays in the model. The calculation formula is as follows:

[0161]

[0162] In the formula, T represents the observation time period for model stability evaluation, and V t represents the output variance of the model at time t. V init represents the output variance of the model at the initial moment. S MsA The value of S is between 0 and 1, where 1 indicates complete stability and 0 indicates no stability.

[0163] Step S08: Conduct a comprehensive evaluation with weights to obtain the model score.

[0164] Specifically, conduct a weighted comprehensive evaluation of each dimension of the model to calculate the final score (MCE) of the model. The calculation formula is as follows:

[0165] MCE = η1·ME all + η2·ERS + η3·ITP + η4·EC + η5·RC

[0166] In the formula, η1, η2, η3, η4, η5 represent the weight coefficients of the accuracy evaluation, robustness evaluation, interpretability evaluation, efficiency evaluation, and reliability evaluation of the model respectively. Their values are between 0 and 1, and η1 + η2 + η3 + η4 + η5 = 1.

[0167] Therefore, according to what is shown in the evaluation dimension table in step S01, it can be obtained that:

[0168] For the measurement model and the prediction model, η2 = η3 = 0, and η1 + η4 + η5 = 1.

[0169] For the control model and the optimization model, η3 = 0, and η1 + η2 + η4 + η5 = 1.

[0170] For the decision-making model and the simulation model, η2 = η5 = 0, and η1 + η3 + η4 = 1.

[0171] The above steps are used to solve the problem that the industrial intelligent model in the aluminum smelting industry lacks a scientific evaluation method.

[0172] Although the embodiments of the present invention have been shown and described above, it can be understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Modifications, alterations, substitutions, and variations made by those of ordinary skill in the art to the above embodiments all fall within the scope of the present invention.

Claims

1. An industrial intelligent model evaluation method for aluminum smelting industry, characterized in that: The steps are as follows: Step S01: obtaining the industrial intelligence model to be evaluated, and classifying the model according to the model usage characteristics; Step S02: Confirm the input and output data requirements of the model to be evaluated and reserve a data interface. Step S03: Selecting data for model evaluation; Step S04: pre-processing the data; Step S05: input data into the model and obtain model output data; Step S06: constructing an evaluation data set; Step S07: Evaluate each evaluation dimension of the model; Step S08: Perform weighted comprehensive evaluation to obtain a model score.

2. The industrial intelligent model evaluation method for aluminum smelting industry according to claim 1, characterized in that: In the step S01, when obtaining the industrial intelligence model to be evaluated, if the industrial intelligence model is being used in the industrial system, the industrial intelligence model is first completely exported, and then the model is evaluated, and the industrial intelligence model of the aluminum smelting industry is divided into: soft measurement model, prediction model, control model, optimization model, decision model and industrial simulation model; In the step S02, the category of the exported industrial intelligence model is confirmed, the input and output data requirements of the model are confirmed, including the dimension and format of the data, and the data interface of the model is developed in the software system.

3. The industrial intelligent model evaluation method for aluminum smelting industry according to claim 1, characterized in that: In step S04, the method used for preprocessing the data used for model evaluation includes the following steps: Step S41: unify data sampling interval; Step S42: data missing check and processing; Step S43: data noise detection and processing; Specifically: in the evaluation process, outliers are screened out from the noise data, and outlier screening is performed for each indicator data; First calculate the standard deviation of the sample: Among them, x i is the sample value, is the sample mean, n is the sample size; Then calculate the error of each individual data point from the mean: Determine |Δx i |The size of the i |>ρ·σ, then x i The values ​​marked as suspected outliers are between 2.0 and 4.0; If three consecutive values ​​x i ,x i+1 ,x i+2 If the absolute values ​​of the errors of are all greater than ρ·σ, then it is determined that these three values ​​are not accidental outliers, and x is retained. i ,x i+1 ,x i+2 Three values. Step S44: standardizing or normalizing the data to unify the data dimension; Step S45: According to the selected model input requirements, a structured data set indexed by time is constructed for model evaluation.

4. The industrial intelligent model evaluation method for aluminum smelting industry according to claim 1, characterized in that: In the step S07, the evaluation content of the model includes: S07-A model accuracy evaluation; S07-B model robustness evaluation; S07-C model interpretability evaluation; S07-D model efficiency evaluation; S07-E model reliability evaluation; different types of models require different dimensions to be evaluated; among them, the measurement model and the prediction model are evaluated for their accuracy, efficiency and reliability; the control model and the optimization model are evaluated for their accuracy, robustness, efficiency and reliability; the decision model and the simulation model are evaluated for their accuracy, interpretability and efficiency.

5. The model evaluation content according to claim 4, characterized in that S07-A Model accuracy evaluation, using the evaluation data set constructed in step S06 to evaluate the accuracy of the model: For a model with only one output variable, the root mean square error (RMSE) is used to evaluate whether the model output is accurate: In the formula, y i is the output of each calculation of the model, Represents the model output control data; For a model with M output variables, the root mean square relative error (RMSRE) is used to evaluate the accuracy of the multi-output model: Where y ij represents the output value of the jth run of the i-th output variable of the model, represents the control value of the jth run of the i-th output variable of the model; If the output value y ij When it contains 0, update RMSRE to When evaluating the overall accuracy of the model, all model outputs are weighted summed using the following formula: In the formula, θ j is the weight of the jth output. If all outputs are equally important, then θ j =1.

6. The model evaluation content according to claim 4, characterized in that S07-B: Model robustness evaluation is to artificially add random noise to the data set constructed in step S03 to form samples with perturbation noise or adversarial samples, and input the samples into the model to check whether the output of the model is still accurate; If the model has only one output, the empirical robustness score is as follows: In the formula, n represents the number of samples, y i Represents the model output obtained by inputting the original sample into the model, Represents the model output obtained by inputting noisy samples or adversarial samples into the model. Represents a distance metric function, using L2 distance (Euclidean distance, dimension K): Or cosine similarity: If the model has M outputs, the empirical robustness score formula is: The value of the empirical robustness score is between 0 and 1. The closer the value is to 1, the better the robustness of the model is and the less affected it is by perturbations and adversarial samples.

7. The model evaluation content according to claim 4, characterized in that The S07-C model interpretability evaluation adopts the subjective judgment of technical experts. The interpretability of industrial models is evaluated by technical experts to see whether the model interpretation is reasonable, whether it conforms to the actual situation, and whether it can clearly reveal the decision logic of the model, and manually give an interpretability ITP score.

8. The model evaluation content according to claim 4, characterized in that The S07-D model efficiency evaluation includes three aspects: model training or establishment efficiency, model calculation efficiency, and model resource consumption. The efficiency evaluation formula is as follows: Where, T Train Represents the time taken to train or build the model, T Calc It represents the time taken by the model to calculate each result, R represents the computing power resources used by the model, α, β, γ represent the weight coefficients of each efficiency, and α+β+γ=1.

9. The model evaluation content according to claim 4, characterized in that The reliability evaluation of S07-E model is the output data consistency index S CDI and model stability analysis index S MSA The weighted sum of : RC=ω1S CDI +ω2S MSA ; Where ω1 and ω2 are weight coefficients, ranging from 0 to 1, and ω1+ω2=1; Data consistency index S CDI It is used to measure the output consistency of the model at different times and in different environments. The calculation formula is as follows: Where N is the number of tests, μ i represents the overall accuracy of the model in the i-th test, represents the average overall accuracy of the model output in all tests, σ i Represents the standard deviation of the model's output data in the i-th test, S CDI The value of is between 0 and 1, 1 indicates perfect consistency and 0 indicates no consistency; Model stability analysis index S MSA It is used to measure the stability of the model during long-term operation and observe whether the model fluctuates or decays. The calculation formula is as follows: Where T represents the observation period for model stability assessment, V t represents the output variance of the model at time t, V init Represents the output variance of the model at the initial moment, S MSA The value of is between 0 and 1, with 1 indicating complete stability and 0 indicating no stability.

10. The industrial intelligent model evaluation method for aluminum smelting industry according to claim 1, characterized in that: In step S08, the evaluation of each dimension of the model in step S07 is finally weighted and comprehensively evaluated to calculate the final score of the model (MCE), and the calculation formula is as follows: MCE=η1·ME all +η2·ERS+η3·ITP+η4·EC+η5·RC Where η1, η2, η3, η4, η5 represent the weight coefficients of the model's accuracy evaluation, robustness evaluation, interpretability evaluation, efficiency evaluation, and reliability evaluation, respectively, with values ​​between 0 and 1, and η1+η2+η3+η4+η5=1; That is, according to the evaluation dimension table in step S01, the weight is specifically expressed as: The η2=η3=0 and η1+η4+η5=1 of the measurement model and the prediction model; The control model and the optimization model have η3=0, η1+η2+η4+η5=1; For the decision model and simulation model, η2=η5=0, η1+η3+η4=1.

Images