Multiplexing model confidence assessment method, apparatus, and server

Abstract

This invention provides a method, apparatus, and server for evaluating the confidence of a reused model, relating to the technical field of model evaluation. The method includes: comparing a reused model with an instance library to determine the similarity of conditional parameters, and constructing a similarity matrix using the conditional parameter similarity; performing conditional parameter sensitivity analysis on the reused model to determine sensitivity weights; using a preset similarity calculation model, based on the sensitivity weights and the similarity matrix, determining the model similarity between the reused model and each model in the instance library; and using an error expectation calculation model, based on the model similarity, determining the expected error between the simulated value of the reused model and the experimental value of the actual model, and determining the confidence of the reused model and the confidence evaluation result based on the expected error value. This invention can significantly improve computational accuracy and efficiency.

Description

Technical Field

[0001] This invention relates to the technical field of model evaluation, and in particular to a method, apparatus and server for evaluating the confidence of a reusable model. Background Technology

[0002] Currently, the Delphi method, also known as the expert survey weighting method, is commonly used to determine the influence weights of conditional parameters on the output results. However, this method is easily affected by the number of experts, their domain knowledge, and their subjective judgment. Related technologies suggest that sensitivity analysis or experimental design can be used to obtain objective parameter sensitivity results. However, for models with many conditional parameters and complex relationships between them, the number of samplings required for sensitivity analysis or experimental design increases exponentially, resulting in extremely high computational costs. Furthermore, when simultaneously evaluating conditional parameters with well-defined mechanisms (i.e., pure function factors) and conditional parameters involving black-box models (i.e., non-pure function factors), the computational accuracy is affected by the sample points being sampled. Therefore, using sensitivity analysis or experimental design alone results in low computational accuracy and efficiency, making it difficult to handle high-dimensional problems. Summary of the Invention

[0003] In view of this, the purpose of the present invention is to provide a method, apparatus and server for evaluating the confidence of a reusable model, which can significantly improve computational accuracy and efficiency.

[0004] In a first aspect, embodiments of the present invention provide a method for evaluating the confidence of a reused model. The method is applied to a reused model confidence evaluation system and includes: acquiring a reused model and an instance library; determining the similarity of conditional parameters by comparing the reused model and the instance library; constructing a similarity matrix using the conditional parameter similarity; performing conditional parameter sensitivity analysis on the reused model to determine sensitivity weights; determining the model similarity between the reused model and each model in the instance library using a preset similarity calculation model, based on the sensitivity weights and the similarity matrix; and determining the expected error value between the simulated value of the reused model and the experimental value of the actual model using an error expectation calculation model, based on the model similarity; and determining the confidence of the reused model and the confidence evaluation result based on the expected error value.

[0005] In one implementation, the step of determining the similarity of conditional parameters by comparing the reused model and the instance library includes: using a static data similarity calculation model and a dynamic data similarity calculation model respectively to compare and analyze the static data and dynamic data in the conditional parameters to determine the similarity of the conditional parameters.

[0006] In one implementation, the step of performing sensitivity analysis on conditional parameters of a reused model to determine sensitivity weights includes: selecting pure function factors and impure function factors from the conditional parameters according to a preset factor value range; performing sensitivity analysis and variance analysis on the pure function factors and impure function factors respectively to determine the sensitivity analysis results and variance analysis results, wherein the sensitivity analysis results are used to represent the significance of the influence of pure function factors on the model output results, and the variance analysis results are used to determine the significance of the influence of impure function factors on the model output results; and performing sensitivity normalization processing on the sensitivity analysis results and variance analysis results to determine the sensitivity weights of the conditional parameters.

[0007] In one implementation, the steps of performing sensitivity analysis and variance analysis on pure function factors and impure function factors respectively to determine the sensitivity analysis results and variance analysis results include: performing sensitivity analysis on pure function factors to determine the sensitivity analysis results, and determining the factor with the highest sensitivity among the pure function factors as the reference factor; setting up orthogonal experiments on impure function factors based on the impure function factors and the reference factor, and performing variance analysis on the simulation results of the orthogonal experiments to determine the variance analysis results.

[0008] In one implementation, the step of determining the model similarity between the reused model and each model in the instance library based on the sensitivity weight and the similarity matrix using a preset similarity calculation model includes: determining the model similarity by multiplying the sensitivity weight by the transpose of the similarity matrix.

[0009] In one implementation, before the step of determining the expected error between the simulated value of the reused model and the experimental value of the actual model based on model similarity using the expected error calculation model, the method includes: obtaining a test model from an instance library, and determining the expected error between the simulated value and the experimental value of the test model, the relative error between the test model and the actual error, and the average relative error based on a preset relative error calculation model; and determining the regularization term coefficient with the smallest average relative error as the target regularization term coefficient by performing global optimization processing on the test model.

[0010] In one implementation, the step of determining the expected error between the simulated value of the reused model and the experimental value of the actual model based on model similarity using an error expectation calculation model includes: substituting the model similarity, the simulated value of the reused model, and the experimental value of the actual model into the error expectation calculation model to determine the expected error value, wherein the error expectation calculation model is expressed as:

[0011] in, For loss function, For instance library The predicted values ​​of a simulation model, For the experimental values ​​corresponding to the actual model, The regularization term characterizes the complexity of the model. The coefficient of the objective regularization term is used to balance model error and model complexity.

[0012] Secondly, embodiments of the present invention also provide a reuse model confidence assessment device. The device is applied to a reuse model confidence assessment system and includes: a similarity matrix calculation module, which acquires a reuse model and an instance library, and determines the conditional parameter similarity by comparing the reuse model and the instance library, thereby constructing a similarity matrix using the conditional parameter similarity; a sensitivity weight calculation module, which performs conditional parameter sensitivity analysis on the reuse model to determine the sensitivity weight; and a confidence assessment module, which uses a preset similarity calculation model, based on the sensitivity weight and the similarity matrix, to determine the model similarity between the reuse model and each model in the instance library, and uses an error expectation calculation model, based on the model similarity, to determine the expected error value between the simulated value of the reuse model and the experimental value of the actual model, thereby determining the confidence of the reuse model and the confidence assessment result based on the expected error value.

[0013] Thirdly, embodiments of the present invention also provide a server, including a processor and a memory, the memory storing computer-executable instructions executable by the processor, the processor executing the computer-executable instructions to implement any of the methods provided in the first aspect.

[0014] Fourthly, embodiments of the present invention also provide a computer-readable storage medium storing computer-executable instructions, which, when invoked and executed by a processor, cause the processor to implement any of the methods provided in the first aspect.

[0015] The embodiments of the present invention bring the following beneficial effects: This invention provides a method, apparatus, and server for evaluating the confidence of a reused model. After acquiring a reused model and an instance library, the method compares the reused model and the instance library to determine the similarity of conditional parameters, constructs a similarity matrix using the conditional parameter similarity, performs conditional parameter sensitivity analysis on the reused model to determine sensitivity weights, and finally uses a preset similarity calculation model. Based on the sensitivity weights and the similarity matrix, it determines the model similarity between the reused model and each model in the instance library. It also uses an error expectation calculation model, based on the model similarity, to determine the expected error between the simulation value of the reused model and the experimental value of the actual model. Based on the expected error value, the confidence of the reused model and the confidence evaluation result are determined. This invention significantly improves the accuracy of the confidence evaluation results of the reused model by comprehensively considering the influence weights of each conditional parameter on the output results of the reused model, the model errors of the reused model, the standard model, and each instance model, as well as the complexity of the model.

[0016] Other features and advantages of the invention will be set forth in the description which follows, and will be apparent in part from the description, or may be learned by practicing the invention. The objects and other advantages of the invention are realized and obtained in accordance with the structures particularly pointed out in the description, claims and drawings.

[0017] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0018] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0019] Figure 1 A flowchart illustrating a confidence assessment method for a reused model provided in an embodiment of the present invention; Figure 2 A schematic diagram illustrating the specific process of a confidence assessment method for a reusable model based on factor analysis and expectation test, provided for an embodiment of the present invention; Figure 3 A schematic diagram illustrating the specific process of a method for calculating the sensitivity weights of conditional parameters based on sensitivity analysis and experimental design, provided in an embodiment of the present invention; Figure 4 This is a schematic diagram of the structure of a reusable model confidence assessment device provided in an embodiment of the present invention; Figure 5This is a schematic diagram of the structure of a server provided in an embodiment of the present invention. Detailed Implementation

[0020] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the present invention will be clearly and completely described below in conjunction with the embodiments. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0021] Currently, the confidence level of a reuse model can be measured by expected risk. To evaluate:

[0022] The inputs and outputs of the standard model It is a random variable that follows a joint probability distribution. Expected risk loss function The integral after multiplying by the joint probability distribution gives the expected risk as theoretically the standard model. The average cost loss of the joint probability distribution characterizes the standard model’s ability to fit the reused model and all instance models. Therefore, it is also called the expected loss or risk function. The loss function can be obtained through certain calculation criteria, but the joint probability distribution is unknown, so the expected risk cannot be solved.

[0023] Furthermore, considering that the instance models are known, related techniques propose that the instance models can be used to approximate the entire model, thus obtaining the fitting ability of the standard model to the instance models, and using empirical risk. To indicate:

[0024] Empirical risk calculates the loss function of all instance models and then averages them, given the sample size. As it approaches infinity, empirical risk approaches expected risk. However, empirical risk is prone to overfitting, which causes it to perform well in instance models but poorly in reused models. At the same time, empirical risk does not take into account the characteristics of different scenario models.

[0025] Since a sufficient number of instance models are often unavailable in practical applications, or large-scale instance libraries cannot be accessed due to limitations in computing resources, existing technologies add a penalty term (i.e., a regularization term) to the empirical risk to reduce the risk of overfitting and thus obtain structural risk. Calculation method:

[0026] in, The regularization term characterizes the complexity of the model being evaluated. is the regularization coefficient, used to control the strength of regularization in order to balance empirical risk and model complexity.

[0027] Although structural risk introduces complexity into the evaluated model and reduces the risk of overfitting, it does not consider the similarity between the reused model, the standard model, and the instance models, nor does it consider the different influences of each input parameter on the output results. Therefore, the reused model confidence assessment method, apparatus, and server provided in this invention can divide all conditional parameters into pure function factors and impure function factors. The significance of changes in input conditional parameters on the model output results is calculated separately through sensitivity analysis and experimental design. The pure function factor with the highest sensitivity is introduced into the impure function factor experimental design. Using the sensitivity of this pure function factor as a reference value, the results of the pure function factor sensitivity analysis and the impure function factor experimental design are calibrated and normalized to obtain the sensitivity weights of the conditional parameters. Compared to schemes where all conditional parameters participate in a single evaluation process (either all in sensitivity analysis or all in experimental design), this reduces the computational scale and improves computational accuracy and efficiency.

[0028] See Figure 1 The diagram shows a flowchart of a confidence assessment method for a reuse model. The method mainly includes the following steps S102 to S106: Step S102: Obtain the reuse model and instance library, and determine the condition parameter similarity by comparing the reuse model and instance library, so as to construct a similarity matrix using the condition parameter similarity. The similarity matrix consists of the condition parameter similarity of the reuse model and each instance library model. In one implementation, the condition parameters can be adjusted according to the equipment design requirements to construct the reuse model, and the standard model and other similar models that have been tested and verified are summarized as the instance library.

[0029] Step S104: Perform conditional parameter sensitivity analysis on the reuse model to determine the sensitivity weights.

[0030] Step S106: Using a preset similarity calculation model, based on sensitivity weights and a similarity matrix, determine the model similarity between the reused model and each model in the instance library. Then, using an error expectation calculation model, based on the model similarity, determine the expected error between the simulated value of the reused model and the experimental value of the actual model. Based on the expected error value, determine the confidence level of the reused model and the confidence level evaluation result. In one implementation, after determining the confidence level evaluation result, it is necessary to evaluate the confidence level of the reused model and conduct experimental verification. Specifically, when evaluating the confidence level of the reused model, it is necessary to consider the expected error between the simulated value of the reused model and the experimental value of the actual model. Calculate the confidence score of the reuse model:

[0031] The confidence level C of the reused model characterizes the reliability of the simulation values ​​of the reused model. In the equipment design process, the expected value of the error between the simulation values ​​of the reused model and the actual model test values ​​can be used as a reference. Based on the evaluation results of the confidence level C of the reuse model, the condition parameters of the reuse model are corrected. By performing the above confidence level evaluation steps, the confidence level of the corrected reuse model is evaluated until a condition parameter setting scheme and reuse model with a confidence level that meets the requirements are obtained.

[0032] In addition, physical experiments can be conducted based on the final determined condition parameters to verify whether the design scheme meets the requirements. At the same time, the reuse model and its verification experiment can be included in the instance library for the design and development of reuse models for similar scenarios in the future.

[0033] The influence weights of each model condition parameter in the instance library on the model's output. Conditional parameter sensitivity variance Loss function These are inherent data that do not change with the parameter settings of the reuse model. Therefore, after completing the initial confidence assessment of the reuse model, it is only necessary to supplement the existing data with new instance data to obtain new data. , and This greatly reduces the computational burden by eliminating the need to perform traversal calculations again. Subsequent evaluations only require traversing the instance library when calculating the conditional parameter similarity S between the reuse model and each instance library model.

[0034] The confidence assessment method for reused models provided in this invention divides all conditional parameters into pure function factors and impure function factors. It calculates the significance of the influence of changes in input conditional parameters on the model output results through sensitivity analysis and experimental design. The pure function factor with the highest sensitivity is introduced into the experimental design of impure function factors. Using the sensitivity of the pure function factor as a reference value, the results of sensitivity analysis of pure function factors and experimental design of impure function factors are calibrated and normalized to obtain the sensitivity weights of conditional parameters. This method can significantly improve the calculation accuracy and efficiency.

[0035] See Figure 2 The diagram shows a specific flowchart of a confidence assessment method for a reuse model based on factor analysis and expectation test. This embodiment of the invention also provides an implementation method for assessing the confidence of a reuse model, as detailed in (1) to (4) below: (1) Construct a reuse model and instance library, and calculate the condition parameter similarity and similarity matrix. Specifically, when constructing a reuse model, the condition parameters can be adjusted according to the equipment design requirements based on the standard model or other similar models to obtain the reuse model. When constructing an instance library, the standard model and other similar models that have been tested and verified can be summarized as an instance library. When calculating the condition parameter similarity, the reuse model and the instance library model need to be compared one by one. Among them, the condition parameters are the parameters that determine the model architecture. For example, for a rocket engine model, the shape of the propellant grain will directly affect the model architecture. Therefore, its characteristic parameters, such as the generation rate of combustion products, can be used as the condition parameters of the model. Within the applicable range of the model, the influence of the nozzle throat diameter, nozzle expansion ratio, and nozzle back pressure on the confidence of the output result (thrust) is certain. Their values ​​do not affect the model architecture. Therefore, they are not used as the condition parameters of the model.

[0036] In one implementation, static data similarity calculation models and dynamic data similarity calculation models can be used respectively to compare and analyze the static and dynamic data in the conditional parameters to determine the similarity of the conditional parameters. Specifically, for the similarity of static data:

[0037] in, To reuse a certain conditional parameter of the model, These are the conditional parameters corresponding to the instance library model.

[0038] For dynamic data, the similarity is represented by the square root of the product of the mean accuracy and the grey relational degree based on sequence similarity and proximity: Let [denote...]. For the first The sequence at time... The value, This is the first sequence. This is the second sequence. The mean of all time points in the first sequence is given. The mean of all time points in the second sequence. For sequence For sequence The mean accuracy, For sequence For sequence The grey relational degree is calculated by using the reused model parameter sequence as the first sequence and the instance library model parameter sequence as the second sequence. For sequence The accuracy of the mean can be calculated using the following formula:

[0039] Let sequence , ,like Make ( for If a sequence is a constant vector, then it is called a sequence. Fully correlated sequence ,like ,say and Positive correlation; if ,say and Negative correlation.

[0040] sequence For sequence The grey relational degree can be calculated through the following steps: 1. Connect the sequences with straight lines respectively. and sequence At each point in time, we obtained and Their respective sequence curves.

[0041] 2. The sequence curve is shifted vertically to obtain the shifted sequence curve. Starting point and sequence curve The starting points coincide. Clearly, the sequence obtained by translation... Fully correlated sequence Calculate the sequence For sequence The grey relational degree is equivalent to calculating the sequence For sequence The grey relational degree.

[0042] 3. Calculate the sequence curve and sequence curve The area between them, i.e., the similarity factor S:

[0043] 4. Calculate the sequence curve slope and sequence curve The slope between :

[0044]

[0045] 5. Calculate the similarity influence factor of grey relational analysis. :

[0046] in, Factors with positive and negative correlation:

[0047] 6. Calculate the sequence For sequence Grey relational analysis based on sequence similarity and proximity:

[0048] Calculate the dynamic data similarity based on the accuracy of the dynamic data mean and the grey relational analysis based on sequence similarity and proximity:

[0049] If a model in the instance library does not involve a certain condition parameter of the reused model, the similarity is recorded as 0.

[0050] In another implementation, a similarity matrix is ​​constructed: the similarity matrix S consists of the conditional parameter similarity between the reused model and each instance library model.

[0051] Where m is the number of models contained in the instance library, and n is the number of condition parameters of the reused model. The similarity between the j-th conditional parameter of the reused model and the corresponding conditional parameter of the i-th model in the instance library.

[0052] (2) When calculating sensitivity weights: Conduct sensitivity analysis of conditional parameters for the reuse model and calculate the influence weight of each conditional parameter on the output of the reuse model. .

[0053] (3) Calculate the similarity between the reused model and each model in the instance library. The model similarity can be determined by multiplying the sensitivity weights by the transpose of the similarity matrix. Specifically, the similarity between the reused model and the instance library models... Based on sensitivity weight The similarity matrix S is obtained as follows:

[0054] (4) The expected value of the error between the simulation value of the reused model and the experimental value of the actual model is determined by calculating the regularization term and its coefficient. A test model is obtained from the instance library, and based on the preset relative error calculation model, the expected value of the error between the simulation value and the experimental value of the test model, the relative error with the actual error, and the average relative error are determined. Then, through global optimization of the test model, the regularization term coefficient with the smallest average relative error is determined as the target regularization term coefficient. After determining the target regularization term coefficient, the model similarity, the simulation value of the reused model, and the experimental value of the actual model are substituted into the error expectation calculation model to determine the expected error value. The error expectation calculation model is expressed as:

[0055] in, For loss function, For instance library The predicted values ​​of a simulation model, For the experimental values ​​corresponding to the actual model, The regularization term characterizes the complexity of the model. The coefficients of the objective regularization term are used to balance model error and model complexity; the loss function... The specific form is determined based on the relationship between the source of loss and the output result, including but not limited to mean absolute error, mean square error, etc.

[0056] Specifically, for each instance library model, sensitivity analysis of the conditional parameters is performed to calculate the influence weight of each conditional parameter on the model's output:

[0057] If a model does not involve a certain condition parameter, the sensitivity of that parameter is recorded as 0.

[0058] For each instance library model, calculate the sensitivity variance of the conditional parameters:

[0059] In this model, if a certain conditional parameter is not involved, its sensitivity is treated as zero in the variance calculation. The sensitivity is greater than or equal to 0, and the sum of the conditional parameter sensitivities for each model equals 1. Therefore, the variance of the conditional parameter sensitivity falls within the range of [0, 0.25]. The larger the variance, the lower the model complexity. The regularization term is obtained by subtracting the variance of the model's conditional parameter sensitivity from 0.25.

[0060] Using one model from the instance library as the test model and the remaining models as the test instance library, calculate the similarity between the test model and the models in the test instance library. Then, using each model in the instance library as a test model, calculate and test the similarity between the model and the model in the instance library. Substitute these similarities, along with the conditional parameter sensitivity variance, into the formula for the expected error of the model simulation value and the experimental value.

[0061] Calculate the relative error between the expected value and the actual error of the simulation and experimental values ​​of the verification model, one by one:

[0062] The relative errors between the expected and actual errors of the simulation and experimental values ​​of the verification model are averaged:

[0063] We conduct global optimization in the real number domain to find the average of the relative errors between the expected and actual errors of the simulation and experimental values ​​of the verification model. The smallest value of the regularization coefficient is used as the regularization coefficient value for calculating the expected value of the error between the simulation value of the reused model and the experimental value of the actual model.

[0064] In other words, the above method for calculating the regularization term and its coefficient involves performing sensitivity analysis on each conditional parameter of the instance library model, calculating the influence weight of each conditional parameter on the model's output, calculating the variance of the conditional parameter sensitivity, and subtracting the variance of the model's conditional parameter sensitivity from 0.25 to obtain the regularization term. The value of is determined by taking a certain model in the instance library as the test model and the remaining models as the test instance library. The relative error between the expected value and the actual error of the test model's simulation value and experimental value is calculated one by one. Global optimization in the real number domain is carried out to find the regularization coefficient value that minimizes the average value of the relative error between the expected value and the actual error of the test model's simulation value and experimental value. This value is then used as the regularization coefficient value for calculating the expected value of the error between the reused model's simulation value and the actual model's experimental value.

[0065] Finally, the similarity between the reused model and the instance library model, the loss function of the standard model and other experimentally validated similar models, the regularization term of the reused model and its coefficients are substituted into the formula for the expected value of the error between the simulated value of the reused model and the experimental value of the actual model (i.e., the error expectation calculation model) to obtain the expected value of the error between the simulated value of the reused model and the experimental value of the actual model. The above method for calculating the expected value of the error comprehensively considers the influence weight of each condition parameter on the output result of the reused model, the model error of the reused model, the standard model and each instance model, and the complexity of the model. It can obtain more accurate and more comprehensive confidence evaluation results of the reused model than traditional methods.

[0066] See Figure 3 The diagram shows a specific flowchart of a method for calculating the sensitivity weights of conditional parameters based on sensitivity analysis and experimental design. This embodiment of the invention also provides an implementation method for calculating the weights of conditional parameters, as detailed in (1) to (3) below: (1) Based on the preset factor value range, pure function factors and non-pure function factors are selected from the condition parameters. In one implementation, pure function factors are determined by selecting pure function factors from all condition parameters. Pure function factors are factors that output the same when the input is the same and do not produce side effects. The behavior of pure function factors is not affected by the environment. For the same input, the same output will be returned. Pure functions do not modify the external state or produce additional effects during execution. The value range of pure function factors is determined by first determining the value range of all condition parameters according to the equipment design requirements, and then determining the value range of pure function factors according to the pure function factor selection results.

[0067] In one implementation, non-pure function factors are identified: all conditional parameters that do not meet the screening criteria for pure function factors are considered as non-pure function factors and participate in experimental design and analysis of variance. The weights of their influence on the model output are calculated, and the range of values ​​for non-pure function factors is determined based on the value range of all the above conditional parameters and the screening results for non-pure function factors.

[0068] (2) Sensitivity analysis and variance analysis are performed on pure function factors and impure function factors respectively to determine the results of sensitivity analysis and variance analysis. The results of sensitivity analysis are used to represent the significance of the influence of pure function factors on the model output results, and the results of variance analysis are used to determine the significance of the influence of impure function factors on the model output results. Specifically, sensitivity analysis can be performed on pure function factors to determine the results of sensitivity analysis, and the factor with the highest sensitivity among the pure function factors can be determined as the reference factor. Based on the impure function factors and the reference factor, orthogonal experiments of impure function factors are set up, and variance analysis is performed on the simulation results of the orthogonal experiments to determine the results of variance analysis. In one implementation, sensitivity analysis is used to evaluate the model output. Methods for determining the sensitivity of results to changes in input condition parameters include commonly used factor sensitivity analysis methods such as the Morris method, FAST method, and Sobol' method. Different factor sensitivity analysis methods have different calculation schemes and their own application characteristics. A suitable factor sensitivity analysis method can be selected based on the actual situation, such as factor size, interaction effects, computational scale, and computational accuracy, to calculate the sensitivity of the pure function factors of the model. Furthermore, the pure function factor with the highest sensitivity needs to be introduced as a reference. The condition parameter with the highest sensitivity is selected from all pure function factors and introduced into the non-pure function factor experimental design process. The sensitivity of this pure function factor will be used as a reference value to calibrate the results of pure function factor sensitivity analysis and non-pure function factor experimental design.

[0069] In another implementation, orthogonal experimental design is used for non-pure function factors. Orthogonal experimental design is a method for studying multi-factor, multi-level experiments. It can select representative, uniformly distributed, and comparable points from the full experiment based on the range of values ​​of the non-pure function factors. With the fewest number of experiments, it can obtain results equivalent to the full experiment. According to the orthogonal experimental design scheme, simulation is carried out based on the reused model to obtain simulation results for analysis of variance. After the simulation is completed according to the orthogonal experimental design scheme, analysis of variance is performed on the simulation results to obtain the significance of the influence of each non-pure function factor and its interaction on the model output results.

[0070] (3) Perform sensitivity normalization on the sensitivity analysis results and variance analysis results to determine the sensitivity weight of the conditional parameters. In one implementation, when normalizing the sensitivity of the conditional parameters, it is necessary to summarize the sensitivity analysis of pure function factors and the experimental design results of non-pure function factors. The pure function factors introduced into the experimental design process of non-pure function factors are used as a common reference to normalize the sensitivity of all conditional parameters. Then, the sensitivity normalization result of the conditional parameters is used as the sensitivity weight of the conditional parameters.

[0071] In other words, the above-mentioned method for calculating sensitivity weights divides all conditional parameters into pure function factors and impure function factors. The significance of the changes in input conditional parameters on the model output results is calculated through sensitivity analysis and experimental design, respectively. The pure function factor with the highest sensitivity is introduced into the experimental design of impure function factors. The sensitivity of the pure function factor is used as a reference value to calibrate the results of the sensitivity analysis of pure function factors and the experimental design of impure function factors. The results are then normalized to obtain the sensitivity weights of the conditional parameters. This method can significantly reduce the computational scale and improve computational accuracy and efficiency.

[0072] In summary, the influence weights of each model condition parameter in the instance library on the model output result in this invention are as follows: Conditional parameter sensitivity variance Loss function These are inherent data that do not change with the parameter settings of the reuse model. Therefore, after completing the initial confidence assessment of the reuse model, it is only necessary to supplement the existing data with new instance data to obtain new data. , and This eliminates the need for re-performing traversal calculations, significantly reducing the computational burden. Subsequent evaluation only requires calculating the conditional parameter similarity between the reused model and each instance library model. The instance library is traversed at that time.

[0073] Furthermore, compared to existing technologies, the method for calculating the expected value of the error between the simulation value of the reused model and the experimental value of the actual model in this invention comprehensively considers the influence weight of each condition parameter on the output result of the reused model, the model error of the reused model, the standard model and each instance model, and the complexity of the model. This method can obtain more accurate and more comprehensive confidence evaluation results of the reused model than traditional methods.

[0074] Finally, the proposed method for calculating the sensitivity weights of conditional parameters based on sensitivity analysis and experimental design divides all conditional parameters into pure function factors and impure function factors. Sensitivity analysis and experimental design are used to calculate the significance of changes in input conditional parameters on the model output results. The pure function factor with the highest sensitivity is introduced into the impure function factor experimental design. Using the sensitivity of this pure function factor as a reference value, the results of the pure function factor sensitivity analysis and the impure function factor experimental design are calibrated and normalized to obtain the sensitivity weights of the conditional parameters. Compared to schemes where all conditional parameters participate in a single evaluation process (either all in sensitivity analysis or all in experimental design), this method reduces the computational scale and improves computational accuracy and efficiency.

[0075] Regarding the confidence assessment method for reused models provided in the foregoing embodiments, this invention provides a confidence assessment device for reused models. This device is applied to a confidence assessment system for reused models. See [link to relevant documentation]. Figure 4 The diagram shows a structural schematic of a confidence assessment device for a reused model, which includes the following parts: The similarity matrix calculation module 402 obtains the reused model and instance library, and determines the conditional parameter similarity by comparing the reused model and instance library, so as to construct a similarity matrix using the conditional parameter similarity. Sensitivity weight calculation module 404 performs conditional parameter sensitivity analysis on the reuse model to determine the sensitivity weight; The confidence assessment module 406 uses a preset similarity calculation model to determine the model similarity between the reused model and each model in the instance library based on sensitivity weights and similarity matrices. It also uses an error expectation calculation model to determine the expected error between the simulation value of the reused model and the experimental value of the actual model based on the model similarity. The confidence of the reused model and the confidence assessment result are then determined based on the expected error value.

[0076] The reuse model confidence evaluation device provided in this application embodiment can significantly improve computational accuracy and computational efficiency.

[0077] In one implementation, when performing the step of determining the similarity of condition parameters by comparing the reuse model and the instance library, the similarity matrix calculation module 402 is further used to: use the static data similarity calculation model and the dynamic data similarity calculation model respectively to compare and analyze the static data and dynamic data in the condition parameters to determine the similarity of the condition parameters.

[0078] In one embodiment, when performing sensitivity analysis on the conditional parameters of the reused model to determine the sensitivity weights, the sensitivity weight calculation module 404 is further configured to: select pure function factors and impure function factors from the conditional parameters according to a preset factor value range; perform sensitivity analysis and variance analysis on the pure function factors and impure function factors respectively to determine the sensitivity analysis results and variance analysis results, wherein the sensitivity analysis results are used to represent the significance of the pure function factors' influence on the model output results, and the variance analysis results are used to determine the significance of the impure function factors' influence on the model output results; and perform sensitivity normalization processing on the sensitivity analysis results and variance analysis results to determine the sensitivity weights of the conditional parameters.

[0079] In one embodiment, when performing sensitivity analysis and variance analysis for pure function factors and impure function factors respectively to determine the sensitivity analysis results and variance analysis results, the sensitivity weight calculation module 404 is further configured to: perform sensitivity analysis for pure function factors to determine the sensitivity analysis results, and determine the factor with the highest sensitivity among the pure function factors as the reference factor; based on the impure function factors and the reference factor, set up orthogonal experiments for the impure function factors, and perform variance analysis processing on the simulation results of the orthogonal experiments to determine the variance analysis results.

[0080] In one embodiment, when performing the step of determining the model similarity between the reused model and each model in the instance library based on the sensitivity weight and the similarity matrix using a preset similarity calculation model, the confidence evaluation module 406 is further used to: determine the model similarity by multiplying the sensitivity weight by the transpose of the similarity matrix.

[0081] In one embodiment, before performing the step of using the error expectation calculation model to determine the expected error between the simulated value of the reused model and the experimental value of the actual model based on model similarity, the confidence assessment module 406 is further configured to: obtain the test model from the instance library, and based on the preset relative error calculation model, determine the expected error between the simulated value and the experimental value of the test model, the relative error between the test model and the actual error, and the average relative error; and determine the regularization term coefficient with the smallest average relative error as the target regularization term coefficient by performing global optimization processing on the test model.

[0082] In one embodiment, when performing the step of determining the expected error between the simulated value of the reused model and the experimental value of the actual model based on model similarity using the expected error calculation model, the confidence assessment module 406 is further configured to: substitute the model similarity, the simulated value of the reused model, and the experimental value of the actual model into the expected error calculation model to determine the expected error, wherein the expected error calculation model is expressed as:

[0083] in, For loss function, For instance library The predicted values ​​of a simulation model, For the experimental values ​​corresponding to the actual model, The regularization term characterizes the complexity of the model. The coefficient of the objective regularization term is used to balance model error and model complexity.

[0084] The device provided in this embodiment of the invention has the same implementation principle and technical effect as the aforementioned method embodiment. For the sake of brevity, any parts not mentioned in the device embodiment can be referred to the corresponding content in the aforementioned method embodiment.

[0085] This invention provides a server, specifically, the server includes a processor and a storage device; the storage device stores a computer program, which, when run by the processor, executes the method described in any of the above embodiments.

[0086] Figure 5 This is a schematic diagram of the structure of a server provided in an embodiment of the present invention. The server 100 includes: a processor 50, a memory 51, a bus 52 and a communication interface 53. The processor 50, the communication interface 53 and the memory 51 are connected through the bus 52. The processor 50 is used to execute executable modules, such as computer programs, stored in the memory 51.

[0087] The memory 51 may include high-speed random access memory (RAM) or non-volatile memory, such as at least one disk storage device. Communication between this system network element and at least one other network element is achieved through at least one communication interface 53 (which can be wired or wireless), such as the Internet, wide area network, local area network, metropolitan area network, etc.

[0088] Bus 52 can be an ISA bus, PCI bus, or EISA bus, etc. The bus can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 5 The symbol is represented by a single double-headed arrow, but this does not mean that there is only one bus or one type of bus.

[0089] The memory 51 is used to store programs. After receiving an execution instruction, the processor 50 executes the programs. The method executed by the device for defining the flow process disclosed in any of the foregoing embodiments of the present invention can be applied to the processor 50 or implemented by the processor 50.

[0090] Processor 50 may be an integrated circuit chip with signal processing capabilities. In implementation, each step of the above method can be completed by the integrated logic circuitry in the hardware of processor 50 or by instructions in software form. Processor 50 can be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc.; it can also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this invention. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this invention can be directly embodied in the execution of a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software modules can reside in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. The storage medium is located in memory 51. The processor 50 reads the information in memory 51 and, in conjunction with its hardware, completes the steps of the above method.

[0091] The computer program product of the readable storage medium provided in the embodiments of the present invention includes a computer-readable storage medium storing program code. The instructions included in the program code can be used to execute the methods described in the foregoing method embodiments. For specific implementation, please refer to the foregoing method embodiments, which will not be repeated here.

[0092] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, essentially, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0093] Finally, it should be noted that the above-described embodiments are merely specific implementations of the present invention, used to illustrate the technical solutions of the present invention, and not to limit it. The scope of protection of the present invention is not limited thereto. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that any person skilled in the art can still modify or easily conceive of changes to the technical solutions described in the foregoing embodiments within the technical scope disclosed in the present invention, or make equivalent substitutions for some of the technical features; and these modifications, changes, or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be covered within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A method for evaluating the confidence of a reused model, characterized in that, The method is applied to a confidence evaluation system for reused models, and the method includes: Obtain a reused model and an instance library, and determine the conditional parameter similarity by comparing the reused model and the instance library, so as to construct a similarity matrix using the conditional parameter similarity; Sensitivity analysis of conditional parameters is performed on the reuse model to determine the sensitivity weights; Using a preset similarity calculation model, based on the sensitivity weight and the similarity matrix, the model similarity between the reused model and each model in the instance library is determined. Using an error expectation calculation model, based on the model similarity, the expected error value between the simulation value of the reused model and the experimental value of the actual model is determined, so as to determine the confidence level of the reused model and the confidence level evaluation result based on the expected error value.

2. The confidence assessment method for reuse models according to claim 1, characterized in that, The step of determining the similarity of conditional parameters by comparing the reused model with the instance library includes: The static and dynamic data similarity calculation models are used respectively to compare and analyze the static and dynamic data in the condition parameters to determine the similarity of the condition parameters.

3. The confidence assessment method for reuse models according to claim 1, characterized in that, The step of performing conditional parameter sensitivity analysis on the reuse model and determining the sensitivity weights includes: Based on the preset range of factor values, pure function factors and impure function factors are selected from the condition parameters; Sensitivity analysis and variance analysis are performed on the pure function factors and the impure function factors respectively to determine the results of the sensitivity analysis and variance analysis. The results of the sensitivity analysis are used to represent the significance of the influence of the pure function factors on the model output results, and the results of the variance analysis are used to determine the significance of the influence of the impure function factors on the model output results. The sensitivity analysis results and the variance analysis results are subjected to sensitivity normalization processing to determine the sensitivity weights of the conditional parameters.

4. The confidence assessment method for reuse models according to claim 3, characterized in that, The steps of performing sensitivity analysis and variance analysis on the pure function factors and the impure function factors respectively, to determine the sensitivity analysis results and variance analysis results, include: Sensitivity analysis is performed on the pure function factors to determine the sensitivity analysis results, and the factor with the highest sensitivity among the pure function factors is determined as the reference factor. Based on the impure function factors and the reference factors, an orthogonal experiment of the impure function factors is set up, and the simulation results of the orthogonal experiment are subjected to variance analysis to determine the variance analysis results.

5. The confidence assessment method for reuse models according to claim 1, characterized in that, The step of determining the model similarity between the reused model and each model in the instance library based on the sensitivity weight and the similarity matrix using a preset similarity calculation model includes: The model similarity is determined by multiplying the sensitivity weight by the transpose of the similarity matrix.

6. The confidence assessment method for reuse models according to claim 1, characterized in that, Before the step of using the error expectation calculation model to determine the expected error between the simulated value of the reused model and the experimental value of the actual model based on the model similarity, the following steps are included: The test model is obtained from the instance library, and based on the preset relative error calculation model, the expected value of the error between the simulated value and the experimental value of the test model, the relative error and the average value of the relative error are determined. By performing global optimization on the test model, the regularization coefficient with the smallest average relative error is determined as the target regularization coefficient.

7. The confidence assessment method for reuse models according to claim 1, characterized in that, The steps of determining the expected error between the simulated value of the reused model and the experimental value of the actual model based on the model similarity using an error expectation calculation model include: Substituting the model similarity, the simulation value of the reused model, and the experimental value of the actual model into the error expectation calculation model, the expected error value is determined, wherein the error expectation calculation model is expressed as: in, For loss function, For instance library The predicted values ​​of a simulation model, For the experimental values ​​corresponding to the actual model, The regularization term characterizes the complexity of the model. The coefficient of the objective regularization term is used to balance model error and model complexity.

8. A confidence assessment device for a reused model, characterized in that, The device is used in a confidence evaluation system for reused models, and the device includes: The similarity matrix calculation module acquires a reused model and an instance library, and determines the conditional parameter similarity by comparing the reused model and the instance library, so as to construct a similarity matrix using the conditional parameter similarity. The sensitivity weight calculation module performs conditional parameter sensitivity analysis on the reuse model to determine the sensitivity weights. The confidence assessment module uses a preset similarity calculation model to determine the model similarity between the reused model and each model in the instance library based on the sensitivity weight and the similarity matrix. It also uses an error expectation calculation model to determine the expected error between the simulation value of the reused model and the experimental value of the actual model based on the model similarity. The confidence of the reused model and the confidence assessment result are determined based on the expected error value.

9. A server, characterized in that, The method includes a processor and a memory, the memory storing computer-executable instructions executable by the processor, the processor executing the computer-executable instructions to implement the method of any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions that, when invoked and executed by a processor, cause the processor to perform the method according to any one of claims 1 to 7.

Images