Simulation system credibility evaluation method and system based on data quality

Abstract

The invention discloses a simulation system credibility evaluation method and system based on data quality, and the method comprises the steps: obtaining and analyzing the original information of a simulation experiment, so as to obtain the influence factors of the simulation experiment, the original information comprises a simulation target document, a simulation scene configuration document, a digital model document and a simulation subsystem document, and the influence factors comprise credibility influence factors and simulation result influence factors; constructing a confidence evaluation index system of the simulation experiment according to the credibility influence factors, and setting the index weight of each layer of index; generating multiple groups of input data of a simulation experiment according to the simulation result influence factors, detecting the input data according to a preset data quality detection standard, and generating a detection result; and performing a simulation experiment by using the detected input data, obtaining evaluation data, evaluating the evaluation data by using the confidence evaluation index system, and generating a confidence evaluation result of the simulation experiment.

Description

Technical Field

[0001] The present invention relates to the field of display technology, and in particular to a method and system for evaluating the credibility of a simulation system based on data quality. Background Art

[0002] Simulation technology is an effective means to solve the problem that real system experiments are too difficult or too costly. With the continuous development of simulation technology and the continuous improvement of the value of simulation applications, the application fields of simulation are becoming more and more extensive, and the complexity of simulation systems is gradually increasing, which leads to higher and higher requirements for simulation correctness and credibility. For complex simulation systems, if credibility assessment is not performed, users cannot determine the extent to which their simulation execution results meet the simulation purpose and whether they are usable for the intended engineering application. If incorrect simulation results are not discovered in time, they will lead to major decision-making errors or actual application accidents. Evaluating the correctness and credibility of complex simulation systems is an important task in the theoretical development of simulation technology and the construction of simulation system applications.

[0003] At present, the theories, methods, technologies, specifications and tools of simulation credibility need to develop synchronously with the development of simulation technology and simulation applications. However, due to the rapid development of simulation application related technologies, the credibility assessment of complex simulation systems involves high-dimensional spaces with multiple scenarios, multiple objects, multiple models, multiple levels, multiple goals and multiple stages, which makes the execution of simulations have uncertainty, fuzziness and complexity, thus increasing the difficulty of credibility assessment. In addition, with the development of technologies such as digital twins, simulation systems need to manage a large number of digital models of simulation objects, and simulation execution will also generate a large amount of process and result data. These data closely related to simulation are the evaluation objects that cannot be ignored in credibility assessment. At present, the methods for simulation credibility assessment are mainly based on the verification, validation and accreditation (VV&A) technology of modeling and simulation (M&S), forming an evaluation method throughout the entire life cycle of the simulation system. However, in practice, the credibility assessment of complex simulation systems faces many problems and challenges. First, the existing credibility assessment methods lack consideration of the complexity of simulation scenarios. A large number of simulation influencing factors require a large amount of sampling to form a comprehensive set of simulation results. Second, the simulation system directly performs simulation based on the connected digital model. The verification of simulation results may face the lack of sufficient prototype system data support. Third, in the case of missing reference data for simulation objects, expert scoring will improve the subjective and objective synthesis of various types of credibility assessments. Finally, there is a large amount of data content in the simulation system. If data quality control is not performed, the accuracy of the credibility assessment results will be seriously affected.

[0004] In summary, for the current complex simulation systems, how to improve the credibility evaluation results of simulation experiments has become a technical problem that needs to be urgently solved by technical personnel in this field. Summary of the invention

[0005] In order to solve at least one of the above problems, a first embodiment of the present invention provides a simulation system credibility assessment method based on data quality, comprising: Collecting original information of the simulation experiment in the simulation system, analyzing the original information and obtaining influencing factors of the simulation experiment, the original information includes the simulation target document, simulation scenario configuration document, digital model document and simulation subsystem document of the simulation experiment, and the influencing factors include credibility influencing factors and simulation result influencing factors; Constructing a confidence evaluation index system of the simulation experiment according to the credibility influencing factors, and setting the index weights of each layer of the confidence evaluation index system; Generating multiple groups of input data for the simulation experiment according to the factors affecting the simulation results, and testing each group of input data according to a preset data quality testing standard and generating a testing result; The simulation experiment is performed using input data whose detection results meet the detection threshold and evaluation data is obtained. The evaluation data is evaluated using the confidence evaluation index system and a confidence evaluation result of the simulation experiment is generated.

[0006] For example, in the simulation system credibility assessment method provided in some embodiments of the present application, The collecting original information of the simulation experiment in the simulation system, analyzing the original information and obtaining the influencing factors of the simulation experiment further includes: Analyze the original information and obtain relevant data of the simulation experiment, wherein the relevant data includes simulation system data, scene environment data, object individual performance data, inter-object collaboration data and simulation result data; The influencing factors are extracted according to the relevant data, wherein the credibility influencing factors include simulation system credibility factors, data and model credibility factors, simulation process credibility factors and simulation result credibility factors, and the simulation result influencing factors include model input variables, scenario parameters and simulation system parameters.

[0007] For example, in the simulation system credibility assessment method provided in some embodiments of the present application, the confidence assessment index system of the simulation experiment is constructed according to the credibility influencing factors, and the index weights of the indicators of each layer of the confidence assessment index system are set, which further includes: The simulation system credibility factor, data and model credibility factor, simulation process credibility factor and simulation result credibility factor are used as first-level evaluation indicators; Decomposing the first-level evaluation indicators step by step according to the experimental scenario of the simulation experiment to form a hierarchical confidence evaluation indicator system; The indicator weights of the indicators at each level in the confidence evaluation indicator system are set.

[0008] For example, in the simulation system credibility assessment method provided in some embodiments of the present application, The step of decomposing the first-level evaluation indicators step by step according to the experimental scenario of the simulation experiment and forming a hierarchical confidence evaluation indicator system further includes: The simulation system credibility factor includes high-level architecture credibility, and the high-level architecture credibility includes attribute update delay performance, interaction instance sending performance and interaction instance receiving performance; The model credibility factors include simulation model credibility, simulation data credibility and simulation algorithm credibility. The simulation model credibility includes conceptual model credibility, mathematical model credibility and software model credibility. The simulation data credibility includes solution accuracy, solution speed and calculation method. The simulation algorithm credibility includes data source quality, metadata accuracy, experimental data accuracy and data integrity. The credibility factors of the simulation process include the credibility of the simulation scheme design, the credibility of the simulation boundary condition setting, the credibility of the simulation decision rule, the credibility of the simulation implementation process, the control performance of the simulation system and the data recording performance of the simulation system; The credibility factors of the simulation results include the credibility of the knowledge base, the credibility of the analysis rule base, the credibility of the reasoning mechanism, the credibility of the visualization unit, the performance of the exercise playback, the performance of the analysis statistics and the performance of the data quality management; The step of setting the indicator weights of each layer of indicators in the confidence evaluation indicator system further includes: The simulation credibility score of the simulation experiment is: C=λ A C A +λ M C M +λ P C P +λ R C R , where C is the simulation credibility score, C A is the credibility factor of the simulation system, λ A is the weight of the credibility factor of the simulation system, C M is the model credibility factor, λ M is the weight of the model credibility factor, C P is the credibility factor of the simulation process, λP is the weight of the credibility factor of the simulation process, C R is the credibility factor of the simulation results, λ R is the weight of the credibility factor of the simulation results; The credibility factors of the simulation system are: , where C A1 is the high-level architecture trustworthiness, λ A1 is the weight of the high-level architecture trustworthiness, , C A1 The i-th third-level indicator, is the weight of the corresponding i-th third-level indicator; The model credibility factors are: , where C Mi C M The i-th secondary index of is the weight of the corresponding secondary indicator, C M1i C M1 The i-th third-level indicator, is the weight of the corresponding i-th third-level indicator, C M2i C M2 The i-th third-level indicator, is the weight of the corresponding i-th third-level indicator, C M3i C M3 The i-th third-level indicator, is the weight of the corresponding i-th third-level indicator; The credibility factors of the simulation process are: , where C Pi C P The i-th secondary index of is the weight of the corresponding i-th secondary indicator; The credibility factors of simulation results are: , where C Ri C R The i-th secondary index of is the weight of the corresponding i-th secondary indicator.

[0009] For example, in the simulation system credibility assessment method provided in some embodiments of the present application, the generating of multiple groups of input data of the simulation experiment according to the simulation result influencing factors, and testing each group of input data according to the preset data quality testing standard and generating the test results further includes: Determine the model input variables, scenario parameters and simulation system parameters involved in the simulation experiment according to the simulation result influencing factors, and obtain the value ranges of the model input variables, scenario parameters and simulation system parameters respectively; Generating multiple groups of input data for the simulation experiment according to the value ranges of the model input variables, scenario parameters and simulation system parameters; Determining data content of data quality to be tested in the multiple groups of input data, as well as data quality testing dimensions and data quality problem categories; According to the preset data quality detection standard, each group of input data is tested and a detection result is generated to determine whether the detection result meets the detection threshold. If it does not meet the detection threshold, jump to generating multiple groups of input data for the simulation experiment according to the value range of the model input variables, scenario parameters and simulation system parameters.

[0010] For example, in the simulation system credibility assessment method provided in some embodiments of the present application, the step of testing each group of input data according to a preset data quality testing standard and generating a testing result further includes: Determine the data quality problem determination standard, detection method and detection process based on the data quality detection dimension and data quality problem category to form the data quality detection standard; The data content of the input data is detected according to the data quality detection standard and a detection result is generated.

[0011] For example, in the simulation system credibility assessment method provided in some embodiments of the present application, the use of input data whose detection results meet the detection threshold to perform the simulation experiment and obtain evaluation data, and the use of the confidence assessment index system to evaluate the evaluation data and generate the confidence assessment result of the simulation experiment further includes: The evaluation data is aggregated and calculated step by step according to the confidence evaluation index system to obtain the credibility score of the simulation experiment as the confidence evaluation result.

[0012] The second embodiment of the present invention provides a simulation system credibility assessment system based on data quality, including an information collection and analysis module, a confidence assessment index system construction module, an input data generation module, and a simulation experiment detection and confidence assessment module, wherein: The information collection and analysis module is used to collect the original information of the simulation experiment in the simulation system, analyze the original information and obtain the influencing factors of the simulation experiment, the original information includes the simulation target document, simulation scenario configuration document, digital model document and simulation subsystem document of the simulation experiment, and the influencing factors include credibility influencing factors and simulation result influencing factors; The confidence evaluation index system construction module is used to construct the confidence evaluation index system of the simulation experiment according to the credibility influencing factors, and set the index weights of each layer of the confidence evaluation index system; The input data generation module is used to generate multiple groups of input data for the simulation experiment according to the simulation result influencing factors, and to detect each group of input data according to a preset data quality detection standard and generate a detection result; The simulation experiment detection and confidence assessment module is used to use the input data whose detection results meet the detection threshold to perform the simulation experiment and obtain assessment data, use the confidence assessment indicator system to evaluate the assessment data and generate the confidence assessment result of the simulation experiment.

[0013] A third embodiment of the present invention provides a computer-readable storage medium having a computer program stored thereon. When the program is executed by a processor, the method described in the first embodiment is implemented.

[0014] A fourth embodiment of the present invention provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method described in the first embodiment when executing the program.

[0015] The beneficial effects of the present invention are as follows: In view of the existing problems, the present invention develops a method and system for evaluating the credibility of a simulation system based on data quality, obtains influencing factors by collecting and analyzing the original information of the simulation experiment, establishes a confidence evaluation index system according to the credibility influencing factors, generates input data for the simulation experiment according to the simulation result influencing factors and performs data quality detection, uses data that meets the data quality requirements to perform simulation experiments and obtain evaluation data, uses the established confidence evaluation index system to evaluate the evaluation data and obtain confidence evaluation results, effectively improves the accuracy of credibility evaluation of complex simulation systems, thereby making up for the problems existing in the prior art and having broad application prospects. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0017] Figure 1 A flowchart showing a method for evaluating the credibility of a simulation system according to an embodiment of the present invention is shown; Figure 2A schematic diagram showing the structure of a simulation system credibility evaluation system according to an embodiment of the present invention is shown; Figure 3 A structural block diagram showing the original information analysis according to an embodiment of the present invention is shown; Figure 4 A schematic structural diagram of a computer device according to another embodiment of the present invention is shown. DETAILED DESCRIPTION

[0018] In order to more clearly illustrate the present invention, the present invention is further described below in conjunction with preferred embodiments and accompanying drawings. Similar components in the accompanying drawings are represented by the same reference numerals. It should be understood by those skilled in the art that the content specifically described below is illustrative rather than restrictive, and should not be used to limit the scope of protection of the present invention.

[0019] Among the related technologies, the credibility assessment method of simulation system mainly adopts the assessment technology based on the VV&A framework. The related research work includes the credibility assessment method of simulation conceptual model of complex large system, the simulation credibility control process, etc. The existing schemes focus more on solving the problems of evaluation index system construction, subjectivity and uncertainty of evaluation quantification, etc. For the credibility assessment tasks that are gradually based on massive simulation-related data, there is a lack of evaluation result aggregation mechanism for data quality control.

[0020] In view of the above situation, if Figure 1 As shown, an embodiment of the present invention provides a simulation system credibility assessment method based on data quality, comprising: Collecting original information of the simulation experiment in the simulation system, analyzing the original information and obtaining influencing factors of the simulation experiment, the original information includes the simulation target document, simulation scenario configuration document, digital model document and simulation subsystem document of the simulation experiment, and the influencing factors include credibility influencing factors and simulation result influencing factors; Constructing a confidence evaluation index system of the simulation experiment according to the credibility influencing factors, and setting the index weights of each layer of the confidence evaluation index system; Generating multiple groups of input data for the simulation experiment according to the factors affecting the simulation results, and testing each group of input data according to a preset data quality testing standard and generating a testing result; The simulation experiment is performed using input data whose detection results meet the detection threshold and evaluation data is obtained. The evaluation data is evaluated using the confidence evaluation index system and a confidence evaluation result of the simulation experiment is generated.

[0021] In this embodiment, in order to improve the credibility evaluation results of complex simulation systems, by fully collecting and organizing relevant data involved in the evaluation indicators, and after quality control of the input data, combined with the constructed confidence evaluation indicator system, the heterogeneous credibility evaluation data is reasonably aggregated, and finally the credibility evaluation results are obtained. Specifically, this embodiment collects the original information of the simulation experiment in the simulation system, wherein the original information is the original information of the current simulation experiment, for example, the simulation experiment for the first project needs to collect the original information of the first project, and the simulation experiment for the second project needs to collect the original information of the second project, and then analyzes the original information and obtains the influencing factors of the simulation experiment, the influencing factors include credibility influencing factors and simulation result influencing factors, establishes a confidence evaluation indicator system according to the credibility influencing factors, generates the input data of the simulation experiment according to the simulation result influencing factors and performs data quality detection, uses the data that meets the data quality requirements to perform the simulation experiment and obtain the evaluation data, and uses the established confidence evaluation indicator system to evaluate the evaluation data and obtain the confidence evaluation results, thereby effectively improving the accuracy of the credibility evaluation of the complex simulation system.

[0022] To further illustrate the specific implementation of this embodiment, the reliability evaluation of the simulation system is taken as an example. Figure 2 As shown, a simulation system credibility assessment system based on data quality includes an information acquisition and analysis module, a confidence assessment index system construction module, an input data generation module, and a simulation experiment detection and confidence assessment module, wherein the information acquisition and analysis module is used to acquire the original information of the simulation experiment in the simulation system, analyze the original information and obtain the influencing factors of the simulation experiment, the original information includes the simulation target document, the simulation scenario configuration document, the digital model document and the simulation subsystem document of the simulation experiment, the influencing factors include the credibility influencing factors and the simulation result influencing factors; the confidence assessment index system construction module is used to collect ... according to the confidence assessment index system construction module, and obtain the influencing factors of the simulation experiment according to the confidence assessment index system construction module. The credibility influencing factors construct the confidence evaluation index system of the simulation experiment, and set the index weights of the indicators at each level of the confidence evaluation index system; the input data generation module is used to generate multiple groups of input data for the simulation experiment according to the simulation result influencing factors, and detect each group of input data according to the preset data quality detection standards and generate detection results; the simulation experiment detection and confidence evaluation module is used to use the input data whose detection results meet the detection threshold to conduct the simulation experiment and obtain evaluation data, and use the confidence evaluation index system to evaluate the evaluation data and generate the confidence evaluation results of the simulation experiment. Specifically including the following steps: The first step is to collect original information of the simulation experiment in the simulation system, analyze the original information and obtain the influencing factors of the simulation experiment.

[0023] In this embodiment, the simulation target document, simulation scenario configuration document, digital model document in the simulation system, and each simulation subsystem document in the simulation system of the simulation experiment currently running are first used as original information. That is, compared with the related art, this embodiment adds considerations about the complexity of the simulation scenario and the digital model to the original information, increases the data source and data type of the original information, and analyzes based on the expanded original information to form the influencing factors affecting the simulation results of the simulation experiment.

[0024] Specifically, Figure 3 As shown, further comprising: First, the original information is analyzed and relevant data of the simulation experiment is obtained, wherein the relevant data includes simulation system data, scene environment data, object individual performance data, inter-object collaboration data and simulation result data.

[0025] In this embodiment, relevant data is obtained by analyzing original information, and is divided into the above five categories according to the function and application purpose of the relevant data.

[0026] Secondly, the influencing factors are extracted based on the relevant data. The credibility influencing factors include simulation system credibility factors, data and model credibility factors, simulation process credibility factors and simulation result credibility factors. The simulation result influencing factors include model input variables, scenario parameters and simulation system parameters.

[0027] In this embodiment, based on the divided five types of data, data items that affect the simulation results and credibility are further extracted from the relevant data and further divided into credibility influencing factors and simulation result influencing factors that affect the output results of the simulation system.

[0028] The second step is to construct a confidence evaluation index system for the simulation experiment according to the credibility influencing factors, and set the index weights of the indicators at each level of the confidence evaluation index system.

[0029] In this embodiment, the confidence is divided into the architecture and resource management related to the simulation system itself, and the model and algorithm data related to the experimental operation, and a first-level evaluation index for the confidence of the complex simulation system is established. Based on the key characteristics of the first-level evaluation index and the specific simulation experiment scenario, it is further decomposed step by step until the indicator item is an indivisible indicator item with a single characteristic, and finally a hierarchical confidence evaluation indicator system is constructed.

[0030] Specifically, it further includes: Firstly, the simulation system credibility factor, data and model credibility factor, simulation process credibility factor and simulation result credibility factor are taken as primary evaluation indicators.

[0031] In this embodiment, four types of indicators are formed based on the architecture and resource management related to the simulation system itself, as well as the model and algorithm data related to the experimental operation, namely, the reliability factor of the simulation system, C A , Data and Model Credibility Factor C M , Credibility Factor of Simulation Process C P and the reliability factor C of the simulation results R .

[0032] Secondly, according to the experimental scenario of the simulation experiment, the first-level evaluation indicators are decomposed level by level to form a hierarchical confidence evaluation indicator system.

[0033] In this embodiment, each first-level credibility evaluation index is decomposed to generate at least one second-level evaluation index, and the second-level evaluation index is then decomposed until it cannot be further decomposed, thereby forming a multi-level confidence evaluation index system.

[0034] Specifically, the reliability factor C of the simulation system A Decompose, including at least one secondary evaluation indicator: High Level Architecture (HLA) based high-level architecture credibility C A1 ; Further decompose the secondary evaluation indicators, high-level architecture credibility C A1 Decomposed into at least three third-level evaluation indicators, such as attribute update delay performance C A11 , Interaction instance sending performance C A12 and the interaction instance receives performance C A13 ; Attribute update delay performance C A11 , Interaction instance sending performance C A12 and the interaction instance receives performance C A13 It is an indivisible indicator item with a single characteristic and cannot be divided further, so the credibility factor of the simulation system forms a three-level evaluation indicator.

[0035] Similarly, for the model credibility factor C M Decompose it into at least three secondary evaluation indicators: including the credibility of the simulation model C M1 , Simulation data credibility C M2 and the credibility of the simulation algorithm C M3 ; Further decompose the secondary evaluation indicators, the simulation model credibility C M1 Includes at least three third-level evaluation indicators: conceptual model credibility C M11 , Credibility of mathematical model C M12 and software model credibility C M13 , the reliability of the simulation data C M2 Includes at least three third-level evaluation indicators: solution accuracy CM21 , solve the speed C M22 and calculation method C M23 , the credibility of the simulation algorithm C M3 Includes at least four third-level evaluation indicators: Data source quality C M31 , metadata accuracy C M32 、Accuracy of experimental data C M33 and data integrity C M34 ; Each of the three-level evaluation indicators is an indivisible indicator item with a single characteristic and cannot be divided further, so the model credibility factor forms a three-level evaluation indicator.

[0036] Similarly, for the reliability factor C of the simulation process P Decompose it into at least six secondary evaluation indicators: including the credibility of the simulation scheme design C P1 , Simulation boundary condition setting credibility C P2 , Credibility of simulation decision rules C P3 , Credibility of simulation implementation process C P4 , Simulation system control performance C P5 and simulation system data recording performance C P6 ; Each secondary evaluation index is an indivisible index item with a single characteristic and cannot be divided further, so the credibility factor of the simulation process forms a secondary evaluation index.

[0037] Similarly, the reliability factor C of the simulation results R Decompose it into at least seven secondary evaluation indicators: including the credibility of the knowledge base C R1 , analyze the credibility of the rule base C R2 , credibility of reasoning mechanism C R3 , Visualization Unit Credibility C R4 , Practice playback performance C R5 , Analyze statistical performance C R6 and data quality management performance C R7 ; Each secondary evaluation index is an indivisible index item with a single characteristic and cannot be divided further, so the credibility factor of the simulation result forms the secondary evaluation index.

[0038] Finally, the indicator weights of the indicators at each level in the confidence evaluation indicator system are set.

[0039] In this embodiment, the weights of the evaluation indicators are set using the hierarchical analysis method, which specifically includes: First, weights are set for the first-level evaluation indicators. Specifically, the simulation credibility score of the simulation experiment is: , where C is the simulation credibility score, C A is the credibility factor of the simulation system, λ Ais the weight of the credibility factor of the simulation system, C M is the model credibility factor, λ M is the weight of the model credibility factor, C P is the credibility factor of the simulation process, λ P is the weight of the credibility factor of the simulation process, C R is the credibility factor of the simulation results, λ R is the weight of the credibility factor of the simulation results.

[0040] Second, set weights for multi-level evaluation indicators.

[0041] Specifically, the reliability factor C of the simulation system A Includes a secondary evaluation indicator, namely, high-level architecture credibility C A1 , then set the weight of the secondary evaluation index, , where C A1 is the high-level architecture trustworthiness, λ A1 is the weight of the high-level architecture reliability. By analogy, the high-level architecture reliability C A1 Including three three-level evaluation indicators, the attribute update delay performance C is set respectively A11 , Interaction instance sending performance C A12 and the interaction instance receives performance C A13 The weight of , C A1 The i-th third-level indicator, is the weight of the corresponding third-level indicator. Similarly, the model credibility factor is: , where C Mi C M The i-th secondary index of is the weight of the corresponding secondary indicator, C M1i C M1 The i-th third-level indicator, is the weight of the corresponding i-th third-level indicator, C M2i C M2 The i-th third-level indicator, is the weight of the corresponding i-th third-level indicator, C M3i C M3 The i-th third-level indicator, is the weight of the corresponding third-level indicator. Similarly, the credibility factor of the simulation process is: , where C Pi C P The i-th secondary index of is the weight of the corresponding i-th secondary index. Similarly, the credibility factor of the simulation result is: , where C Ri C R The i-th secondary index of is the weight of the corresponding i-th secondary indicator.

[0042] In this embodiment, the analytic hierarchy process is used to set weights for the evaluation indicators. The indicator weights of each level of the credibility evaluation indicator system are expressed in numerical values, and the numerical values ​​are determined by multi-level evaluation results. For example, the evaluation results are determined by scores given by multiple experts, and the weights of the multi-level credibility evaluation indicator system are determined based on this.

[0043] For example, in a specific example, the first-level indicators are: simulation system credibility factor C A is 0.1, the reliability factor C of the simulation process P is 0.2, the data and model credibility factor C M is 0.3, the reliability factor C of the simulation results R is 0.4. Simulation system credibility factor C A Secondary indicator: High-level architecture credibility C A1 is 0.1; the credibility factor of the simulation process C P Secondary indicator: Credibility of simulation scheme design C P1 Set the credibility C to 0.02 and the simulation boundary conditions P2 is 0.02, the credibility of the simulation decision rule C P3 is 0.02, the reliability of the simulation implementation process C P4 is 0.02, simulation system control performance C P5 is 0.06, and the simulation system data recording performance C P6 is 0.06; model credibility factor C M Secondary evaluation index: Simulation model credibility C M1 is 0.09, the reliability of simulation data C M2 is 0.12, and the reliability of the simulation algorithm C M3 is 0.09; the reliability factor of the simulation result C R Secondary indicator: Knowledge base credibility C R1 is 0.04, the credibility of the analysis rule base C R2 is 0.04, the credibility of the reasoning mechanism C R3 is 0.04, the visualization unit credibility C R4 0.04, drill playback performance C R5 0.08, analysis of statistical performance C R6 is 0.08, and data quality management performance C R7is 0.08. High-level architecture credibility C A1 The third-level indicator of 0.1: attribute update delay performance C A11 The performance of the interactive instance is 0.03 and C A12 is 0.04, and the interaction instance receives performance C A13 is 0.03; the credibility of the simulation model C M1 The third level indicator: conceptual model credibility C M11 The mathematical model credibility is 0.027. M12 is 0.036, and the software model credibility C M13 is 0.027; the reliability of simulation data C M2 The third level indicator: solution accuracy C M21 = 0.036, solve for speed C M2 2 is 0.048, and calculation method C M23 is 0.036; the credibility of the simulation algorithm C M3 The third level indicator: Data source quality C M31 The metadata accuracy is 0.009. M32 The experimental data accuracy is 0.018. M33 is 0.027, and data integrity C M34 It is 0.036.

[0044] The third step is to generate multiple groups of input data for the simulation experiment according to the factors affecting the simulation results, and to detect each group of input data according to a preset data quality detection standard and generate a detection result.

[0045] In this embodiment, the simulation result influencing factors are to generate multiple sets of input data required for the operation of the simulation system according to the evaluation results and the value sampling method, wherein the input data includes model input variables, scenario parameters and simulation system parameters. Specifically, the following steps are included: First, the model input variables, scenario parameters and simulation system parameters involved in the simulation experiment are determined according to the factors affecting the simulation results, and the value ranges of the model input variables, scenario parameters and simulation system parameters are respectively obtained.

[0046] In this embodiment, it is assumed that z simulation result influencing factors X are obtained in step 1. 1 , X 2 ,……,X z , including the model input variables, scenario parameters and simulation system parameters required for simulation operation, and determining the influencing factors X for each simulation result i The value range of the subjective influencing factors is set through multiple expert evaluations, and the remaining influencing factors are generated using the Latin hypercube sampling method.

[0047] Second, multiple groups of input data for the simulation experiment are generated according to the value ranges of the model input variables, scenario parameters and simulation system parameters.

[0048] In this embodiment, each simulation result influencing factor X i The value range of each simulation result influencing factor is divided into k subspaces, a sample point is randomly selected in each subspace, and the sample points of each simulation result influencing factor are combined to form a complete sample. Repeat the process until the model input variables and operation parameter sets X={X1, X2, …, X3} required for k sets of simulation runs are generated. k}, where X i =(X 1 i , X 2 i , ..., X z i ).

[0049] In a specific example, there are three simulation influencing factors: temperature (X 1 ), humidity (X 2 ) and pressure (X 3 ). First determine the value range, temperature (X 1 ): 20°C to 30°C, humidity (X 2 ): 40% to 60%, pressure (X 3 ): 1 bar to 2 bar. Then divide the subspace and divide the value range of each variable into 3 subspaces (k=3): then temperature (X 1 ) are: [20, 23.33], [23.33, 26.67], [26.67, 30]; then humidity (X 2 ) are: [40, 46.67], [46.67, 53.33], [53.33, 60]; then the pressure (X 3 ) are: [1, 1.33], [1.33, 1.67], [1.67, 2]. Again, Latin hypercube sampling is used to randomly select a sample point in each subspace: temperature (X 1 ) are: 21°C, 24°C, 28°C, humidity (X 2 ) are: 43%, 50%, 57%, pressure (X 3) are: 1.1 bar, 1.5 bar, 1.8 bar. Next is the combined sample points, combining the sample points of each variable to form a complete sample: Sample 1 is: (21°C, 43%, 1.1 bar), Sample 2 is: (24°C, 50%, 1.5 bar), Sample 3 is: (28°C, 57%, 1.8 bar). Finally, the input data is generated as the sample set X={X1,X2,X3}: X1=(21,24,28), X2=(43,50,57), X3=(1.1,1.5,1.8). This example uses the Latin hypercube sampling method to generate three sets of sample sets of factors affecting the simulation results. Each sample set contains the values ​​of three simulation factors and can be used for simulation runs.

[0050] Third, determine the data content of the data quality to be tested in the multiple groups of input data, as well as the data quality testing dimensions and data quality problem categories.

[0051] In this embodiment, the data content whose data quality needs to be tested in the basic data of the simulation experiment includes the model input variables, scenario parameters and simulation system parameters required for the simulation operation. The data quality dimensions to be tested include data integrity, correctness, accuracy, consistency, redundancy and timeliness. The data quality problem categories include three categories: erroneous data, pathological data and abnormal data.

[0052] Fourth, each group of input data is tested according to the preset data quality detection standard and a test result is generated to determine whether the test result meets the detection threshold. If it does not meet the detection threshold, jump to generating multiple groups of input data for the simulation experiment based on the value range of the model input variables, scenario parameters and simulation system parameters.

[0053] In this embodiment, the data quality detection standard is determined based on the above data quality detection dimension and data quality problem category, and detection is performed according to the data quality detection standard. If the detection result meets the detection threshold, the subsequent operation is performed. If the detection result does not meet the detection threshold, the above operation is repeated until the input data that meets the detection threshold is generated. Specifically, the following steps are included: First, based on the data quality detection dimensions and data quality problem categories, data quality problem judgment standards, detection methods and detection processes are determined to form the data quality detection standards.

[0054] Secondly, the data content of the input data is detected according to the data quality detection standard and a detection result is generated.

[0055] In this embodiment, data quality detection standards are defined for the input data of the simulation experiment, such as defining the expected level and requirements of each dimension of data quality; defining a rule set for each data quality dimension on the influencing factors, including a data integrity rule set Q I , correctness rule set Q R , accuracy rule set Q P , consistency rule set Q C , redundancy rule set Q D , timeliness rule set Q T The rule set is a specific implementation of the data quality detection standard, which is used to evaluate and detect whether the data meets the data quality detection standard. The detection method adopts a rule-based detection method, that is, to detect data quality problems according to the defined rule set. The detection process includes, for example, the user setting quality detection rules, setting data quality problem definitions based on the quality detection rules, performing data quality detection, and determining data quality scores. Set scoring weights for each type of data quality problem, such as the integrity weight W QI , correctness weight W QR , accuracy weight W QP , consistency weight W QC , redundancy weight W QD , timeliness weight W QT .

[0056] In this embodiment, the data quality detection process of simulation input data is performed on the sample set X of model input variables, scenario parameters and simulation system parameters required for simulation operation; first, data quality detection is performed based on the rule set, and the rule set includes the data integrity rule set Q I , correctness rule set Q R , accuracy rule set Q P , consistency rule set Q C , redundancy rule set Q D , timeliness rule set Q T , calculate the ratio of various data quality problem data, integrity problem ratio P I , correctness question ratio P R , accuracy problem ratio P P , consistency problem ratio P C , redundancy problem ratio P D , the timeliness problem ratio P T , the data quality related credibility index score S is calculated based on the data quality problem ratio and weight, that is, ,When the data quality related credibility indicator score S meets the set scoring threshold, the data is considered credible, and the next step is executed. If the scoring threshold is not met, the data is regenerated.

[0057] In a specific example, an e-commerce platform needs to perform quality checks on user transaction data to ensure the integrity, correctness, accuracy, consistency, redundancy, and timeliness of the data.

[0058] Data quality testing standards include: integrity rule set Q I :Ensure that all transaction records contain basic information such as user ID, transaction time, transaction amount, etc. Correctness rule set Q C : Ensure that the transaction amount is positive and the transaction time format is correct. Accuracy rule set Q A :Ensure that the transaction amount is consistent with the actual payment amount. Consistency rule set Q U :Ensure that the transaction records of the same user in different databases are consistent. Redundancy rule set Q R :Ensure that there are no duplicate transaction records in the database. Timeliness rule set Q T :Ensure that the transaction record is up to date and no older than 24 hours. Weight settings include: Completeness weight W I =0.2, correctness weight W C = 0.2, accuracy weight W A = 0.2, consistency weight W U = 0.15, redundancy weight W R =0.15, timeliness weight W T =0.1.

[0059] The data quality testing process includes: First, data collection: extract the transaction records of the last 24 hours from the database. Second, rule testing: integrity problem ratio P I =0.05 (5% of the records lack basic information), the correctness problem ratio P C =0.02 (2% of recorded transaction amounts are negative), accuracy problem ratio P A =0.03 (3% of the recorded amounts are inconsistent), consistency problem ratio P U =0.01 (1% of the records are inconsistent), redundancy problem ratio P R =0.04 (4% of records are repeated), the timeliness problem ratio P T =0.06 (6% of the records are older than 24 hours), calculate the credibility index score S:

[0060] Regarding the evaluation results: If S≥0.9, the data is considered credible and the next step can be performed. If S<0.9, the data does not meet the credibility requirements and needs to be regenerated. In this example, the credibility index score S=0.9665 meets the set score threshold, so the input data is considered credible and subsequent data analysis or processing can continue. If the score does not meet the requirements, the data needs to be cleaned or regenerated to improve data quality.

[0061] The fourth step is to use the input data whose detection results meet the detection threshold to conduct the simulation experiment and obtain evaluation data, and use the confidence evaluation index system to evaluate the evaluation data and generate a confidence evaluation result of the simulation experiment.

[0062] In this embodiment, based on each set of simulation input data X i Run the simulation system test t times and collect the simulation result output data set R i ={R i1 , R i2 , ..., R ik}, where R ij For the input data X i The j-th simulation system experiment running result data set is obtained. The evaluation data is aggregated step by step according to the confidence evaluation index system to obtain the credibility score of the simulation experiment as the confidence evaluation result. Specifically, it includes: statistical processing of the data set, calculating the mean, variance and other statistics of each data, and expressing each simulation data result in the form of confidence interval. In this embodiment, an expert evaluation scoring mechanism is adopted for the subjective evaluation index in the simulation confidence, v experts are selected, and a weight W is set for each expert. E1 , W E2 ,…,W Ev , where the sum of the expert vector weights is 1. Assuming there are 5 experts, for example, the weight vector is (0.15, 0.2, 0.25, 0.3, 0.1). The expert score is for each credibility evaluation indicator C xy or C xyz Give the score, where C xy is an inseparable secondary evaluation indicator, C xyz It is an indivisible three-level evaluation indicator, which can be scored on a 10-point scale but is not limited to it. Each group of simulation input data X is calculated by weighted average. i The result dataset R after running the simulation i Index C xy Expert credibility assessment score , where C Ej xy is the index C of the jth expert's test on the i-th group xyScoring; Evaluation index C xy The score is In this example, the scores of various evaluation indicators are based on the confidence evaluation indicator system of the simulation experiment constructed in the second step, and the credibility scores are calculated step by step to obtain the final simulation system credibility score C.

[0063] In a specific example, suppose that a set of simulation input data X is subjected to 5 simulation system tests, and the resulting data set R is as follows: R={R1}. R1={230,240,235,250,245}, first calculate the mean: Mean(R1)=240; secondly calculate the variance: Var(R1)=50; again obtain the confidence interval, the calculation of the confidence interval usually requires the use of the sample mean, sample standard deviation and sample size, as well as a confidence level (such as 95%). For small samples (usually less than 30), the t distribution can be used to calculate the confidence interval, and for large samples, the normal distribution can be used. This example uses a 95% confidence level and uses the critical value of the t distribution for small samples (for example, for 5 samples, the degrees of freedom are 4, and the critical value of the t distribution is approximately 2.776). Then the confidence interval = mean ± critical value of the t distribution × standard deviation / =[231.34,248.66]. Therefore, for this set of data, the 95% confidence interval is approximately [231.34, 248.66].

[0064] The confidence level of the above simulation project needs to be evaluated. We selected 5 experts to evaluate the simulation results, and the weights of each expert were 0.15, 0.2, 0.25, 0.3, and 0.1. The sum of these weights is 1. For example, if the five experts scored a certain indicator as 8, 9, 7, 6, and 5, and this example uses a 10-point scoring system, the score of the indicator is 0.15×8+0.2×9+0.25×7+0.3×6+0.1×5=7.05. After obtaining the score of each indicator, the weight of each indicator multiplied by its score is added to get the confidence score of the entire system.

[0065] Five evaluation indicators are selected for the simulation system to be evaluated, and the weight and score of each indicator are as follows: Indicator 1: weight 0.2, score 8; Indicator 2: weight 0.3, score 7; Indicator 3: weight 0.1, score 9; Indicator 4: weight 0.25, score 6; Indicator 5: weight 0.15, score 8.5. According to the weight and score of each indicator, the confidence score of the entire system is calculated. The calculation formula is as follows: Confidence score = ∑ (indicator weight × indicator score), the confidence score of this embodiment = (0.2×8) +(0.3×7)+(0.1×9) +(0.25×6) +(0.15×8.5) = 7.375, that is, the confidence score of the simulation system is 7.375, and the full score is 10 points.

[0066] At this point, the feasibility assessment of the simulation system is completed. This embodiment obtains influencing factors by collecting and analyzing the original information of the simulation experiment, establishes a confidence assessment index system according to the credibility influencing factors, generates input data for the simulation experiment according to the simulation result influencing factors and performs data quality detection, uses data that meets the data quality requirements to conduct simulation experiments and obtain assessment data, uses the established confidence assessment index system to evaluate the assessment data and obtain confidence assessment results, and effectively improves the accuracy of the credibility assessment of complex simulation systems.

[0067] It is worth noting that this embodiment is only used to illustrate the specific implementation method of the present application. The present application does not specifically limit whether the simulation system includes a digital model. It can be applied to the present embodiment and can also be applied to the credibility assessment of other complex simulation systems. Those skilled in the art should choose an appropriate method according to the actual application requirements, which will not be elaborated here.

[0068] Based on the simulation system credibility evaluation method of the above embodiment, an embodiment of the present invention further provides a simulation system credibility evaluation system based on data quality, such as Figure 2 As shown, it includes information collection and analysis module, confidence evaluation index system construction module, input data generation module, and simulation experiment detection and confidence evaluation module, among which, The information collection and analysis module is used to collect the original information of the simulation experiment in the simulation system, analyze the original information and obtain the influencing factors of the simulation experiment, the original information includes the simulation target document, simulation scenario configuration document, digital model document and simulation subsystem document of the simulation experiment, and the influencing factors include credibility influencing factors and simulation result influencing factors; The confidence evaluation index system construction module is used to construct the confidence evaluation index system of the simulation experiment according to the credibility influencing factors, and set the index weights of each layer of the confidence evaluation index system; The input data generation module is used to generate multiple groups of input data for the simulation experiment according to the simulation result influencing factors, and to detect each group of input data according to a preset data quality detection standard and generate a detection result; The simulation experiment detection and confidence assessment module is used to use the input data whose detection results meet the detection threshold to perform the simulation experiment and obtain assessment data, use the confidence assessment indicator system to evaluate the assessment data and generate the confidence assessment result of the simulation experiment.

[0069] In this embodiment, influencing factors are obtained by collecting and analyzing the original information of the simulation experiment, a confidence evaluation index system is established according to the credibility influencing factors, input data of the simulation experiment is generated according to the simulation result influencing factors and data quality detection is performed, simulation experiments are carried out using data that meets the data quality requirements and evaluation data is obtained, the established confidence evaluation index system is used to evaluate the evaluation data and obtain confidence evaluation results, thereby effectively improving the accuracy of credibility evaluation of complex simulation systems.

[0070] Another embodiment of the present invention provides a computer-readable storage medium, on which a computer program is stored, which is implemented when executed by a processor: collecting original information of a simulation experiment in a simulation system, analyzing the original information and obtaining influencing factors of the simulation experiment, the original information includes a simulation target document, a simulation scenario configuration document, a digital model document and a simulation subsystem document of the simulation experiment, and the influencing factors include credibility influencing factors and simulation result influencing factors; constructing a confidence evaluation index system of the simulation experiment according to the credibility influencing factors, and setting the index weights of each layer of the confidence evaluation index system; generating multiple groups of input data of the simulation experiment according to the simulation result influencing factors, and detecting each group of input data according to a preset data quality detection standard and generating a detection result; using the input data whose detection results meet the detection threshold to perform the simulation experiment and obtain evaluation data, using the confidence evaluation index system to evaluate the evaluation data and generate the confidence evaluation result of the simulation experiment. In practical applications, the computer-readable storage medium can adopt any combination of one or more computer-readable media. The computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination thereof. More specific examples of computer-readable storage media (a non-exhaustive list) include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in conjunction with an instruction execution system, device, or device.

[0071] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, which carry computer-readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0072] The program code embodied on the computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0073] Computer program code for performing the operations of the present invention may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0074] like Figure 4 As shown, a structural schematic diagram of a computer device provided by another embodiment of the present invention. Figure 4 The computer device T12 shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0075] like Figure 4 As shown, the computer device T12 is in the form of a general-purpose computing device. The components of the computer device T12 may include, but are not limited to: one or more processors or processing units T16, a system memory T28, and a bus T18 connecting different system components (including the system memory T28 and the processing unit T16).

[0076] Bus T18 represents one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, a graphics acceleration port, a processor, or a local bus using any of a variety of bus structures. For example, these architectures include but are not limited to an Industry Standard Architecture (ISA) bus, a Micro Channel Architecture (MAC) bus, an Enhanced ISA bus, a Video Electronics Standards Association (VESA) local bus, and a Peripheral Component Interconnect (PCI) bus.

[0077] The computer device T12 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by the computer device T12, including volatile and non-volatile media, removable and non-removable media.

[0078] The system memory T28 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) T30 and / or cache memory T32. The computer device T12 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, the storage system T34 may be used to read and write non-removable, non-volatile magnetic media ( Figure 4 not shown, usually called a "hard drive"). Although Figure 4 Not shown in the figure, a disk drive for reading and writing a removable non-volatile disk (e.g., a "floppy disk"), and an optical disk drive for reading and writing a removable non-volatile optical disk (e.g., a CD-ROM, a DVD-ROM, or other optical medium) may be provided. In these cases, each drive may be connected to the bus T18 via one or more data medium interfaces. The memory T28 may include at least one program product having a set (e.g., at least one) of program modules that are configured to perform the functions of the various embodiments of the present invention.

[0079] A program / utility T40 having a set (at least one) of program modules T42 may be stored in, for example, memory T28, such program modules T42 including, but not limited to, an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment. The program modules T42 typically perform the functions and / or methods of the embodiments described herein.

[0080] The computer device T12 may also communicate with one or more external devices T14 (e.g., keyboard, pointing device, display T24, etc.), one or more devices that enable a user to interact with the computer device T12, and / or any device that enables the computer device T12 to communicate with one or more other computing devices (e.g., network card, modem, etc.). Such communication may be performed via an input / output (I / O) interface T22. Furthermore, the computer device T12 may also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN) and / or public network, such as the Internet) via a network adapter T20. Figure 4 As shown, the network adapter T20 communicates with other modules of the computer device T12 via the bus T18. Figure 4 Not shown, other hardware and / or software modules may be used in conjunction with computer device T12, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0081] The processing unit T16 executes various functional applications and data processing by running programs stored in the system memory T28, for example, implementing a simulation system credibility assessment method based on data quality provided in an embodiment of the present invention.

[0082] In view of the existing problems, the present invention develops a method and system for evaluating the credibility of a simulation system based on data quality, obtains influencing factors by collecting and analyzing the original information of the simulation experiment, establishes a confidence evaluation index system according to the credibility influencing factors, generates input data for the simulation experiment according to the simulation result influencing factors and performs data quality detection, uses data that meets the data quality requirements to perform simulation experiments and obtain evaluation data, uses the established confidence evaluation index system to evaluate the evaluation data and obtain confidence evaluation results, effectively improves the accuracy of credibility evaluation of complex simulation systems, thereby making up for the problems existing in the prior art and having broad application prospects.

[0083] Obviously, the above embodiments of the present invention are merely examples for clearly illustrating the present invention, and are not limitations on the implementation methods of the present invention. For ordinary technicians in the relevant field, other different forms of changes or modifications can be made based on the above description. It is impossible to list all the implementation methods here. All obvious changes or modifications derived from the technical solution of the present invention are still within the protection scope of the present invention.

Claims

1. A method for evaluating the credibility of a simulation system based on data quality, characterized in that: include: Collecting original information of the simulation experiment in the simulation system, analyzing the original information and obtaining influencing factors of the simulation experiment, the original information includes the simulation target document, simulation scenario configuration document, digital model document and simulation subsystem document of the simulation experiment, and the influencing factors include credibility influencing factors and simulation result influencing factors; Constructing a confidence evaluation index system of the simulation experiment according to the credibility influencing factors, and setting the index weights of each layer of the confidence evaluation index system; Generating multiple groups of input data for the simulation experiment according to the factors affecting the simulation results, and testing each group of input data according to a preset data quality testing standard and generating a testing result; The simulation experiment is performed using input data whose detection results meet the detection threshold and evaluation data is obtained. The evaluation data is evaluated using the confidence evaluation index system and a confidence evaluation result of the simulation experiment is generated.

2. The simulation system credibility evaluation method according to claim 1, characterized in that: The collecting original information of the simulation experiment in the simulation system, analyzing the original information and obtaining the influencing factors of the simulation experiment further includes: Analyze the original information and obtain relevant data of the simulation experiment, wherein the relevant data includes simulation system data, scene environment data, object individual performance data, inter-object collaboration data and simulation result data; The influencing factors are extracted according to the relevant data, wherein the credibility influencing factors include simulation system credibility factors, data and model credibility factors, simulation process credibility factors and simulation result credibility factors, and the simulation result influencing factors include model input variables, scenario parameters and simulation system parameters.

3. The simulation system credibility evaluation method according to claim 2, characterized in that: The step of constructing the confidence evaluation index system of the simulation experiment according to the credibility influencing factors and setting the index weights of each layer of the confidence evaluation index system further includes: The simulation system credibility factor, data and model credibility factor, simulation process credibility factor and simulation result credibility factor are used as first-level evaluation indicators; Decomposing the first-level evaluation indicators step by step according to the experimental scenario of the simulation experiment to form a hierarchical confidence evaluation indicator system; The indicator weights of the indicators at each level in the confidence evaluation indicator system are set.

4. The simulation system credibility evaluation method according to claim 3 is characterized in that: The step of decomposing the first-level evaluation indicators step by step according to the experimental scenario of the simulation experiment and forming a hierarchical confidence evaluation indicator system further includes: The simulation system credibility factor includes high-level architecture credibility, and the high-level architecture credibility includes attribute update delay performance, interaction instance sending performance and interaction instance receiving performance; The model credibility factors include simulation model credibility, simulation data credibility and simulation algorithm credibility. The simulation model credibility includes conceptual model credibility, mathematical model credibility and software model credibility. The simulation data credibility includes solution accuracy, solution speed and calculation method. The simulation algorithm credibility includes data source quality, metadata accuracy, experimental data accuracy and data integrity. The credibility factors of the simulation process include the credibility of the simulation scheme design, the credibility of the simulation boundary condition setting, the credibility of the simulation decision rule, the credibility of the simulation implementation process, the control performance of the simulation system and the data recording performance of the simulation system; The credibility factors of the simulation results include the credibility of the knowledge base, the credibility of the analysis rule base, the credibility of the reasoning mechanism, the credibility of the visualization unit, the performance of the exercise playback, the performance of the analysis statistics and the performance of the data quality management; The step of setting the indicator weights of each layer of indicators in the confidence evaluation indicator system further includes: The simulation credibility score of the simulation experiment is: C=λ A C A +λ M C M +λ P C P +λ R C R , where C is the simulation credibility score, C A is the credibility factor of the simulation system, λ A is the weight of the credibility factor of the simulation system, C M is the model credibility factor, λ M is the weight of the model credibility factor, C P is the credibility factor of the simulation process, λ P is the weight of the credibility factor of the simulation process, C R is the credibility factor of the simulation results, λ R is the weight of the credibility factor of the simulation results; The credibility factors of the simulation system are: , where C A1 is the high-level architecture trustworthiness, λ A1 is the weight of the high-level architecture trustworthiness, , C A1 The i-th third-level indicator, is the weight of the corresponding i-th third-level indicator; The model credibility factors are: , where C Mi C M The i-th secondary index of is the weight of the corresponding secondary indicator, C M1i C M1 The i-th third-level indicator, is the weight of the corresponding i-th third-level indicator, C M2i C M2 The i-th third-level indicator, is the weight of the corresponding i-th third-level indicator, C M3i C M3 The i-th third-level indicator, is the weight of the corresponding i-th third-level indicator; The credibility factors of the simulation process are: , where C Pi C P The i-th secondary index of is the weight of the corresponding i-th secondary indicator; The credibility factors of simulation results are: , where C Ri C R The i-th secondary index of is the weight of the corresponding i-th secondary indicator.

5. The simulation system credibility evaluation method according to claim 1, characterized in that: The step of generating multiple groups of input data for the simulation experiment according to the factors affecting the simulation results, and testing each group of input data according to a preset data quality testing standard and generating a testing result further includes: Determine the model input variables, scenario parameters and simulation system parameters involved in the simulation experiment according to the simulation result influencing factors, and obtain the value ranges of the model input variables, scenario parameters and simulation system parameters respectively; Generating multiple groups of input data for the simulation experiment according to the value ranges of the model input variables, scenario parameters and simulation system parameters; Determining data content of data quality to be tested in the multiple groups of input data, as well as data quality testing dimensions and data quality problem categories; According to the preset data quality detection standard, each group of input data is tested and a detection result is generated to determine whether the detection result meets the detection threshold. If it does not meet the detection threshold, jump to generating multiple groups of input data for the simulation experiment according to the value range of the model input variables, scenario parameters and simulation system parameters.

6. The method for evaluating the credibility of a simulation system according to claim 5, characterized in that: The testing of each group of input data according to the preset data quality testing standard and generating the testing results further includes: Determine the data quality problem determination standard, detection method and detection process based on the data quality detection dimension and data quality problem category to form the data quality detection standard; The data content of the input data is detected according to the data quality detection standard and a detection result is generated.

7. The method for evaluating the credibility of a simulation system according to claim 6, characterized in that: The step of using the input data whose detection results satisfy the detection threshold to perform the simulation experiment and obtain evaluation data, and using the confidence evaluation index system to evaluate the evaluation data and generate the confidence evaluation result of the simulation experiment further includes: The evaluation data is aggregated and calculated step by step according to the confidence evaluation index system to obtain the credibility score of the simulation experiment as the confidence evaluation result.

8. A simulation system credibility assessment system based on data quality, characterized in that: It includes information collection and analysis module, confidence evaluation index system construction module, input data generation module, and simulation experiment detection and confidence evaluation module, among which, The information collection and analysis module is used to collect the original information of the simulation experiment in the simulation system, analyze the original information and obtain the influencing factors of the simulation experiment, the original information includes the simulation target document, simulation scenario configuration document, digital model document and simulation subsystem document of the simulation experiment, and the influencing factors include credibility influencing factors and simulation result influencing factors; The confidence evaluation index system construction module is used to construct the confidence evaluation index system of the simulation experiment according to the credibility influencing factors, and set the index weights of each layer of the confidence evaluation index system; The input data generation module is used to generate multiple groups of input data for the simulation experiment according to the simulation result influencing factors, and to detect each group of input data according to a preset data quality detection standard and generate a detection result; The simulation experiment detection and confidence assessment module is used to use the input data whose detection results meet the detection threshold to perform the simulation experiment and obtain assessment data, use the confidence assessment indicator system to evaluate the assessment data and generate the confidence assessment result of the simulation experiment.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

10. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the method according to any one of claims 1 to 7 is implemented.

Images