Intelligent model interpretable metering method
By evaluating the credibility of the dataset of the intelligent model and adopting a multi-index evaluation method, the problem of insufficient dataset credibility in the measurement of intelligent model interpretability is solved, and the accurate measurement and evaluation of model interpretability is realized.
Patent Information
- Application Number
- CN202511067032.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-07-31
- Publication Date
- 2025-11-18
AI Technical Summary
In existing technologies, the interpretability measurement methods for intelligent models lack credible and reliable dataset evaluation metrics, which makes it impossible to accurately assess the interpretability of the models.
By defining the evaluation tasks, collecting and processing data, selecting credible evaluation indicators, preparing evaluation resources and environment, implementing credible dataset evaluation, and calculating evaluation indicators, the credibility of the dataset is evaluated using indicators such as repeatability, completeness, accuracy, consistency, traceability, and uncertainty. These indicators are then input into an intelligent model to calculate interpretability indicators.
It enables accurate measurement of the interpretability of intelligent models, ensures the credibility and reliability of input data, and improves the accuracy and credibility of model interpretability assessment.
Smart Images
Figure CN120975249A_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The application belongs to the technical field of metrology, and particularly relates to an intelligent model explainable metrology method. BACKGROUND
[0002] At present, intelligent models in various industries are uneven in intelligence level, and the explainability of intelligent models is difficult to describe, and a suitable explainable metrology method for intelligent models cannot be found. Common explainability calculation indicators include explainability score, accuracy, reliability, and local explanation. For example, if the recall rate and the precision rate are higher, the model classification ability is reliable, and the explainability is stronger, otherwise the explainability is weaker.
[0003] Reliability is a key indicator of intelligent model explainability metrology, and reliability evaluation is a key step to ensure the reliability of intelligent models and is one of the important technologies to realize explainable artificial intelligence. The reliability evidence of an intelligent model refers to the relevant indicators used to measure its reliability and can be extracted from the system. When evaluating the reliability of an intelligent model, three aspects are usually considered: the reliability of training data, the reliability of learning models, and the reliability of prediction results.
[0004] In an intelligent model, data plays a key role, and the reliability of the training data set has a direct impact on the overall reliability of the system. Ensuring and evaluating the reliability of the data set is crucial to measuring the reliability of the intelligent model. Therefore, a scheme for evaluating the reliability of data from the reliability of the data source is proposed.
[0005] If there is a big controversy over the reliability of data resources in the explainability metrology of intelligent models, the reliability of the input data cannot be determined, and the explainability evaluation result obtained cannot be trusted by users.
[0006] In the current intelligent model explainability method, the data set input by the model lacks reliable and reliable sources and evaluation indicators. Therefore, in the traditional process of explainability evaluation and metrology of intelligent models, the reliability of the input data cannot be determined.
[0007] If the reliability of the input data cannot be determined, the accuracy and reliability of the indicators obtained by the explainability metrology method cannot be determined in the explainability metrology process. SUMMARY
[0008] (I) Technical problem to be solved
[0009] The technical problem to be solved by the application is to provide an intelligent model explainability metrology method, which realizes the explainability of the model by calculating the explainability indicators of the model.
[0010] (II) Technical scheme
[0011] In order to solve the above technical problems, the present application provides an intelligent model explainable measurement method, comprising the following steps:
[0012] Step 1, clear evaluation task
[0013] Firstly, the data to be evaluated is determined, the user evaluation requirement and the evaluation object are determined, including the specific parameters of the data set to be evaluated: data format, data label, field attribute, data type and structure, and the intelligent model of the explainable index to be evaluated. Based on the analysis of the evaluation object and the evaluation requirement, the evaluation task is determined, and the evaluation method is selected according to the task;
[0014] Step 2, data collection and processing, the data set includes:
[0015] (1) data set of specific field;
[0016] (2) open data set: data set publicly disclosed by organization or individual;
[0017] (3) simulation data set: data simulated in real environment, data with specific characteristics generated;
[0018] (4) real world data set: data set collected from real world;
[0019] For the errors, abnormal values, repeated data and noises existing in the data set, cleaning and processing are carried out;
[0020] Step 3, select data set credible evaluation index
[0021] Step 4, prepare data set credible evaluation resources and environmental conditions
[0022] Prepare for the development and implementation of the evaluation task, configure relevant resources and environment;
[0023] Step 5, implement data set credible evaluation
[0024] Under the configured evaluation resources and environment, the data to be evaluated is evaluated, the evaluation process is monitored, and the process data and result data are obtained;
[0025] Step 6, calculate the data set credible evaluation index
[0026] According to the index calculation formula in step 3, the evaluation index value of the data to be evaluated is calculated;
[0027] Step 7, adopt index evaluation method to identify data set, identify the data set after the index evaluation is qualified as credible data set;
[0028] Step 8, data set input into the intelligent model to be evaluated
[0029] The identified credible data set after evaluation is input into the intelligent model to be tested as a training data set, and relevant explainability indexes are calculated.
[0030] Step 9, intelligent model explainability index calculation
[0031] The credibility, accuracy, explainability score and local explainability index are selected as the intelligent model explainability calculation indexes, and the index calculation formula is substituted into the credible data set and the intelligent model to be tested to calculate the evaluation index value.
[0032] The application also provides a system for implementing the method.
[0033] (III) beneficial effects
[0034] The application provides a method for measuring the explainability of an intelligent model by using a data set that has passed performance evaluation, referred to as a credible data set, as the input of the intelligent model. The method proposes a set of indexes for evaluating the credibility of the data set: repeatability, integrity, accuracy, consistency, traceability and uncertainty. The data set that has passed the evaluation of the above indexes is identified as a credible data set. The explainability of the model to be tested is evaluated by using the credible data set as the input of the model. The credibility, accuracy, explainability score and local explainability are calculated as the indexes for evaluating the explainability. The above indexes can be used to determine the explainability level of the intelligent model. BRIEF DESCRIPTION OF DRAWINGS
[0035] Figure 1 The method flowchart of the application. DETAILED DESCRIPTION
[0036] In order to make the purpose, content and advantages of the application more clear, the specific embodiments of the application are described in further detail below in combination with the drawings and examples.
[0037] The application relates to a method for measuring the explainability of an intelligent model. A data set that has passed evaluation, referred to as a credible data set (a data set that has passed the evaluation of indexes such as repeatability, integrity, accuracy, consistency, traceability and uncertainty), is used as the input of the intelligent model. The explainability of the model is measured by calculating the corresponding explainability indexes.
[0038] The method is to take a performance evaluation qualified data set, referred to as a trusted data set (repeatability, integrity, accuracy, consistency, traceability, uncertainty, etc. Evaluation of indicators of data set) as the input of the intelligent model to measure the explainability of the intelligent model. The method proposes a set of indicators for evaluating the credibility of the data set: repeatability, integrity, accuracy, consistency, traceability, and uncertainty, a total of six indicators. The data set that passes the evaluation of the above indicators is identified as a trusted data set. The explainability of the model under test is evaluated by inputting the trusted data set into the model. The credibility, accuracy, explainability score, and local explanation are calculated as indicators for explainability evaluation. The above indicators can be used to determine the explainability level of the intelligent model.
[0039] Trusted data set overview:
[0040] 1. Evaluation requirements of trusted data set
[0041] The evaluation of the trusted data set is mainly based on the definition of standard reference data in JJF1001, while considering the latest achievements at home and abroad. The "strict evaluation" in the definition is mainly embodied in the evaluation of the standard attributes of the standard reference data, and is completed by using the evaluation method; the "accuracy verification" in the definition is mainly embodied in the metrological properties of the standard reference data, and is completed by using the verification method. The list of materials is shown in Appendix C and Appendix D.
[0042] 2. Source identification of standard reference data
[0043] 1) Resource identification, including but not limited to:
[0044] ① Standard series: GB (national standard), GJB (national military standard), JJG (measurement and calibration regulations, measurement and calibration system table), JJF (other measurement technical specifications), ISO, IEC, DE (German standard), local standard, industry standard (market supervision)
[0045] ② Publication series: DOI (digital resource identifier), ISBN (publication number)
[0046] ③ Discipline series: Ministry of Education code, National Natural Science Foundation code
[0047] ④ Personnel series: ORCID (international general personnel code)
[0048] ⑤ Others.
[0049] 2) Qualification certificate, including but not limited to: measurement certificate (CMA), CNAS test certificate, identification certificate, comparison report, and others.
[0050] 3. Strict evaluation
[0051] Including but not limited to: integrity, consistency, utility, reliability.
[0052] 4. Accuracy verification
[0053] Including but not limited to: accuracy, traceability, repeatability, relevance.
[0054] The following is a brief description of the evaluation method and process of the present application: Figure 1 The evaluation method and process of the present application are described as follows:
[0055] Step 1, clear evaluation task
[0056] First, the data to be evaluated needs to be clear, and the user's evaluation needs and evaluation objects need to be understood, such as the specific parameters of the data set to be tested: data format, data label, field attribute, data type and structure, etc., and the intelligent model of the explainable index to be tested, based on the analysis of the evaluation object and the evaluation demand, the evaluation task is clear, and the evaluation method is selected according to the task.
[0057] Step 2, data collection and processing
[0058] (1) Data set provided by professional institutions or research institutions: professional institutions provide data sets in specific fields.
[0059] (2) Open data set: data set publicly provided by organizations or individuals for researchers or developers to use. These data sets may be real-world data or artificially generated data.
[0060] (3) Simulation data set: simulate data in real environment, artificially generate data with specific characteristics.
[0061] (4) Real-world data set: data set collected from real world.
[0062] In order to ensure the credibility of the data set of the test AI system, the following methods can be used to control the credibility:
[0063] When selecting and collecting data, the source, authenticity, integrity, diversity and representativeness of the data set need to be considered. The data set should have a certain scale and coverage. For the problems of error, abnormal value, repeated data, noise and other problems existing in the data set, cleaning and processing are needed.
[0064] Step 3, select data set credible evaluation index
[0065] Evaluation index as follows:
[0066] 1) Repeatability evaluation index definition see Table 1.
[0067] Table 1 Repeatability evaluation index
[0068]
[0069] 2) The integrity (completeness) evaluation index definition is shown in Table 2.
[0070] Table 2 Integrity (completeness) evaluation index
[0071]
[0072] 3) The accuracy evaluation index definition is shown in Table 3.
[0073] Table 3 Accuracy evaluation index
[0074]
[0075]
[0076] 4) The consistency evaluation index definition is shown in Table 4.
[0077] Table 4 Consistency evaluation index
[0078]
[0079]
[0080] 5) The traceability (traceability) evaluation index definition is shown in Table 5.
[0081] Table 5 Traceability (traceability) evaluation index
[0082]
[0083] 6) The uncertainty evaluation index definition is shown in Table 6.
[0084] Table 6 Uncertainty evaluation index
[0085]
[0086] Step 4, prepare data set credible evaluation resources and environmental conditions
[0087] Prepare for the development and implementation of the evaluation task, configure the relevant resources and environment: prepare the data to be evaluated, evaluation index, evaluation method, software and hardware operating environment, etc.
[0088] Step 5, implement data set credible evaluation
[0089] Under the configured evaluation resources and environment, implement the evaluation of the data to be evaluated, monitor the evaluation process and obtain process data and result data.
[0090] Step 6, calculate the evaluation index of the data set credibility
[0091] According to the index calculation formula in 3, the evaluation data is calculated to obtain the evaluation index value.
[0092] Step 7, index evaluation
[0093] Combined with the following index evaluation method, the final evaluation result is obtained.
[0094] (1) Objective weighting method
[0095] Use existing weight assignment algorithm, including but not limited to entropy method, fuzzy comprehensive evaluation method, grey correlation method, etc.
[0096] (2) Subjective weighting method
[0097] According to the importance of influencing factors in specific application scenarios, the weight is assigned artificially, including but not limited to expert consultation method, rating method, AHP method, etc.
[0098] (3) Mixed weighting method
[0099] The combination of subjective weighting and objective weighting method not only ensures the objectivity and credibility of weighting, but also minimizes the uncertainty of weighting.
[0100] The data set after index evaluation is qualified can be identified as a credible data set.
[0101] Step 8, data set input into the intelligent model to be evaluated
[0102] The credible data set identified after evaluation is qualified as the training data set input into the intelligent model to be evaluated, and the relevant interpretability index is calculated.
[0103] Step 9, intelligent model interpretability index calculation
[0104] Select the credibility, accuracy, interpretability score, and local interpretation as the intelligent model interpretability calculation index, and according to the index calculation formula, substitute the credible data set and the intelligent model to be evaluated to calculate the evaluation index value.
[0105] Step 10, interpretable evaluation result and analysis
[0106] Combined with the index evaluation method in step 7, the final evaluation result is obtained.
[0107] Step 11, obtain the evaluation result and report
[0108] Comprehensive evaluation process and its results, issue the corresponding evaluation report.
[0109] Evaluation related appendix:
[0110] The evaluation original record reference format, data material review list and data review list refer to appendix B, C, D.
[0111] Appendix A (normative) Evaluation example
[0112] A.1 File format
[0113] The interface function is developed in C / python language, supports multi-threading, and can be compiled into 32-bit or 64-bit version.
[0114] A.2 Interface function
[0115] The evaluation interface function is shown in Table A.1.
[0116] Table A.1
[0117]
[0118] A.3 Interface function description
[0119] The definition of each test interface is as follows:
[0120] (1) Data to be tested loading interface:
[0121] Number: 1
[0122] Interface name: Evaluation_inputdata
[0123] Interface function: used to realize the access of data to be tested
[0124] Interface input parameter symbol: inputdata
[0125] Interface input parameter meaning: data to be tested
[0126] Interface input parameter data type: image data or other data types
[0127] Communication protocol: TCP / IP protocol, HTTP protocol
[0128] (2) Data set evaluation index selection and calculation interface:
[0129] Number: 2
[0130] Interface name: Evaluation_inputindex
[0131] Interface function: select the index in 5.1 and implement the calculation process.
[0132] Interface input parameter symbol: cal_index
[0133] Interface input parameter meaning: calculation evaluation index interface input parameter data type: no communication protocol: TCP / IP protocol, HTTP protocol
[0134] (3) Data set evaluation method selection interface:
[0135] No.: 3
[0136] Interface name: Evaluation_inputmethod
[0137] Interface function: used to introduce the selected evaluation method, which selects a certain evaluation method for evaluation.
[0138] Interface input parameter symbol: cal_method
[0139] Interface input parameter meaning: selection of evaluation method interface input parameter data type: no communication protocol: TCP / IP protocol, HTTP protocol
[0140] (4) Data set evaluation result output and display value:
[0141] No.: 4
[0142] Interface name: Evaluation_outputresult
[0143] Interface function: used to output the calculated index results and display.
[0144] Interface output parameter symbol: out_result
[0145] Interface output parameter meaning: output and display index.
[0146] Interface output parameter data type: string communication protocol: TCP / IP protocol, HTTP protocol
[0147] (5) Result storage:
[0148] No.: 5
[0149] Interface name: Evaluation_storageresult
[0150] Interface function: used to store the evaluation index data obtained after calculation
[0151] Interface output parameter symbol: store_result
[0152] Interface output parameter meaning: store results
[0153] Interface output parameter data type: string
[0154] Communication protocol: TCP / IP protocol, HTTP protocol
[0155] (6) Result visualization:
[0156] No. 6
[0157] Interface name: Evaluation_Visualizationresult
[0158] Interface function: used for inputting the calculation results of evaluation indicators into the visualization tool for visualization display
[0159] Interface output parameter symbol: vision_result
[0160] Interface output parameter meaning: evaluation results
[0161] Interface output parameter data type: image data
[0162] Communication protocol: TCP / IP protocol, HTTP protocolA.4 Uncertainty evaluationA.4.1 Environmental conditions:
[0163] Temperature: 21.0℃, relative humidity: 42%.
[0164] A.4.2 Data collection:
[0165] Collect the temperature and humidity dial data set.
[0166] A.4.3 Measured object:
[0167] Temperature and humidity dial data set and dial data set reading identification model.
[0168] A.4.4 Measurement method:
[0169] The temperature and humidity dial data set is used as the data to be evaluated, the evaluation method is selected as the complete inspection method, the evaluation indicators are calculated, the index calculation results are collected and analyzed by the index evaluation method, and the data set that meets the evaluation requirements is identified as the reliable data. The reliable data set is input into the dial data set reading identification model to be evaluated, and the evaluation results of the interpretability indicators of the model are obtained by calculation.
[0170] A.4.5 Uncertainty sources:
[0171] a) Uncertainty component V1 introduced by data set evaluation method
[0172] b) Uncertainty component V2 introduced by data set evaluation process
[0173] c) Uncertainty component V3 introduced by data set to be evaluated
[0174] d) Uncertainty component V4 introduced by data set reliability index evaluation method
[0175] The final result of the uncertainty is u i .
[0176] A.4.6 Evaluation of standard uncertainty
[0177] The data set of the temperature and humidity table to be measured has uncertainty, the selection of the evaluation method introduces uncertainty, the evaluation process introduces uncertainty, and different index evaluation methods also introduce uncertainty.A.4.7 Calculation of combined standard uncertainty
[0178] The summary of uncertainty components is shown in Table A.1.
[0179] Table A.1 Summary of uncertainty components
[0180]
[0181] The combined standard uncertainty is:
[0182]
[0183] A.4.8 Expanded uncertaintyA.4.8.1 Calculation of expanded uncertainty
[0184] Take the confidence probability p = 95%, v eff → ∞, look up the t distribution table to get k = 2
[0185] The expanded uncertainty U = ku c = 2 x 0.34 = 0.68
[0186] The relative expanded uncertainty is:
[0187] A.4.8.2 Uncertainty of other intelligent models of measurement and control equipment
[0188] The interpretability evaluation index and uncertainty of the intelligent model can be obtained by the above method.
[0189] Appendix B (normative) Reference format of evaluation original record
[0190] Evaluation result record
[0191]
[0192] Appendix C (normative) Data information review list record
[0193]
[0194] Appendix D (normative) Data review list record
[0195]
[0196]
[0197] The above merely describes the preferred embodiments of the present application, and it should be pointed out that, for those skilled in the art, several improvements and modifications can be made without departing from the technical principles of the present application, and these improvements and modifications should also be considered as the protection scope of the present application.
Claims
1. An intelligent model-interpretable measurement method, characterized in that, Includes the following steps: Step 1: Define the assessment task First, clearly define the data to be evaluated, determine the user's evaluation needs and the evaluation objects, including the specific parameters of the dataset to be evaluated: data format, data labels, field attributes, data types and structures, as well as the intelligent model of the interpretable indicators to be evaluated. Based on the analysis of the evaluation objects and evaluation needs, clarify the evaluation tasks, and select the evaluation methods according to the tasks. Step 2, Data Collection and Processing. The dataset includes: (1) Data sets in specific fields; (2) Open datasets: Data sets that are made public by an organization or individual; (3) Simulation dataset: Data with specific characteristics generated by simulating real-world environments; (4) Real-world datasets: Data sets collected from the real world; The dataset contains errors, outliers, duplicate data, and noise. These are cleaned and processed. Step 3: Select reliable evaluation metrics for the dataset. The evaluation indicators are as follows: 1) The definitions of repeatability evaluation indicators are shown in Table 1: Table 1 Repeatability Evaluation Indicators 2) The definitions of integrity evaluation indicators are shown in Table 2: Table 2 Integrity Evaluation Indicators 3) The definitions of accuracy evaluation indicators are shown in Table 3: Table 3 Accuracy Evaluation Indicators 4) The definitions of the consistency evaluation indicators are shown in Table 4: Table 4 Consistency Evaluation Indicators 5) The definitions of traceability evaluation indicators are shown in Table 5: Table 5. Traceability Evaluation Indicators 6) The definitions of uncertainty evaluation indicators are shown in Table 6: Table 6 Uncertainty Evaluation Indicators Step 4: Prepare the dataset for credibility assessment resources and environmental conditions To prepare for the assessment task and its implementation, relevant resources and environment should be allocated. Step 5: Conduct a dataset credibility assessment The evaluation data to be evaluated is carried out in the configured evaluation resources and environment, the evaluation process is monitored and process data and result data are obtained; Step 6: Calculate the reliability metrics for the dataset. Based on the indicator calculation formula in step 3, calculate the value of the evaluation indicator from the data to be evaluated. Step 7: Use one of the following evaluation methods to identify the dataset: (1) Objective Assignment Method (2) Subjective Assignment Method Weights are assigned based on the importance of influencing factors in specific application scenarios; (3) Hybrid empowerment method: combining subjective empowerment with objective empowerment methods; Data sets that pass the indicator evaluation are recognized as trustworthy data sets; Step 8: Input the dataset into the intelligent model to be evaluated. The trusted datasets that have passed the evaluation are used as training datasets and input into the intelligent model to be evaluated to calculate relevant interpretability metrics. Step 9: Calculation of interpretable indicators by intelligent models Credibility, accuracy, interpretability score, and local interpretability index are selected as the interpretability calculation indicators for intelligent models. Based on the indicator calculation formula, the values of the evaluation indicators are calculated by substituting the credible dataset and the intelligent model under test.
2. The method as described in claim 1, characterized in that, It also includes step 10, interpretable evaluation results and analysis: combining the indicator evaluation methods in step 7, to obtain the final evaluation results.
3. The method as described in claim 1, characterized in that, It also includes step 11, obtaining the evaluation results and report: combining the evaluation process and its results, the corresponding evaluation report is obtained.
4. The method as described in claim 1, characterized in that, In step 2, when collecting data, the following methods are used for credibility control: consider the source, authenticity, completeness, diversity, and representativeness of the dataset.
5. The method as described in claim 1, characterized in that, Step 4 involves configuring relevant resources and environment, including preparing the data to be evaluated, evaluation indicators, evaluation methods, and software and hardware operating environment.
6. The method as described in claim 1, characterized in that, Objective assignment methods include entropy method, fuzzy comprehensive evaluation method, and grey relational analysis method.
7. The method as described in claim 1, characterized in that, Weighting methods include expert consultation, rating and scoring, and AHP (Analytic Hierarchy Process).
8. The method according to any one of claims 1 to 7, characterized in that, This method is applied in the field of metrology.
9. A system for implementing the method as described in any one of claims 1 to 8.
10. The system as described in claim 9, characterized in that, This system is applied in the field of metrology.