Method and system for evaluation of measurement uncertainty based on machine learning

Abstract

The application discloses a method and system for evaluating measurement result uncertainty based on machine learning, and relates to the technical field of measurement data processing, comprising the following steps: constructing a basic non-consistency score based on the measurement prediction bias in the calibration sample, combining the prediction uncertainty to form a non-consistency score, extracting a confidence interval threshold according to the confidence level, implementing boundary correction on the prediction interval in the double-channel evaluation set to form an interval calibration data set; identifying boundary risk samples based on the distribution difference between online evaluation samples and training samples, performing interval expansion on the interval calibration data set, triggering a dynamic retest mechanism when the uncertainty after expansion is higher than the uncertainty threshold, performing interval update correction on the interval calibration data set, and outputting an uncertainty evaluation set. The application realizes adaptive correction under complex working conditions, thereby achieving the effect of more reliable and more robust measurement results.

Description

Technical Field

[0001] This invention relates to the field of measurement data processing technology, and in particular to a method and system for evaluating the uncertainty of measurement results based on machine learning. Background Technology

[0002] With the continuous development of intelligent sensing, online detection, industrial process monitoring, and complex equipment condition awareness technologies, measurement results have gradually evolved from single instrument readings to comprehensive outputs driven by multi-source correlated data. In recent years, machine learning methods have been widely used for measurement result modeling, error compensation, soft measurement estimation, and anomaly identification, especially suitable for result prediction under nonlinear, multivariable, and strongly coupled operating conditions. Meanwhile, interval representation, sample calibration, distribution drift identification, and online update mechanisms for measurement reliability are also gradually becoming important research directions in measurement data processing.

[0003] While existing technologies can improve the fitting accuracy of measurement results by using machine learning, the characterization of measurement uncertainty often remains at the level of single discrete statistics or empirical thresholds. It is difficult to take into account both the propagation characteristics of input disturbances and the fluctuation characteristics of model cognition, resulting in the uncertainty boundary lacking sample specificity and dynamic adaptability. Summary of the Invention

[0004] In view of the aforementioned existing problems, the present invention is proposed.

[0005] Therefore, this invention provides a machine learning-based method for evaluating measurement uncertainty, which solves the problem that measurement uncertainty is difficult to characterize from two sources and be dynamically calibrated for evaluation.

[0006] To solve the above-mentioned technical problems, the present invention provides the following technical solution: In a first aspect, the present invention provides a method for evaluating the uncertainty of measurement results based on machine learning, which includes: acquiring measurement-related data and preprocessing it, dividing it into training samples, calibration samples and online evaluation samples, constructing a measurement result prediction model based on the training samples and outputting an initial prediction representation set; Based on the initial prediction characterization set, sensitive information on the propagation relationship of input disturbances is extracted to determine the propagation uncertainty, and cognitive response information on the discreteness of the model output is extracted to determine the prediction uncertainty, forming a dual-channel evaluation set; Based on the measurement prediction bias in the calibration samples, a basic non-consistency score is constructed. Combined with the prediction uncertainty weighting correction, an uncertainty non-consistency score is formed. The confidence interval threshold is extracted according to the confidence level quantile. Boundary correction is applied to the prediction interval in the dual-channel evaluation set to form an interval calibration dataset. Based on the distribution difference between online evaluation samples and training samples, boundary risk samples are identified. The interval calibration dataset is expanded. When the uncertainty after expansion is higher than the uncertainty threshold, a dynamic retesting mechanism is triggered to update and correct the interval calibration dataset, and an uncertainty evaluation set is output.

[0007] As a preferred embodiment of the machine learning-based measurement result uncertainty evaluation method of the present invention, the specific steps of acquiring measurement correlation data and preprocessing it to divide it into training samples, calibration samples, and online evaluation samples are as follows: Acquire measurement-related data, and sequentially perform outlier identification and removal, missing value completion, and data standardization on the measurement-related data to output a standardized basic dataset. Feature extraction and feature selection are performed on the standardized basic dataset, the relevant features of the model input are retained and the corresponding model learning feature space is constructed, and the preprocessed modeling sample set is output. Using the preprocessed modeling sample set as the processing object, the main samples under historical working conditions are divided into training samples according to the working condition stratification rules, the remaining samples under the working conditions corresponding to the training samples are divided into calibration samples, and the samples collected under subsequent working conditions are divided into online evaluation samples.

[0008] As a preferred embodiment of the machine learning-based measurement result uncertainty evaluation method of the present invention, the specific steps of constructing a measurement result prediction model based on training samples and outputting an initial prediction representation set are as follows: Using the feature variables in the training samples as input and the corresponding measurement results as model output, the random forest regression model is fitted and trained to establish the feature-result correspondence, and the measurement result prediction model is obtained after ensemble calculation. The calibration samples are input into the measurement result prediction model. Each decision tree performs splitting judgment and regression calculation on the feature variables in the calibration samples. The initial prediction representation set is output through ensemble mean calculation.

[0009] As a preferred embodiment of the machine learning-based measurement result uncertainty evaluation method of the present invention, the steps of extracting sensitivity information of input disturbance propagation relationship based on initial prediction representation set to determine propagation uncertainty, and extracting cognitive response information of model output discreteness to determine prediction uncertainty, forming a dual-channel evaluation set, are as follows: The initial prediction characterization set is parsed and separated to extract disturbance characterization data and response characterization data; Based on the disturbance characterization data, an input disturbance processing task is constructed, and input disturbance analysis is performed to establish the input disturbance propagation relationship, thereby forming disturbance propagation data; Aggregate and characterize the disturbance propagation data to determine the sensitive information of the input disturbance propagation relationship and form the propagation uncertainty; Discrete correlation analysis is performed on the discrete representation results of the model output in the response representation data to extract cognitive response information, and the discreteness of the cognitive response information is characterized to determine the prediction uncertainty. The propagation uncertainty and prediction uncertainty are aligned and encapsulated according to the correspondence of the same sample to form a dual-channel evaluation set.

[0010] As a preferred embodiment of the machine learning-based measurement result uncertainty evaluation method of the present invention, the specific steps for constructing the basic inconsistency score based on the measurement prediction bias in the calibration sample are as follows: The sample predictions, propagation uncertainties, and prediction uncertainties in the dual-channel evaluation set are matched one-to-one with the actual measurement results in the calibration samples according to the same sample identifier to form a calibration association dataset. Based on the calibration association dataset, the measurement prediction deviation between the actual measurement results and the predicted values ​​of each calibration sample is calculated, and each measurement prediction deviation is constructed as a basic inconsistency score.

[0011] As a preferred embodiment of the machine learning-based measurement result uncertainty evaluation method of the present invention, the specific steps for forming an uncertainty inconsistency score by combining the weighted correction of prediction uncertainty are as follows: Read the prediction uncertainty of the sample corresponding to the basic non-consistent score, and perform sample-level weight mapping on the prediction uncertainty to form an uncertainty weight set; The uncertainty inconsistency score is formed by weighting and correcting the basic inconsistency score set based on the uncertainty weight set.

[0012] As a preferred embodiment of the machine learning-based measurement result uncertainty evaluation method of the present invention, the specific steps of extracting confidence interval thresholds according to confidence level quantiles and performing boundary correction on the prediction intervals in the dual-channel evaluation set to form an interval calibration dataset are as follows: The inconsistency scores of uncertainty are sorted, and the scores at the corresponding quantile positions are extracted according to the confidence level to determine the confidence interval threshold. Read the sample predicted values, propagation uncertainty, and prediction uncertainty from the dual-channel evaluation set. Center on the sample predicted values, construct the prediction interval for each sample based on the propagation uncertainty and prediction uncertainty, apply the confidence interval threshold to the upper and lower boundaries of each prediction interval, perform boundary correction processing, and output the interval correction result set. The interval correction result set, as well as the upper and lower boundaries of the corrected prediction interval, are encapsulated at the sample level to output the interval calibration dataset.

[0013] As a preferred embodiment of the machine learning-based measurement result uncertainty assessment method of the present invention, the specific steps for identifying boundary risk samples based on the distribution difference between online evaluation samples and training samples are as follows: Read the input features from the training samples, perform statistical calculations on each input feature, and obtain the distribution reference parameters corresponding to the training samples; Read the input features from the online evaluation samples, calculate the difference between each online evaluation sample and the distribution reference parameters, determine the distribution difference of each online evaluation sample, and form a distribution difference dataset; The boundary risk assessment threshold is determined based on the distribution difference dataset, and the distribution difference of each online assessment sample is compared with the boundary risk assessment threshold to identify boundary risk samples.

[0014] As a preferred embodiment of the machine learning-based measurement result uncertainty evaluation method of the present invention, the specific steps for outputting the uncertainty evaluation set are as follows: Read the prediction intervals corresponding to the boundary risk samples in the interval calibration dataset, and perform interval expansion processing on the corresponding prediction intervals according to the distribution differences of the boundary risk samples to form an interval expansion result set; The expanded uncertainty of each boundary risk sample is calculated based on the interval expansion result set, and the expanded uncertainty is compared with the uncertainty threshold. Boundary risk samples with expanded uncertainty higher than the uncertainty threshold are identified as retest trigger samples, and the dynamic retest mechanism is triggered. Based on the dynamic retesting mechanism, the retesting trigger sample is re-measured to obtain the retesting measurement value, and the corresponding prediction interval is shifted and corrected according to the positional relationship between the retesting measurement value and the original prediction interval, and the uncertainty assessment set is output.

[0015] Secondly, the present invention provides a measurement result uncertainty evaluation system based on machine learning, comprising: a modeling and prediction module for acquiring measurement-related data and preprocessing it, dividing it into training samples, calibration samples and online evaluation samples, constructing a measurement result prediction model based on the training samples and outputting an initial prediction representation set; The dual-channel module is used to extract sensitive information about the propagation relationship of input disturbances based on the initial prediction characterization set, determine the propagation uncertainty, and extract cognitive response information about the discreteness of the model output to determine the prediction uncertainty, thus forming a dual-channel evaluation set. The interval calibration module is used to construct a basic non-consistency score based on the measurement prediction deviation in the calibration sample, combine it with the prediction uncertainty weighted correction to form an uncertainty non-consistency score, extract the confidence interval threshold according to the confidence level quantile, and perform boundary correction on the prediction interval in the dual-channel evaluation set to form an interval calibration dataset. The update evaluation module is used to identify boundary risk samples based on the distribution difference between online evaluation samples and training samples. It performs interval expansion on the interval calibration dataset. When the uncertainty after expansion is higher than the uncertainty threshold, it triggers a dynamic retest mechanism to update and correct the interval calibration dataset and outputs an uncertainty evaluation set.

[0016] The beneficial effects of this invention are as follows: by constructing a processing flow that combines training, calibration and online evaluation, and by evaluating propagation uncertainty and prediction uncertainty in a dual-channel manner, the uncertainty of measurement results is accurately characterized and calibrated within a range, thereby improving the reliability of measurement results; at the same time, by expanding the range of boundary risk samples and triggering dynamic retest correction, adaptive correction under complex working conditions is achieved, thus achieving a more reliable and robust measurement result. Attached Figure Description

[0017] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is a flowchart of a machine learning-based method for evaluating the uncertainty of measurement results.

[0019] Figure 2 This is a schematic diagram of a machine learning-based system for evaluating the uncertainty of measurement results.

[0020] Figure 3 A flowchart for forming a dual-channel evaluation set.

[0021] Figure 4 A flowchart for updating the output of the uncertainty assessment set. Detailed Implementation

[0022] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0023] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the spirit of the invention. Therefore, the invention is not limited to the specific embodiments disclosed below.

[0024] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The phrase "in one embodiment" appearing in different places in this specification does not necessarily refer to the same embodiment, nor is it a single or selective embodiment that is mutually exclusive with other embodiments.

[0025] Reference Figures 1-4 As one embodiment of the present invention, this embodiment provides a method for evaluating the uncertainty of measurement results based on machine learning, including the following steps: S1. Acquire measurement-related data and preprocess it to divide it into training samples, calibration samples and online evaluation samples. Based on the training samples, construct a measurement result prediction model and output the initial prediction representation set.

[0026] It should be noted that the output records of the measuring equipment corresponding to the current measurement process are collected, and the environmental monitoring records, equipment operation records, operation execution records and historical reference records corresponding to the current measurement period are retrieved simultaneously and summarized according to the same measurement batch or the same sample identifier.

[0027] S1.1. Obtain measurement-related data, and sequentially perform outlier identification and removal, missing value completion, and data standardization on the measurement-related data to output a standardized basic dataset.

[0028] It should be noted that after obtaining the measurement correlation data, outlier identification and removal are first performed on the measurement correlation data. Abnormal records are identified and deleted according to the allowable range, historical fluctuation range, and distribution of samples in the same batch of measurement correlation data. Then, missing value completion is performed on the measurement correlation data after outlier identification and removal. Missing fields are checked one by one and the corresponding completion methods are used to complete the missing value. Finally, the measurement correlation data after missing value completion is subjected to data standardization processing to unify the numerical scale of various parameters in the measurement correlation data and output a standardized basic dataset.

[0029] S1.2. Perform feature extraction and feature filtering on the standardized basic dataset, retain the features associated with the model input, construct the corresponding model learning feature space, and output the preprocessed modeling sample set.

[0030] It should be noted that candidate features are extracted from the standardized basic dataset, and feature screening is performed on the candidate features to remove those with high repetition, weak correlation, or abnormal fluctuations, while retaining the model input related features. Then, the model input related features are aligned according to the sample labels and arranged in a fixed order to form a unified feature vector, forming a feature matrix, and constructing the model learning feature space. Finally, the model learning feature space is matched one-to-one with the measurement results corresponding to each sample, and the preprocessed modeling sample set is output.

[0031] S1.3. Using the preprocessed modeling sample set as the processing object, the main samples under historical working conditions are divided into training samples according to the working condition stratification rules, the remaining samples under the working conditions corresponding to the training samples are divided into calibration samples, and the samples collected under subsequent working conditions are divided into online evaluation samples.

[0032] It should be noted that, taking the preprocessed modeling sample set as the processing object, the samples are first stratified according to the working condition identifiers corresponding to each sample in the preprocessed modeling sample set, and the main samples in each working condition layer under historical working conditions are classified as training samples; then, calibration samples are classified from the remaining samples in the working condition layer corresponding to the training samples that have not been classified as training samples; finally, the newly collected samples under subsequent working conditions are identified according to the collection time sequence, and the samples collected under subsequent working conditions are classified as online evaluation samples.

[0033] S1.4. Using the feature variables in the training samples as input and the corresponding measurement results as model output, the random forest regression model is fitted and trained to establish the feature-result correspondence, and the measurement result prediction model is obtained after ensemble calculation.

[0034] It should be noted that when fitting and training a random forest regression model using feature variables from the training samples as input and corresponding measurement results as model output, the feature variables of each sample in the training samples are first organized according to the sample identifiers and their corresponding measurement results are then input into the random forest regression model. Next, the random forest regression model generates multiple regression decision trees based on different subsets of samples in the training samples. Each regression decision tree performs node splitting and regression learning according to the correspondence between the feature variables and the measurement results, gradually forming a regression path from the feature variables to the measurement results. After training each regression decision tree, they are combined to form a random forest regression model. Finally, by integrating the outputs of each regression decision tree, a measurement result prediction model is obtained.

[0035] S1.5 Input the calibration samples into the measurement result prediction model, perform split judgment and regression calculation on the feature variables in the calibration samples through each decision tree, and output the initial prediction characterization set through integrated mean calculation.

[0036] It should be noted that after the calibration samples are input into the measurement result prediction model, each decision tree in the model first reads the feature variables in the calibration samples and, according to the node splitting conditions formed during the fitting training process, performs a layer-by-layer splitting judgment on the feature variables in the calibration samples, so that each calibration sample enters the corresponding leaf node. Then, each decision tree performs regression calculation on each calibration sample based on the measurement result regression value associated with the corresponding leaf node, obtaining the measurement result prediction value of the same calibration sample under different decision trees. After that, the multiple measurement result prediction values ​​corresponding to the same calibration sample are integrated and averaged to obtain the sample prediction value corresponding to the calibration sample. Finally, the calibration sample identifier, the multiple measurement result prediction values ​​corresponding to the same calibration sample, and the sample prediction value corresponding to the calibration sample are reconciled and the initial prediction representation set is output.

[0037] Each decision tree performs regression calculations on each calibration sample based on the regression values ​​of the measurement results associated with the corresponding leaf nodes. The expression is as follows: ; in: It is the first In the decision tree and the first A calibration sample enters the leaf node. The corresponding measurement result regression value, and used as the first decision trees for the first Predicted measurement results output from each calibration sample; Is it falling into the first leaf nodes in a decision tree The number of training samples; Is it falling into the first leaf nodes in a decision tree The The actual measurement results corresponding to each training sample; It is the first Feature variables corresponding to each calibration sample Enter the first After establishing a decision tree, the leaf node is determined by node splitting; It is the first Feature variables corresponding to each calibration sample; It is the sequence number of the decision tree in the measurement result prediction model; Is it falling into the first leaf nodes in a decision tree The training sample number.

[0038] S2. Based on the initial prediction characterization set, extract the sensitivity information of the input disturbance propagation relationship, determine the propagation uncertainty, and extract the cognitive response information of the model output discreteness to determine the prediction uncertainty, forming a dual-channel evaluation set.

[0039] S2.1. Perform analytical separation on the initial prediction characterization set to extract disturbance characterization data and response characterization data.

[0040] It should be noted that, firstly, the calibration sample identifier, the predicted values ​​of multiple measurement results corresponding to the same calibration sample, and the predicted values ​​of the sample corresponding to the calibration sample are read from the initial prediction characterization set; then, the predicted values ​​of multiple measurement results corresponding to the same calibration sample are organized into response characterization data, and the calibration sample identifier and its corresponding relationship are organized into disturbance characterization data.

[0041] S2.2 Construct an input disturbance processing task based on the disturbance characterization data, and perform input disturbance analysis to establish the input disturbance propagation relationship and form disturbance propagation data.

[0042] It should be noted that when constructing the input perturbation processing task based on the perturbation characterization data, the corresponding model input correlation features are first read according to the calibration sample identifier, and perturbations are applied to each model input correlation feature in sequence to form a feature group before perturbation and a feature group after perturbation. Then, the feature group before perturbation and the feature group after perturbation are respectively input into the measurement result prediction model. Each decision tree in the measurement result prediction model performs a layer-by-layer split judgment on the feature group before perturbation and the feature group after perturbation according to the node splitting condition, and outputs the predicted value of the measurement result before perturbation and the predicted value of the measurement result after perturbation respectively. Then, the predicted values ​​of the measurement results of the same calibration sample before and after perturbation are compared, the difference between the predicted values ​​of the corresponding measurement results before and after the perturbation of each model input correlation feature is calculated to obtain the change in the predicted value, and the transmission influence of each model input correlation feature on the predicted value of the measurement result is determined based on the change in the predicted value, the input perturbation transmission relationship is established, and perturbation transmission data is formed.

[0043] S2.3. Aggregate and characterize the disturbance propagation data to determine the sensitivity information of the input disturbance propagation relationship and form the propagation uncertainty.

[0044] It should be explained that, according to the calibration sample identifier, the predicted value changes corresponding to each model input correlation feature in the disturbance propagation data are read, and the predicted value changes corresponding to each model input correlation feature under the same calibration sample are centrally organized; then, the centrally organized predicted value changes are aggregated and calculated, and the predicted value changes corresponding to each model input correlation feature are synthesized into an aggregated result that can characterize the overall strength of the input disturbance propagation relationship, and the sensitivity information of the input disturbance propagation relationship is determined based on the aggregated result; the sensitivity information of the input disturbance propagation relationship is mapped to the propagation uncertainty of the same calibration sample, forming the propagation uncertainty that reflects the degree of influence of the input disturbance on the sample predicted value.

[0045] S2.4 Perform discrete correlation analysis on the discrete representation results of the model output in the response representation data, extract cognitive response information, perform discrete representation on the cognitive response information, and determine the prediction uncertainty.

[0046] It should be explained that, according to the calibration sample identifier, multiple predicted values ​​of measurement results corresponding to the same calibration sample are read, and these multiple predicted values ​​are organized accordingly to clarify the discrete distribution state formed by the multiple predicted values ​​of measurement results around the sample predicted value corresponding to the calibration sample. Then, discrete correlation analysis is performed on the degree of deviation and dispersion among the multiple predicted values ​​of measurement results to extract cognitive response information that can characterize the consistency of the output of the measurement result prediction model to the same calibration sample. After characterizing the discreteness of the cognitive response information, the deviation between the multiple predicted values ​​of measurement results corresponding to the same calibration sample and the sample predicted value corresponding to the calibration sample is read, and the multiple deviations are calculated in a centralized manner to obtain the quantitative result characterizing the dispersion strength of the multiple predicted values ​​of measurement results. The quantitative result is then directly mapped to the prediction uncertainty of the same calibration sample. The larger the quantitative result, the higher the prediction uncertainty; the smaller the quantitative result, the lower the prediction uncertainty.

[0047] S2.5 Align and encapsulate the propagation uncertainty and prediction uncertainty according to the correspondence of the same sample to form a dual-channel evaluation set.

[0048] It should be noted that, according to the calibration sample identifier, the propagation uncertainty and prediction uncertainty corresponding to each calibration sample are read separately, and sample-level matching is performed on the propagation uncertainty and prediction uncertainty to confirm the correspondence between the propagation uncertainty and prediction uncertainty corresponding to the same calibration sample. Then, the matched propagation uncertainty and prediction uncertainty are combined and organized according to a unified recording format, so that each calibration sample corresponds to a set of propagation uncertainty and prediction uncertainty. Finally, the combined and organized results are centrally packaged to form a dual-channel evaluation set, which can improve the completeness, pertinence and adaptability of the measurement result uncertainty characterization: on the one hand, by retaining the propagation uncertainty and prediction uncertainty separately, the characterization ambiguity problem caused by directly mixing uncertainty information from different sources can be avoided, so that input disturbance risk and model cognitive risk can be independently identified; on the other hand, in the subsequent interval construction, quantile threshold extraction, interval boundary correction and online sample risk update process, the two types of uncertainty information can be used together to perform more targeted processing on the sample prediction interval, so that the output uncertainty evaluation result is more in line with the actual needs of measurement result reliability evaluation under complex working conditions, and improves the reliability and robustness of measurement result evaluation.

[0049] It should also be noted that the dual-channel evaluation set refers to a sample-level joint evaluation data structure formed by simultaneously encapsulating propagation uncertainty and prediction uncertainty for the same calibration sample. When only propagation uncertainty is used, it is difficult to reflect the discrete state at the model output level; when only prediction uncertainty is used, it is difficult to reflect the degree of influence of input disturbances after they are propagated by the model. Therefore, encapsulating the two types of uncertainty in a unified manner according to the sample correspondence can provide a more complete data foundation for subsequent interval calibration and risk identification. Among them, propagation uncertainty is used to characterize the degree of propagation influence of disturbances in the model input correlation features on the sample prediction value, reflecting the sensitivity of input disturbances in the prediction process; prediction uncertainty is used to characterize the degree of dispersion among multiple measurement result prediction values ​​corresponding to the same calibration sample, reflecting the consistency level and cognitive fluctuation of the measurement result prediction model for the sample output.

[0050] S3. Based on the measurement prediction bias in the calibration samples, construct the basic non-consistency score, combine it with the prediction uncertainty weighted correction to form the uncertainty non-consistency score, extract the confidence interval threshold according to the confidence level quantile, and perform boundary correction on the prediction interval in the dual-channel evaluation set to form the interval calibration dataset.

[0051] S3.1 Match the sample predictions, propagation uncertainties, and prediction uncertainties in the dual-channel evaluation set with the actual measurement results in the calibration samples one by one according to the same sample identifier to form a calibration association dataset.

[0052] It should be noted that the sample predicted values, propagation uncertainties, and prediction uncertainties in the dual-channel evaluation set are extracted, and the actual measurement results in the calibration samples are read. Then, using the same sample identifier as the matching basis, the sample identifiers in the dual-channel evaluation set are compared one by one with the sample identifiers in the calibration samples. The sample predicted values, propagation uncertainties, prediction uncertainties, and actual measurement results with the same sample identifiers are merged into the same sample record. Sample records that fail to match or have missing items are removed. After completing the item-by-item matching of all sample records, a calibration association dataset is formed.

[0053] S3.2 Based on the calibration association dataset, calculate the measurement prediction deviation between the actual measurement results and the predicted values ​​of each calibration sample, and construct the basic inconsistency score for each measurement prediction deviation.

[0054] It should be noted that, based on the calibration association dataset, the actual measurement result and the sample prediction value corresponding to each calibration sample are first extracted, and the actual measurement result and the sample prediction value are subtracted sample by sample to obtain the measurement prediction deviation corresponding to each calibration sample. Then, the absolute value transformation of the measurement prediction deviation corresponding to each calibration sample is performed, so that the measurement prediction deviation corresponding to each calibration sample is transformed from a deviation value with direction into a basic non-consistency score that only represents the magnitude of the deviation, thereby completing the process of constructing basic non-consistency scores for each measurement prediction deviation.

[0055] S3.3 Read the prediction uncertainty of the sample corresponding to the basic inconsistency score, and perform sample-level weight mapping on the prediction uncertainty to form an uncertainty weight set.

[0056] It should be noted that the predicted uncertainty of the sample corresponding to the basic inconsistency score is read, and the predicted uncertainty of each calibration sample is arranged in ascending order. Then, a sample-by-sample weighting process is performed according to the ranking position of the predicted uncertainty of each calibration sample in the total predicted uncertainty, as shown below: ; in: It is the first Uncertainty weights corresponding to each calibration sample; It is the minimum weight value; It is the first The ranking position of the prediction uncertainty corresponding to each calibration sample after sorting the prediction uncertainties of all calibration samples in ascending order; This is the total number of calibration samples; It is the maximum weight value; in, This is the minimum weight value, used to represent the lower limit of the weight of samples with low prediction uncertainty in the weighted correction, and is determined based on the basic correction magnitude that low-risk samples still need to retain. This is the maximum weight value, representing the upper limit of the weight in the weighted correction for samples with high prediction uncertainty. It is determined based on the required increase in correction magnitude for high-risk samples. This allows the prediction uncertainties of samples ranked earlier to be mapped to smaller weights, and the prediction uncertainties of samples ranked later to be mapped to larger weights, thus transforming the prediction uncertainty of each calibration sample into a corresponding weight, forming an uncertainty weight set.

[0057] S3.4. The basic inconsistency score set is weighted and corrected based on the uncertainty weight set to form the uncertainty inconsistency score.

[0058] It should be noted that the basic non-consistency score corresponding to each calibration sample is matched with the uncertainty weight corresponding to each calibration sample item by item, and the basic non-consistency score corresponding to each calibration sample is weighted and calculated with the corresponding uncertainty weight to obtain the correction result corresponding to each calibration sample. The calibration sample with a larger uncertainty weight corresponds to a larger correction range of the basic non-consistency score, and the calibration sample with a smaller uncertainty weight corresponds to a smaller correction range of the basic non-consistency score, thus forming the uncertainty non-consistency score.

[0059] S3.5 Sort the inconsistent uncertainty scores and extract the scores at the corresponding quantile positions according to the confidence level to determine the confidence interval threshold.

[0060] It should be noted that when sorting the uncertainty inconsistency scores and extracting the scores at the corresponding quantile positions based on the confidence level to determine the confidence interval threshold, the uncertainty inconsistency scores corresponding to all calibration samples are first read. All uncertainty inconsistency scores are then compared item by item according to their numerical values ​​and arranged in ascending order to form an ordered sequence of uncertainty inconsistency scores. Next, the confidence level is read, and the corresponding quantile position is located in the ordered sequence of uncertainty inconsistency scores according to the coverage requirements corresponding to the confidence level. After locating the corresponding quantile position, the score at that quantile position is extracted as the confidence interval threshold. When the corresponding quantile position is between two adjacent sorted positions, the score at the later sorted position is extracted as the confidence interval threshold.

[0061] It should also be noted that the confidence level is a target coverage requirement set in advance during the interval calibration process, used to indicate the proportion of the actual measurement results that the corrected prediction interval needs to cover; The confidence level is the accuracy requirement, risk control requirement, or result usage requirement that is predetermined before the measurement task is calibrated in the interval. In the process of “sorting the uncertainty non-consistency scores and extracting the scores at the corresponding quantile positions according to the confidence level to determine the confidence interval threshold”, the “confidence level” is a known input condition used to determine which quantile position to extract the scores from the sorted “uncertainty non-consistency scores”.

[0062] S3.6 Read the sample predicted values, propagation uncertainty, and prediction uncertainty from the dual-channel evaluation set. Center on the sample predicted values ​​and construct the prediction intervals for each sample based on the propagation uncertainty and prediction uncertainty. Apply the confidence interval threshold to the upper and lower boundaries of each prediction interval, perform boundary correction processing, and output the interval correction result set.

[0063] It should be noted that after reading the sample predicted values, propagation uncertainties, and prediction uncertainties from the dual-channel evaluation set, the sample predicted values, propagation uncertainties, and prediction uncertainties in the same sample record are first organized to form the basic content for complete interval generation. Then, using the sample predicted value as the center position, the propagation uncertainties and prediction uncertainties are accumulated or weighted and synthesized to obtain the interval expansion amount of the corresponding sample. The interval expansion amount is then expanded to the upper and lower sides of the sample predicted value to form the initial prediction interval of the corresponding sample. Based on the upper boundary of the initial prediction interval, the confidence interval threshold is further expanded upwards, and based on the lower boundary of the initial prediction interval, the confidence interval threshold is further expanded downwards to complete the boundary correction process and form the interval correction result set.

[0064] S3.7. Perform sample-level encapsulation on the interval correction result set and the upper and lower boundaries of the corrected prediction intervals to output the interval calibration dataset.

[0065] It should be explained that the sample predicted values, the corrected upper boundary of the prediction interval, and the corrected lower boundary of the prediction interval in the interval correction result set are organized into corresponding records one by one, so that each record only retains the sample predicted value, the corrected upper boundary of the prediction interval, and the corrected lower boundary of the prediction interval corresponding to the same sample. The correspondence between the sample predicted value, the corrected upper boundary of the prediction interval, and the corrected lower boundary of the prediction interval is checked item by item. After the item-by-item check is completed, the sample predicted value, the corrected upper boundary of the prediction interval, and the corrected lower boundary of the prediction interval in the same record are merged according to a unified record format to form a complete single-sample interval calibration record. Then, all single-sample interval calibration records are summarized in sequence to form an interval calibration dataset.

[0066] S4. Identify boundary risk samples based on the distribution difference between online evaluation samples and training samples, perform interval expansion on the interval calibration dataset, and trigger a dynamic retest mechanism when the uncertainty after expansion is higher than the uncertainty threshold to update and correct the interval calibration dataset and output the uncertainty evaluation set.

[0067] S4.1 Read the input features in the training samples and perform statistical calculations on each input feature to obtain the distribution reference parameters corresponding to the training samples.

[0068] It should be noted that the input features in the training samples are organized by feature term to form the corresponding value set of each input feature in all training samples. For each input feature's corresponding value set, the central location, dispersion, and distribution range are statistically analyzed. The central location is used to characterize the common value level of the corresponding input feature in the training samples, the dispersion is used to characterize the fluctuation of the corresponding input feature in the training samples, and the distribution range is used to characterize the upper and lower distribution boundaries of the corresponding input feature in the training samples. The central location, dispersion, and distribution range corresponding to each input feature are then organized under the same input feature to form the distribution reference parameters corresponding to each input feature.

[0069] S4.2 Read the input features from the online evaluation samples, calculate the difference between each online evaluation sample and the distribution reference parameters, determine the distribution difference of each online evaluation sample, and form a distribution difference dataset.

[0070] It should be noted that the input features in the online evaluation samples are matched one by one with the distribution reference parameters corresponding to the training samples, and the deviation of each input feature value from the distribution reference parameter is calculated. Then, the deviation results corresponding to each input feature under the same online evaluation sample are summarized to determine the distribution difference corresponding to the same online evaluation sample. The distribution differences corresponding to each online evaluation sample are organized to form a distribution difference dataset.

[0071] S4.3. Determine the boundary risk judgment threshold based on the distribution difference dataset, and compare the distribution difference of each online assessment sample with the boundary risk judgment threshold to identify boundary risk samples.

[0072] It should be noted that the distribution differences at the front position are considered to be the difference level within or close to the training sample distribution range, and the distribution differences at the back position are considered to be the difference level that deviates significantly from the training sample distribution range. Then, the boundary distribution differences used to distinguish between the front and back difference levels are selected from the sorted distribution differences as the boundary risk judgment threshold. The distribution difference corresponding to each online assessment sample is compared with the boundary risk judgment threshold one by one. Online assessment samples with distribution differences less than or equal to the boundary risk judgment threshold are retained as non-boundary risk samples, and online assessment samples with distribution differences greater than the boundary risk judgment threshold are identified as boundary risk samples. All online assessment samples with distribution differences greater than the boundary risk judgment threshold are aggregated to obtain boundary risk samples.

[0073] S4.4 Read the prediction intervals corresponding to the boundary risk samples in the interval calibration dataset, and perform interval expansion processing on the corresponding prediction intervals according to the distribution differences of the boundary risk samples to form an interval expansion result set.

[0074] It should be explained that the boundary risk samples are matched one by one with the interval calibration records in the interval calibration dataset. The upper and lower boundaries of the prediction interval corresponding to each boundary risk sample are identified, and the distribution difference corresponding to each boundary risk sample and the corresponding prediction interval are placed in the same processing record. The distribution differences in the same processing record are compared, and the boundary risk samples with larger distribution differences are identified as boundary risk samples that require a larger interval expansion range, while the boundary risk samples with smaller distribution differences are identified as boundary risk samples that require a smaller interval expansion range. Using the upper and lower boundaries of the prediction interval as the expansion benchmark, the boundary length is increased outside the upper boundary of the prediction interval and outside the lower boundary of the prediction interval according to the interval expansion range corresponding to the distribution difference, so that the upper boundary of the prediction interval moves up and the lower boundary of the prediction interval moves down, forming the expanded upper and lower boundaries of the prediction interval. The boundary risk samples, the expanded upper and lower boundaries of the prediction interval are organized one by one to form the interval expansion result set.

[0075] It should also be noted that the interval expansion process is an adaptive expansion method designed to address the distributional deviation of boundary risk samples relative to training samples. Since the prediction results corresponding to boundary risk samples usually have higher uncertainty, if the original prediction interval is still used directly, it is easy to result in insufficient interval coverage or the actual measurement results falling outside the interval. Therefore, by expanding the prediction interval corresponding to boundary risk samples, the prediction interval can be matched with the actual risk level of the sample, thereby improving the coverage and reliability of the prediction interval for high-risk samples, and providing a processing basis for subsequent uncertainty judgment and dynamic retesting.

[0076] The advantage of interval expansion is that it can expand and correct the prediction interval according to the risk level based on the distribution shift of the boundary risk samples, thereby avoiding the problem of insufficient coverage caused by the original prediction interval being too narrow; at the same time, it does not require uniformly widening the interval for all samples, thus having good pertinence, robustness and adaptability.

[0077] S4.5 Calculate the post-expansion uncertainty of each boundary risk sample based on the interval expansion result set, compare the post-expansion uncertainty with the uncertainty threshold, identify the boundary risk samples with post-expansion uncertainty higher than the uncertainty threshold as retest trigger samples, and trigger the dynamic retest mechanism.

[0078] It should be noted that the boundary risk samples, the upper boundary of the expanded prediction interval, and the lower boundary of the expanded prediction interval in the interval expansion result set are sorted out one by one, and the interval width is calculated for the upper boundary and the lower boundary of the expanded prediction interval in the same record. The size of the interval between the upper boundary and the lower boundary of the expanded prediction interval is determined as the expansion uncertainty of the corresponding boundary risk sample. The expanded uncertainty of all boundary risk samples is compared with the uncertainty threshold one by one. Boundary risk samples with expanded uncertainty less than or equal to the uncertainty threshold are retained as non-retest trigger samples, and boundary risk samples with expanded uncertainty greater than the uncertainty threshold are identified as retest trigger samples. All retest trigger samples are centrally organized, and a retest trigger instruction is issued to the corresponding measurement process to trigger the dynamic retest mechanism.

[0079] It should also be noted that the dynamic retesting mechanism refers to a processing mechanism in which, when the expanded uncertainty corresponding to the boundary risk sample is higher than the uncertainty threshold, the retesting trigger sample is remeasured and the retesting measurement value is obtained. Then, based on the positional relationship between the retesting measurement value and the original prediction interval, the corresponding prediction interval is updated and corrected. The core content of the "dynamic retesting mechanism" includes retesting triggering, remeasurement, obtaining the retesting measurement value, and interval update and correction.

[0080] When the original prediction interval still has high uncertainty after interval expansion, the dynamic retesting mechanism further corrects the corresponding prediction interval by retesting the measured values, thereby reducing the uncertainty deviation of high-risk samples and improving the consistency and reliability between the corresponding prediction interval and the actual measurement results.

[0081] The dynamic retesting mechanism is formed by setting "the expanded uncertainty is higher than the uncertainty threshold" as the retesting trigger condition, "re-execute the measurement to obtain the retested measurement value" as the action to be executed after the trigger, and "update and correct the corresponding prediction interval according to the positional relationship between the retested measurement value and the original prediction interval" as the result correction rule after the retest. The retesting trigger condition, retesting action and retest result correction rule are sequentially linked together.

[0082] The uncertainty threshold is set by statistically analyzing the expanded uncertainty of the calibration sample or historical sample, sorting all the expanded uncertainties according to their numerical values, and then selecting the boundary value that can distinguish between the normal uncertainty range and the high-risk uncertainty range from the sorting results.

[0083] S4.6 Based on the dynamic retesting mechanism, the retesting triggered sample is re-measured to obtain the retesting measurement value, and the corresponding prediction interval is shifted and corrected according to the positional relationship between the retesting measurement value and the original prediction interval, and the uncertainty assessment set is output.

[0084] It should be noted that the measurement is re-executed for the retested trigger sample to obtain the retested measurement value corresponding to the retested trigger sample. Then, the position of the retested measurement value is compared with the original prediction interval corresponding to the retested trigger sample to determine whether the retested measurement value is inside, above, or below the original prediction interval. When the retested measurement value is inside the original prediction interval, the original prediction interval remains unchanged. When the retested measurement value is above the original prediction interval, the distance of the retested measurement value exceeding the upper boundary of the original prediction interval is used as the translation amount, and the original prediction interval is translated upwards as a whole. When the retested measurement value is below the original prediction interval, the distance of the retested measurement value below the lower boundary of the original prediction interval is used as the translation amount, and the original prediction interval is translated downwards as a whole. After completing the translation correction of the prediction interval corresponding to each retested trigger sample, the retested measurement values ​​and the corrected prediction intervals are compiled and summarized to output the uncertainty assessment set.

[0085] This embodiment also provides a measurement result uncertainty evaluation system based on machine learning, including: a modeling and prediction module, used to acquire measurement-related data and preprocess it, divide it into training samples, calibration samples and online evaluation samples, construct a measurement result prediction model based on the training samples and output an initial prediction representation set; The dual-channel module is used to extract sensitive information about the propagation relationship of input disturbances based on the initial prediction characterization set, determine the propagation uncertainty, and extract cognitive response information about the discreteness of the model output to determine the prediction uncertainty, thus forming a dual-channel evaluation set. The interval calibration module is used to construct a basic non-consistency score based on the measurement prediction deviation in the calibration sample, combine it with the prediction uncertainty weighted correction to form an uncertainty non-consistency score, extract the confidence interval threshold according to the confidence level quantile, and perform boundary correction on the prediction interval in the dual-channel evaluation set to form an interval calibration dataset. The update evaluation module is used to identify boundary risk samples based on the distribution difference between online evaluation samples and training samples. It performs interval expansion on the interval calibration dataset. When the uncertainty after expansion is higher than the uncertainty threshold, it triggers a dynamic retest mechanism to update and correct the interval calibration dataset and outputs an uncertainty evaluation set.

[0086] In summary, this invention achieves accurate characterization and interval calibration of measurement result uncertainty by constructing a processing flow that combines training, calibration, and online evaluation, and by conducting dual-channel evaluation of propagation uncertainty and prediction uncertainty, thereby improving the reliability of measurement results. At the same time, by performing interval expansion on boundary risk samples and triggering dynamic retest correction, adaptive correction under complex working conditions is achieved, thus resulting in more reliable and robust measurement results.

[0087] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A method for evaluating the uncertainty of measurement results based on machine learning, characterized in that, include: Acquire measurement-related data and preprocess it to divide it into training samples, calibration samples and online evaluation samples. Based on the training samples, construct a measurement result prediction model and output an initial prediction characterization set. Based on the initial prediction characterization set, sensitive information on the propagation relationship of input disturbances is extracted to determine the propagation uncertainty, and cognitive response information on the discreteness of the model output is extracted to determine the prediction uncertainty, forming a dual-channel evaluation set; Based on the measurement prediction bias in the calibration samples, a basic non-consistency score is constructed. Combined with the prediction uncertainty weighting correction, an uncertainty non-consistency score is formed. The confidence interval threshold is extracted according to the confidence level quantile. Boundary correction is applied to the prediction interval in the dual-channel evaluation set to form an interval calibration dataset. Based on the distribution difference between online evaluation samples and training samples, boundary risk samples are identified. The interval calibration dataset is expanded. When the uncertainty after expansion is higher than the uncertainty threshold, a dynamic retesting mechanism is triggered to update and correct the interval calibration dataset, and an uncertainty evaluation set is output.

2. The method for evaluating the uncertainty of measurement results based on machine learning as described in claim 1, characterized in that, The specific steps for acquiring measurement correlation data and preprocessing it to divide it into training samples, calibration samples, and online evaluation samples are as follows: Acquire measurement-related data, and sequentially perform outlier identification and removal, missing value completion, and data standardization on the measurement-related data to output a standardized basic dataset. Feature extraction and feature selection are performed on the standardized basic dataset, the relevant features of the model input are retained and the corresponding model learning feature space is constructed, and the preprocessed modeling sample set is output. Using the preprocessed modeling sample set as the processing object, the main samples under historical working conditions are divided into training samples according to the working condition stratification rules, the remaining samples under the working conditions corresponding to the training samples are divided into calibration samples, and the samples collected under subsequent working conditions are divided into online evaluation samples.

3. The method for evaluating the uncertainty of measurement results based on machine learning as described in claim 2, characterized in that, The specific steps for constructing a measurement result prediction model based on training samples and outputting an initial prediction representation set are as follows: Using the feature variables in the training samples as input and the corresponding measurement results as model output, the random forest regression model is fitted and trained to establish the feature-result correspondence, and the measurement result prediction model is obtained after ensemble calculation. The calibration samples are input into the measurement result prediction model. Each decision tree performs splitting judgment and regression calculation on the feature variables in the calibration samples. The initial prediction representation set is output through ensemble mean calculation.

4. The method for evaluating the uncertainty of measurement results based on machine learning as described in claim 3, characterized in that, The process involves extracting sensitivity information about the propagation relationship of input perturbations based on the initial prediction representation set, determining the propagation uncertainty, and extracting cognitive response information about the model output discreteness to determine the prediction uncertainty, thus forming a dual-channel evaluation set. The specific steps are as follows: The initial prediction characterization set is parsed and separated to extract disturbance characterization data and response characterization data; Based on the disturbance characterization data, an input disturbance processing task is constructed, and input disturbance analysis is performed to establish the input disturbance propagation relationship, thereby forming disturbance propagation data; Aggregate and characterize the disturbance propagation data to determine the sensitive information of the input disturbance propagation relationship and form the propagation uncertainty; Discrete correlation analysis is performed on the discrete representation results of the model output in the response representation data to extract cognitive response information, and the discreteness of the cognitive response information is characterized to determine the prediction uncertainty. The propagation uncertainty and prediction uncertainty are aligned and encapsulated according to the correspondence of the same sample to form a dual-channel evaluation set.

5. The method for evaluating the uncertainty of measurement results based on machine learning as described in claim 1, characterized in that, The specific steps for constructing the basic inconsistency score based on the measurement prediction bias in the calibration samples are as follows: The sample predictions, propagation uncertainties, and prediction uncertainties in the dual-channel evaluation set are matched one-to-one with the actual measurement results in the calibration samples according to the same sample identifier to form a calibration association dataset. Based on the calibration association dataset, the measurement prediction deviation between the actual measurement results and the predicted values ​​of each calibration sample is calculated, and each measurement prediction deviation is constructed as a basic inconsistency score.

6. The method for evaluating the uncertainty of measurement results based on machine learning as described in claim 1 or 4, characterized in that, The specific steps for forming the uncertainty inconsistency score by combining the weighted correction of prediction uncertainty are as follows: Read the prediction uncertainty of the sample corresponding to the basic non-consistent score, and perform sample-level weight mapping on the prediction uncertainty to form an uncertainty weight set; The uncertainty inconsistency score is formed by weighting and correcting the basic inconsistency score set based on the uncertainty weight set.

7. The method for evaluating the uncertainty of measurement results based on machine learning as described in claim 1 or 4, characterized in that, The steps for extracting confidence interval thresholds based on confidence level quantiles and applying boundary corrections to the prediction intervals in the dual-channel evaluation set to form an interval calibration dataset are as follows: The inconsistency scores of uncertainty are sorted, and the scores at the corresponding quantile positions are extracted according to the confidence level to determine the confidence interval threshold. Read the sample predicted values, propagation uncertainty, and prediction uncertainty from the dual-channel evaluation set. Center on the sample predicted values, construct the prediction interval for each sample based on the propagation uncertainty and prediction uncertainty, apply the confidence interval threshold to the upper and lower boundaries of each prediction interval, perform boundary correction processing, and output the interval correction result set. The interval correction result set, as well as the upper and lower boundaries of the corrected prediction interval, are encapsulated at the sample level to output the interval calibration dataset.

8. The method for evaluating the uncertainty of measurement results based on machine learning as described in claim 7, characterized in that, The specific steps for identifying boundary risk samples based on the distribution differences between online evaluation samples and training samples are as follows: Read the input features from the training samples, perform statistical calculations on each input feature, and obtain the distribution reference parameters corresponding to the training samples; Read the input features from the online evaluation samples, calculate the difference between each online evaluation sample and the distribution reference parameters, determine the distribution difference of each online evaluation sample, and form a distribution difference dataset; The boundary risk assessment threshold is determined based on the distribution difference dataset, and the distribution difference of each online assessment sample is compared with the boundary risk assessment threshold to identify boundary risk samples.

9. The method for evaluating the uncertainty of measurement results based on machine learning as described in claim 1, characterized in that, The specific steps for generating the output uncertainty evaluation set are as follows: Read the prediction intervals corresponding to the boundary risk samples in the interval calibration dataset, and perform interval expansion processing on the corresponding prediction intervals according to the distribution differences of the boundary risk samples to form an interval expansion result set; The expanded uncertainty of each boundary risk sample is calculated based on the interval expansion result set, and the expanded uncertainty is compared with the uncertainty threshold. Boundary risk samples with expanded uncertainty higher than the uncertainty threshold are identified as retest trigger samples, and the dynamic retest mechanism is triggered. Based on the dynamic retesting mechanism, the retesting trigger sample is re-measured to obtain the retesting measurement value, and the corresponding prediction interval is shifted and corrected according to the positional relationship between the retesting measurement value and the original prediction interval, and the uncertainty assessment set is output.

10. A machine learning-based measurement result uncertainty evaluation system, based on the machine learning-based measurement result uncertainty evaluation method according to any one of claims 1 to 9, characterized in that, include: The modeling and prediction module is used to acquire and preprocess measurement-related data, divide it into training samples, calibration samples and online evaluation samples, build a measurement result prediction model based on the training samples and output an initial prediction representation set; The dual-channel module is used to extract sensitive information about the propagation relationship of input disturbances based on the initial prediction characterization set, determine the propagation uncertainty, and extract cognitive response information about the discreteness of the model output to determine the prediction uncertainty, thus forming a dual-channel evaluation set. The interval calibration module is used to construct a basic non-consistency score based on the measurement prediction deviation in the calibration sample, combine it with the prediction uncertainty weighted correction to form an uncertainty non-consistency score, extract the confidence interval threshold according to the confidence level quantile, and perform boundary correction on the prediction interval in the dual-channel evaluation set to form an interval calibration dataset. The update evaluation module is used to identify boundary risk samples based on the distribution difference between online evaluation samples and training samples. It performs interval expansion on the interval calibration dataset. When the uncertainty after expansion is higher than the uncertainty threshold, it triggers a dynamic retest mechanism to update and correct the interval calibration dataset and outputs an uncertainty evaluation set.

Images