Dynamic outlier bias reduction system and method
A computer-implemented method iteratively reduces outlier bias by using error thresholds and optimization models to enhance the accuracy and reliability of data analysis, addressing the subjectivity and bias in existing outlier removal methods.
Patent Information
- Application Number
- JP2025133286
- Authority / Receiving Office
- JP · JP
- Patent Type
- Patents
- Current Assignee / Owner
- Priority Date
- 2013-02-20
- Filing Date
- 2025-08-08
- Publication Date
- 2026-11-19
- Estimated Expiration
- 2034-02-20
AI Technical Summary
Existing methods for removing outlier data in data-driven model development are subjective and can introduce bias, affecting the fairness and representativeness of benchmarks, particularly in greenhouse gas emissions analysis.
A computer-implemented method involving a series of iterative steps to objectively identify and reduce outlier bias using error thresholds and optimization models, ensuring that only data within specified error limits is included in model calculations.
This approach ensures that model predictions are unbiased and representative by iteratively refining model coefficients to minimize errors, improving the accuracy and reliability of data analysis.
Smart Images

Figure 0007954252000009 
Figure 0007954252000010 
Figure 0007954252000011
Abstract
Description
[Technical Field]
[0001] This invention relates to data analysis in which outlier components are removed (or filtered) from the analysis development. The analysis relates to simple statistical calculations or complex operations involving mathematical models that use the data in the development. The filtering of outlier data is for the purpose of data quality and data authentication operations, or for the purpose of calculating representative standards, statistics, data sets to be applied to subsequent analyses, regression analysis, time series analysis, or qualified data for mathematical model development.
[0002] Cross-reference of related applications This continuation-in-part patent application claims priority to U.S. Provisional Patent Application No. 13 / 213,780, entitled "Dynamic Outlier Bias Reduction System and Method," filed on 19 August 2011, which is incorporated herein by reference in its entirety. [Background technology]
[0003] Removing outlier data in criteria or data-driven model development is an important part of preliminary analytical work to ensure that a representative and fair analysis is developed from the underlying data. For example, carbon dioxide (CO2), ozone (O3), water vapor (H2O), hydrofluorocarbons (HFCs), perfluorocarbons (PFCs), chlorofluorocarbons (CFCs), sulfur hexafluoride (SF6), methane (CH4), nitrous oxide (N2O), carbon monoxide (CO), nitrogen oxides (NOx). xTo develop fair benchmarks for greenhouse gas emissions of ) and non-methane volatile organic compounds (NMVOCs), the collected industrial data used in benchmark development must exhibit certain characteristics. Extremely good or bad performance by a few industrial zones should not bias the benchmark calculated for other industrial zones. Including such performance results in the benchmark calculation would be considered unfair or non-representative. In the past, performance outliers were removed through semi-quantitative processes that required subjective input. The current system and method are data-driven. This means that the task is performed as an integral part of model development, rather than in a preliminary analysis or preliminary model development stage.
[0004] The removal of bias can be a subjective process in which justifications for data modification are documented in a prescribed format. However, all forms of outlier removal involve data censoring that may alter the calculation results. Such data filtering may or may not reduce bias or error in the calculation, and in the spirit of full analytical disclosure, strict data removal guidelines and documentation for removing outliers should be included in the analytical results. Therefore, there is a need in the industry to provide new systems and methods for objectively removing outlier data bias using dynamic statistical processes useful for data quality manipulation, data authentication, statistical calculations, or mathematical model development. Outlier removal systems and methods can also be used to classify data into representative categories. In this case, the data is applied to the development of mathematical models customized for each group. In a preferred embodiment, coefficients are defined as multiplicative and additive factors of the mathematical model, and as other, essentially nonlinear numerical parameters. For example, f(x,y,z)=a*x+b*y c In the mathematical model of +d*sin(ez)+f, a, b, c, d, e, and f are all defined as coefficients. The values of these terms are either fixed or become part of the development of the mathematical model. [Prior art documents] [Patent Documents]
[0005] [Patent Document 1] International Publication No. 2007 / 117233(A1) Pamphlet [Overview of the Initiative]
[0006] A preferred embodiment is a computer implementation method for reducing outlier bias, comprising the following steps: (1) selecting a bias criterion; (2) providing a dataset; (3) providing a set of multiple model coefficients; (4) generating a set of multiple predicted values for the completed dataset; (5) generating an error set for the dataset; (6) generating a set of multiple error thresholds based on the error set and the bias criterion; (7) having the processor generate a truncated dataset based on the error set and the set of multiple error thresholds; (8) having the processor generate a set of multiple new model coefficients; and (9) repeating steps (1) to (1) using the set of multiple new model coefficients until a truncated performance termination criterion is met. In a preferred embodiment, the set of multiple predicted values is generated based on the dataset and the set of multiple model coefficients. In a preferred embodiment, the error set includes a set of multiple absolute errors and a set of multiple relative errors generated based on the set of multiple predicted values and the set of multiple target values. In other embodiments, the error set includes a set of values calculated as the difference between a set of multiple predicted values and a set of multiple target values. In other embodiments, the step of generating a set of multiple new coefficients further includes minimizing a set of multiple errors between a set of multiple predicted values and a set of multiple actual values, which can be achieved using a linear or nonlinear optimization model. In a preferred embodiment, the censorship performance termination criterion is based on one standard error and one coefficient of determination.
[0007] Other embodiments include a computer-implemented method for reducing outlier bias. The method includes the following steps: selecting an error criterion, selecting a dataset, selecting a set of a plurality of actual values, selecting an initial set of a plurality of model coefficients, generating a set of a plurality of model predicted values based on the completed dataset and the initial set of the plurality of model coefficients, (1) generating a set of a plurality of errors based on the model predicted values for the completed dataset and the set of the plurality of actual values, (2) generating a set of a plurality of error thresholds based on the completed set of the plurality of errors for the completed dataset and the error criterion, (3) generating an outlier-removed dataset, wherein the filtering is based on the completed dataset and the set of the plurality of error thresholds, (4) generating a set of a plurality of new coefficients based on the filtered dataset and the set of a plurality of old coefficients, wherein the generation of the set of the plurality of new coefficients is performed by a computer processor, (5) generating a set of a plurality of outlier bias-reduced model predicted values based on the filtered dataset and the set of a plurality of new model coefficients, wherein the generation of the set of the plurality of outlier bias-reduced model predicted values is performed by the computer processor, (6) generating a set of a plurality of model performance values based on the model predicted values and the set of the plurality of actual values, repeating steps (1) to (6) until a performance termination criterion is satisfied while using the set of the plurality of new coefficients in place of the set of the plurality of coefficients from the previous iteration, and storing the set of the plurality of model predicted values on a computer data medium.
[0008] Other embodiments include a computer implementation method for reducing outlier bias. This method includes the following steps: selecting a target variable for a facility; selecting a set of multiple actual values for the target variable; identifying multiple variables for the facility related to the target variable; obtaining a dataset for the facility, the dataset containing multiple values for the multiple variables; selecting a bias criterion; selecting a set of multiple model coefficients; (1) generating a set of multiple predicted values based on the completed dataset and the set of multiple model coefficients; (2) generating a set of multiple censored model performance values based on the set of multiple predicted values and the set of multiple actual values; (3) generating a set of errors based on the set of multiple predicted values and the set of multiple actual values for the target variable; and (4) generating a set of errors based on the set of errors and the bias criterion. The steps are: (5) generating a set of multiple error thresholds; (6) the processor generating a truncated dataset based on the dataset and the set of multiple error thresholds; (7) the processor generating a set of multiple new model coefficients based on the truncated dataset and the set of multiple model coefficients; (8) the processor generating a set of multiple new predicted values based on the dataset and the set of multiple new model coefficients; (9) generating a set of multiple new truncated model performance values based on the set of multiple new predicted values and the set of multiple actual values; repeating steps (1) to (8) using the set of multiple new coefficients until a truncated performance termination criterion is met; and storing the set of multiple new model predicted values in a computer data medium.
[0009] Other embodiments include a computer-implemented method for reducing outlier bias. This method includes the following steps. Namely, determining a target variable for a facility, wherein the target variable is a metric related to manufacturing, financial performance, or emissions for an industrial facility; identifying a plurality of variables for the facility, wherein the plurality of variables are a plurality of direct variables for the facility that affect the target variable, and a set of a plurality of transformed variables for the facility each being a function of at least one direct facility variable that affects the target variable; selecting an error criterion including an absolute error and a relative error; obtaining a dataset for the facility, wherein the dataset includes a plurality of values for the plurality of variables; selecting a set of a plurality of actual values of the target variable; selecting an initial set of a plurality of model coefficients; generating a set of a plurality of model prediction values based on the completed dataset and the initial set of the plurality of model coefficients; generating a completed set of a plurality of errors based on the completed set of the plurality of model prediction values and the set of the plurality of actual values, wherein the relative error is expressed by the formula: relative error m =((predicted value m - actual value m ) / actual value m ) 2 (where "m" is a reference number), and the absolute error is expressed by the formula: absolute error m =(predicted value m - actual value m ) 2Steps calculated using: (1) generating a set of multiple model performance values based on the set of multiple model predictions and the set of multiple actual values, wherein the set of multiple model performance values overall includes a first standard error and a first coefficient of determination; (2) generating a set of multiple errors based on the set of multiple model predictions and the set of multiple actual values for the completed dataset; (3) generating a set of multiple error thresholds based on the completed set of multiple errors and the error criteria for the completed dataset; (4) generating a set of outlier-removed dataset by removing data with error values greater than or equal to the error threshold, wherein the filtering is based on the completed dataset and the set of multiple error thresholds; (5) minimizing the error between the set of multiple predictions and the set of multiple actual values using at least one of a linear optimization model and a nonlinear optimization model. (1) to (2) a step of generating a set of multiple outlier-bias-reduced model predictions based on the outlier-removed dataset and the set of multiple model coefficients, wherein the generation of the new model predictions is performed by a computer processor; (3) a step of generating a set of multiple new coefficients based on the outlier-removed dataset and the old set of multiple coefficients, wherein the generation of the set of multiple new coefficients is performed by a computer processor; (4) a step of generating a set of multiple overall model performance values based on the set of multiple new predicted model values and the set of multiple actual values, wherein the set of multiple model performance values includes a second standard error and a second coefficient of determination; and repeating steps (1) to (6) until a performance termination criterion is met, using the set of multiple new coefficients instead of the set of multiple coefficients from the previous iteration, wherein the performance termination criterion includes a standard error termination value and a coefficient of determination termination value,Satisfying the performance termination criterion includes that the standard error termination value is greater than the difference between the first and second standard errors, and the coefficient of determination termination value is greater than the difference between the first and second coefficients of determination, and storing the set of a plurality of new model prediction values in a computer data medium.
[0010] Other embodiments include a computer-implemented method for reducing outlier bias. The method includes the following steps: selecting an error criterion; selecting a data set; selecting a set of a plurality of actual values; selecting an initial set of a plurality of model prediction values; determining a set of a plurality of errors based on the set of the plurality of model prediction values and the set of the plurality of actual values; (1) determining a set of a plurality of error thresholds based on the completed set of the plurality of errors and the error criterion; (2) generating an outlier-removed data set, wherein the filtering is based on the data set and the set of the plurality of error thresholds; (3) generating a set of a plurality of outlier bias-reduced model prediction values based on the outlier-removed data set and a plurality of old model prediction values, wherein the generation of the set of the plurality of outlier bias-reduced model prediction values is performed by a computer processor; (4) determining a set of a plurality of errors based on the set of the plurality of new model prediction values and the set of the plurality of actual values; repeating steps (1) to (4) using the plurality of new model prediction values in place of the set of the plurality of model prediction values from the previous iteration until a performance termination criterion is satisfied; and storing the set of the plurality of outlier bias-reduced model prediction values in a computer data medium.
[0011] Other embodiments include a computer implementation method for reducing outlier bias. This method includes the following steps: determining a target variable for a facility; identifying a plurality of variables for the facility, wherein the plurality of variables are a plurality of direct variables for the facility that affect the target variable and a set of a plurality of transformed variables for the facility, each of which is a function of at least one direct facility variable that affects the target variable; selecting a single error criterion including a single absolute error and a single relative error; obtaining a single dataset including a plurality of values for the plurality of variables; selecting a single set of a plurality of actual values for the target variable; selecting a single initial set of a plurality of model coefficients; generating a single set of a plurality of model predictions by applying the single set of model coefficients to the dataset; determining a single set of a plurality of performance values based on the set of a plurality of model predictions and the set of a plurality of actual values, wherein the set of a plurality of performance values includes a first standard error and a first coefficient of determination; (1) generating a single set of errors based on the set of a plurality of model predictions and the set of a plurality of actual values for the completed dataset, wherein the relative error is given by the formula: Relative Error m =((Predicted value m - Actual value m ) / actual value m ) 2 It is calculated using (where "m" is the reference number), and the absolute error is given by the formula: absolute error m =(predicted value) m - Actual value m ) 2(1) a step calculated using (2) a step of generating a set of multiple error thresholds based on the completed set of multiple errors for the completed dataset and the error criteria; (3) a step of generating an outlier-removed dataset by removing data having more than one error value than the set of multiple error thresholds, wherein the filtering is based on the dataset and the set of multiple error thresholds; (4) a step of generating a set of multiple new coefficients based on the outlier-removed dataset and the set of multiple old coefficients; (5) a step of generating a set of multiple outlier-bias-reduced model predictions based on the outlier-removed dataset and the set of multiple new model coefficients by minimizing the error between the set of multiple predictions and the set of multiple actuals using at least one of a linear optimization model and a nonlinear optimization model; (1) to (2) generate a set of updated performance values based on the set of outlier bias-reduced model predicted values and the set of actual values, wherein the set of updated performance values includes a second standard error and a second coefficient of determination; (3) repeat steps (1) to (2) until a performance termination criterion is met, using the set of new coefficients instead of the set of coefficients from previous iterations, wherein the performance termination criterion includes a standard error termination value and a coefficient of determination termination value, and satisfying the performance termination criterion includes the standard error termination value being greater than the difference between the first and second standard errors and the coefficient of determination termination value being greater than the difference between the first and second coefficients of determination; and (4) store the set of outlier bias reduction factors in a computer data medium.
[0012] Other embodiments include a computer implementation method for evaluating the feasibility of a dataset used when developing a model. This method includes the steps of: providing a target dataset containing a plurality of data values; generating a random target dataset based on the target dataset; selecting a set of a plurality of bias criteria; having the processor generate a single outlier-bias-reduced target dataset based on the dataset and each of the selected bias criteria; having the processor generate a single outlier-bias-reduced random dataset based on a random dataset and each of the selected bias criteria; calculating a set of a plurality of error values for the outlier-bias-reduced dataset and the outlier-bias-reduced random dataset; calculating a set of a plurality of correlation coefficients for the outlier-bias-reduced dataset and the outlier-bias-reduced random dataset; generating a plurality of bias criteria curves for the dataset and the random dataset based on the selected plurality of bias criteria and the corresponding error values and correlation coefficients; and comparing the bias criteria curve for the dataset with the bias criteria curve for the random dataset. The outlier-bias-reduced target dataset and the outlier-bias-reduced random target dataset are generated using a dynamic outlier removal method. The random target dataset may consist of multiple arbitrarily selected data values expanded from multiple values within a range of multiple data values. The set of multiple error values may include a set of multiple standard errors. Here, the set of multiple correlation coefficients includes a set of multiple coefficient of determination values. Another embodiment further includes the step of generating automated advice on the feasibility of the target dataset supporting the expanded model and inversely, based on a comparison of the biased baseline curve for the target dataset and the biased baseline curve for the random target dataset. The advice may be generated based on parameters selected by the analyst, such as correlation coefficient thresholds and / or error thresholds. Yet another embodiment further includes the following steps:Specifically, the steps are: providing a real dataset containing multiple real data values corresponding to the model's predicted values; generating a random real dataset based on the real dataset; the processor generating an outlier-bias-reduced real dataset based on the real dataset and each selected bias criterion; the processor generating an outlier-bias-reduced random real dataset based on the random real dataset and each selected bias criterion; generating a random data plot for each selected bias criterion based on the outlier-bias-reduced random target dataset and the outlier-bias-reduced random real data; generating a realistic data plot for each selected bias criterion based on the outlier-bias-reduced target dataset and the outlier-bias-reduced real target dataset; and comparing the random data plot with the realistic data plot corresponding to each selected bias criterion.
[0013] A preferred embodiment includes a system comprising a server including a processor and a storage subsystem, a database including a dataset and stored by the storage subsystem, and a computer program stored by the storage subsystem. The computer program includes instructions that, when executed, cause the processor to: select a bias criterion; provide a set of multiple model coefficients; select a set of multiple target values; (1) generate a set of multiple predicted values for the dataset; (2) generate a set of errors for the dataset; (3) generate a set of multiple error thresholds based on the error set and the bias criterion; (4) generate a truncated dataset based on the error set and the set of error thresholds; (5) generate a set of multiple new model coefficients; and (6) repeat steps (1) to (5) using the set of multiple new model coefficients until a truncated performance termination criterion is met. In a preferred embodiment, the set of multiple predicted values is generated based on the dataset and the set of multiple model coefficients. In a preferred embodiment, the error set includes a set of multiple absolute errors and a set of multiple relative errors generated based on the set of multiple predicted values and the set of multiple target values. In another embodiment, the error set includes values calculated as the difference between the set of multiple predicted values and the set of multiple target values. In another embodiment, the step of generating the set of multiple new coefficients further includes the step of minimizing the set of multiple errors between the set of multiple predicted values and the set of multiple actual values. This can be achieved using a linear or nonlinear optimization model. In a preferred embodiment, the censorship performance termination criterion is based on one standard error and one coefficient of determination.
[0014] Another embodiment of the present invention includes a system comprising a server including a processor and a storage subsystem, a database including a dataset and stored by the storage subsystem, and a computer program stored by the storage subsystem. The computer program includes instructions that, when executed, cause the processor to: (1) select an error criterion; (2) generate a set of multiple model predictions based on the set of multiple actual values for the completed dataset; (3) generate a set of multiple errors based on the set of multiple actual values for the completed dataset; (4) generate a set of multiple error thresholds based on the set of multiple errors and the error criterion for the completed dataset; (5) generate an outlier-removed dataset, the filtering of which is based on the set of multiple error thresholds for the completed dataset; (6) generate a set of outlier-bias-reduced model values based on the set of multiple coefficients. (5) Generate a set of outlier-reduced model predictions, the generation of which set of multiple outlier-reduced model predictions is performed by a computer processor; (6) Generate a set of multiple new coefficients based on the outlier-removed dataset and the set of multiple old coefficients, the generation of which set of multiple new coefficients is performed by the computer processor; (7) Generate a set of multiple model performance values based on the outlier-reduced model predictions and the set of multiple actual values, repeating steps (1) to (6) until a performance termination criterion is met, using the set of multiple new coefficients instead of the set of multiple coefficients from previous iterations; and storing the overall set of multiple outlier-reduced model predictions in a computer data medium.
[0015] Further embodiments include a system comprising a server including a processor and a storage subsystem, a database stored by the storage subsystem which includes a target variable for a facility, a set of multiple actual values for the target variable, multiple variables for the facility related to the target variable, and a set of values for the multiple variables, and a computer program stored by the storage subsystem. The computer program includes instructions that, when executed, cause the processor to: select a bias criterion; select a set of multiple model coefficients; (1) generate a set of multiple predicted values based on the set of the set of model coefficients and the set of multiple predicted values; (2) generate a set of multiple censored model performance values based on the set of multiple predicted values and the set of multiple actual values for the target variable; (3) generate a set of errors based on the set of multiple predicted values and the set of multiple actual values for the target variable; (4) generate a set of multiple error thresholds based on the set of errors and the bias criterion; (5) generate a set of censored error thresholds based on the set of the set of error thresholds and the set of multiple error thresholds (6) generate a censored dataset; (7) generate a set of multiple new model coefficients based on the censored dataset and the set of multiple model coefficients; (8) generate a set of multiple new predicted values based on the dataset and the set of multiple new model coefficients; (9) generate a set of multiple new censored model performance values based on the set of multiple new predicted values and the set of multiple actual values; repeat steps (1) to (8) using the set of multiple new coefficients until a censored performance termination criterion is met; and store the set of multiple new model predicted values in the storage subsystem.
[0016] Other embodiments include a system comprising a server including a processor and a storage subsystem, a database including a dataset for a facility and stored by the storage subsystem, and a computer program stored by the storage subsystem. The computer program includes instructions that, when executed, cause the processor to: determine a target variable; identify a plurality of variables, the plurality of variables being a plurality of direct variables for the facility that affect the target variable and a set of a plurality of transformed variables for the facility, each being a function of at least one direct variable that affects the target variable; select a single error criterion including a single absolute error and a single relative error; select a set of a plurality of actual values for the target variable; select a single initial set of a plurality of coefficients; generate a set of a plurality of model predictions from the dataset and the single initial set of coefficients; and determine a set of a plurality of errors based on the set of the plurality of model predictions and the set of the plurality of actual values, wherein the relative error is given by the formula: Relative Error m =((Predicted value m - Actual value m ) / actual value m ) 2 It is calculated using (where "m" is the reference number), and the absolute error is given by the formula: absolute error m =(predicted value) m - Actual value m ) 2(1) to generate a set of errors based on the set of model predictions and the set of actual values, the set of performance values including the first standard error and the first coefficient of determination, (2) to generate a set of error thresholds based on the completed set of errors and the error criteria for the completed dataset, (3) to generate a set of outlier-removed dataset by filtering out data having multiple error values outside the set of error thresholds, the filtering being based on the dataset and the set of error thresholds, (4) to generate a set of new model predictions based on the set of outlier-removed dataset and the set of coefficients by minimizing a single error between the set of predictions and the set of actual values using at least one of a linear optimization model and a nonlinear optimization model, the outlier bias (5) generating a set of multiple new coefficients based on the outlier-removed dataset and the set of multiple old coefficients, the generation of the set of multiple new coefficients is performed by the computer processor, (6) generating a set of multiple performance values based on the set of multiple new model predictions and the set of multiple actual values, the set of multiple model performance values including a second standard error and a second coefficient of determination, repeating steps (1) to (6) until a performance termination criterion is met, using the set of multiple new coefficients instead of the set of multiple coefficients from the previous iteration, the performance termination criterion including a standard error termination value and a coefficient of determination, and satisfying the performance termination criterion including the standard error termination value being greater than the difference between the first and second standard errors and the coefficient of determination termination value being greater than the difference between the first and second coefficients of determination, and storing the set of multiple new model predictions in a computer data medium.
[0017] Another embodiment of the present invention includes a server including a processor and a storage subsystem, a database including a dataset and stored by the storage subsystem, and a computer program stored by the storage subsystem. The computer program includes instructions that, when executed, cause the processor to do the following: In other words, (1) select an error criterion, select a dataset, select a set of multiple actual values, select an initial set of multiple model predictions, determine a set of multiple errors based on the set of multiple model predictions and the set of multiple actual values, (2) determine a set of multiple error thresholds based on the completed set of multiple errors and the error criterion, (3) generate an outlier-removed dataset, the filtering of which is based on the dataset and the set of multiple error thresholds, (4) generate a set of multiple outlier-bias-reduced model predictions based on the outlier-removed dataset and the completed set of multiple model predictions, the generation of the set of multiple outlier-bias-reduced model predictions is performed by a computer processor, (5) determine a set of multiple errors based on the set of multiple outlier-bias-reduced model predictions and the corresponding set of multiple actual values, repeat steps (1) to (4) until a performance termination criterion is met, using the set of multiple outlier-bias-reduced model predictions instead of the set of multiple model predictions, and store the set of multiple outlier-bias-reducing factors in a computer data medium.
[0018] Another embodiment of the present invention includes a server including a processor and a storage subsystem, a database including a dataset and stored by the storage subsystem, and a computer program stored by the storage subsystem. The computer program includes instructions that, when executed, cause the processor to do the following: That is, determining one target variable, identifying multiple variables for the facility, wherein the multiple variables are multiple direct variables for the facility that affect the target variable and a set of multiple transformed variables for the facility, each of which is a function of at least one major facility variable that affects the target variable, selecting one error criterion including one absolute error and one relative error, obtaining a set of multiple actual values for the target variable by obtaining a set of multiple data sets including multiple values for the multiple variables, selecting one initial set of multiple coefficients, generating a set of multiple model predictions by applying the set of multiple model coefficients to the data sets, determining a set of multiple performance values based on the set of multiple model predictions and the set of multiple actual values, wherein the set of multiple performance values includes the first standard error and the first coefficient of determination, (1) determining a set of multiple errors based on the set of multiple model predictions and the set of multiple actual values, wherein the relative error is given by the formula: Relative Error k =((Predicted value k - Actual value k ) / actual value k ) 2 It is calculated using (where "k" is the reference number), and the absolute error is given by the formula: absolute error k =(predicted value) k - Actual value k ) 2(2) to calculate using the following: (3) to determine a set of multiple error thresholds based on the set of multiple errors and the error criteria for the completed dataset; (4) to generate a set of multiple new coefficients based on the set of multiple old coefficients, (5) to minimize a single error between the set of multiple predicted values and the set of multiple actual values using at least one of a linear optimization model and a nonlinear optimization model, and to generate a set of multiple outlier-bias-reduced model predicted values based on the set of multiple coefficients. (5) generating a set of multiple updated performance values based on the set of multiple outlier bias-reduced model prediction values and the set of multiple actual values, wherein the set of updated performance values includes a second standard error and a second coefficient of determination; repeating steps (1) to (5) until a performance termination criterion is met, using a set of multiple new coefficients instead of the set of multiple coefficients from previous iterations, wherein the performance termination criterion includes a standard error termination value and a coefficient of determination termination value, and satisfying the performance termination criterion includes the standard error termination value being greater than the difference between the first and second standard errors and the coefficient of determination termination value being greater than the difference between the first and second coefficients of determination; and storing the set of multiple outlier bias reduction factors in a computer data medium.
[0019] Further embodiments include a system for evaluating the feasibility of a dataset used when deploying a model. The system includes a server including a processor and a storage subsystem, a database including a target dataset containing multiple model predictions and stored by the storage subsystem, and a computer program stored by the storage subsystem. The computer program includes instructions that, when executed, cause the processor to do the following: Specifically, this involves generating a random target dataset, selecting a set of bias criteria, generating multiple outlier-bias-reduced datasets based on the target dataset and each selected bias criterion, generating a randomly-bias-reduced outlier-bias-reduced target dataset based on the random target dataset and each selected bias criterion, calculating a set of error values for the outlier-bias-reduced target dataset and the outlier-bias-reduced random target dataset, calculating a set of correlation coefficients for the outlier-bias-reduced target dataset and the outlier-bias-reduced random target dataset, generating multiple bias criterion curves for the target dataset and the random target dataset based on the corresponding error values and correlation coefficients for each selected bias criterion, and comparing the bias criterion curve for the target dataset with the bias criterion curve for the random target dataset. The processor generates the outlier-bias-reduced target dataset and the outlier-bias-reduced random target dataset using a dynamic outlier removal method. The random target dataset may consist of multiple arbitrarily selected data values expanded from multiple values within the range of the multiple data values. Furthermore, the set of multiple error values may include a set of multiple standard errors. The set of multiple correlation coefficients may include a set of multiple coefficient of determination values. In other embodiments, the program further includes instructions that, when executed, cause the processor to generate automated advice based on a comparison between the biased baseline curve for the target dataset and the biased baseline curve for the random target dataset.The advice can be generated based on parameters selected by the analyst, such as correlation thresholds and / or error thresholds. In yet another embodiment, the system's database further includes a real dataset comprising a plurality of real data values corresponding to the model's predicted values, and the program further includes instructions that, when executed, cause the processor to: generate a random real dataset based on the real dataset; generate an outlier-bias-reduced real dataset based on the real dataset and each selected bias criterion; generate an outlier-bias-reduced random real dataset based on the random real dataset and each selected bias criterion; generate a random data plot for each selected bias criterion based on the outlier-bias-reduced random target dataset and the outlier-bias-reduced random real data; generate a real data plot for each selected bias criterion based on the outlier-bias-reduced target dataset and the outlier-bias-reduced real target dataset; and compare the random data plot with the real data plot corresponding to each selected bias criterion.
[0020] Other embodiments include a system for reducing outlier bias in a target variable measured for a facility, and including a computer unit for processing a dataset. The computer unit includes a processor and a storage subsystem, an input unit which receives the dataset to be processed and includes a measuring device which measures a given target variable and provides a corresponding dataset, an output unit which outputs a processed dataset, and a computer program stored by the storage subsystem. The computer program includes instructions that, when executed, cause the processor to perform the following steps: That is, the steps are: selecting a target variable for a facility; identifying multiple variables for the facility related to the target variable; obtaining a dataset for the facility, the dataset containing multiple values for the multiple variables; selecting a bias criterion; selecting a set of multiple model coefficients; (1) generating a set of multiple predicted values for the dataset; (2) generating a set of errors for the dataset; (3) generating a set of multiple error thresholds based on the error set and the bias criterion; (4) generating a censored dataset based on the set of errors and the set of error thresholds; (5) generating a set of multiple new model coefficients; and (6) repeating steps (1) to (5) using the set of multiple new model coefficients until a censored performance termination criterion is met.
[0021] Furthermore, other embodiments include a system for reducing outlier bias in a target variable measured for financial instruments such as equity securities (e.g., common stock) or derivative contracts (e.g., forwards, futures, options, swaps, etc.). The system includes a computer unit for processing a dataset, comprising a processor and a storage subsystem; an input unit for receiving the dataset to be processed, comprising a storage device for storing data about a target variable (e.g., stock price) and providing a corresponding dataset; an output unit for outputting the processed dataset; and a computer program stored by the storage subsystem. The computer program, when executed, includes instructions that cause the processor to perform the following steps: That is, the steps are: selecting a target variable for the financial instrument; identifying multiple variables for the instrument related to the target variable (e.g., dividends, earnings, cash flow, etc.); obtaining a dataset for the financial instrument, the dataset containing multiple values for the multiple variables; selecting a bias criterion; selecting a set of multiple model coefficients; (1) generating a set of multiple predicted values for the dataset; (2) generating a set of errors for the dataset; (3) generating a set of multiple error thresholds based on the error set and the bias criterion; (4) generating a censored dataset based on the error set and the set of error thresholds; (5) generating a set of multiple new model coefficients; and (6) repeating steps (1) to (5) using the set of multiple new model coefficients until a censored performance termination criterion is met. [Brief explanation of the drawing]
[0022] [Figure 1] This flowchart illustrates one embodiment of a method for identifying and removing outliers in the data. [Figure 2]This flowchart illustrates one embodiment of a method for identifying and removing data outliers for data quality manipulation. [Figure 3] This flowchart illustrates one embodiment of a method for identifying and removing outliers for data authentication. [Figure 4] This is an exemplary node for implementing the method of the present invention. [Figure 5] This is an illustrative graph for quantitative evaluation of a dataset. [Figure 6] Figures 6A and 6B are graphs for quantitative evaluation of the dataset in Figure 5, illustrating an arbitrarily sampled dataset and a realistic dataset relative to the entire dataset, respectively. [Figure 7] Figures 7A and 7B are graphs for quantitative evaluation of the dataset in Figure 5, illustrating an arbitrarily sampled dataset and a realistic dataset, respectively, after removing 30% of the data as outliers. [Figure 8] Figures 8A and 8B are graphs for quantitative evaluation of the dataset in Figure 5, illustrating an arbitrarily sampled dataset and a realistic dataset, respectively, after removing 50% of the data as outliers. [Figure 9] This document illustrates an example of a system used to reduce outlier bias in target variables measured for a facility. [Modes for carrying out the invention]
[0023] The following disclosure provides numerous different embodiments or examples of implementing different features of systems and methods for accessing and managing structured content. Specific examples of components, processes, and implementations are described to help clarify the invention. These are merely examples and are not intended to limit the invention from what is described in the claims. Well-known elements are presented without detailed description so as not to obscure preferred embodiments of the invention with unnecessary detail. For the most part, details that are unnecessary for a full understanding of preferred embodiments of the invention are omitted to the extent that such details are within the skill of a person skilled in the art.
[0024] The mathematical description of one embodiment of dynamic outlier bias reduction is as follows:
number
number
[0025]
number
[0026] Another mathematical description of one embodiment of dynamic outlier bias reduction is as follows:
number
number
[0027]
number
[0028] After each iteration in which new model coefficients are calculated from the current censored dataset, the data removed from the previous iteration is added back to the current censored data. This combination encompasses all data values in the completed dataset. The current model coefficients are then applied to the completed dataset to calculate a complete set of predicted values. Absolute and relative errors are calculated for the complete set of predicted values, and a new bias criterion percentile threshold is calculated. A new censored dataset is created by removing all data values whose absolute or relative error is greater than the threshold, and then the nonlinear optimization model is applied to the newly censored dataset to calculate new model coefficients. This process allows us to examine, in each iteration, the possibility that all data values are included in the model dataset. When the model coefficients converge to values that best fit the data, some data values that were excluded in the previous iteration may be included in subsequent iterations.
[0029] In one embodiment, variability in greenhouse gas emissions can lead to overestimation or underestimation of emission outcomes, resulting in bias in model predictions. These non-industrial influences, such as errors in environmental conditions and calculation procedures, can cause results for a particular facility to differ fundamentally from those for similar facilities unless the bias in model predictions is eliminated. Bias in model predictions can also be present due to unique operating conditions.
[0030] If the analyst is confident that the facility's calculations are incorrect or that it possesses unique mitigating characteristics, the bias can be manually removed by simply excluding the facility's data from the calculations. However, when measuring facility performance from many different companies, regions, and countries, precise a priori knowledge of the data details is not practical. Therefore, any data removal procedure based on the analyst's judgment carries the risk of introducing undocumented and unsupported biases into the model results.
[0031] In one embodiment, dynamic outlier bias reduction is applied to a procedure that uses data and a predetermined global error criterion to determine statistical outliers to be removed from model coefficient calculations. This is a data-driven process that identifies outliers using a global error criterion provided by the data, for example, using a percentile function. The use of dynamic outlier bias reduction is not limited to reducing bias in model predictions. Its use in this embodiment is illustrative and illustrative only. Dynamic outlier bias reduction can also be used to remove outliers from, for example, any statistical dataset. This includes, but is not limited to, its use in the calculation of arithmetic mean, linear regression, and trend lines. Outliers are still ranked from the calculation results, but they are not used in the filtered dataset applied to calculate model coefficients or statistical results.
[0032] A commonly used standard procedure for removing outliers is to calculate the standard deviation (σ) of the dataset and simply define all data points outside a 2σ interval from the mean as outliers. This procedure generally involves statistical assumptions that are not actually testable. A description of a dynamic outlier bias reduction method applied in one embodiment of the present invention is summarized in Figure 1, which uses both relative and absolute errors. For example, for facility "m": Relative error m =((Predicted value m - Actual value m ) / actual value m ) 2 (1) Absolute error m =(predicted value) m - Actual value m ) 2 (2) This is the result.
[0033] In step 110, the analyst identifies an error threshold criterion that defines outliers to be removed from the calculation. For example, a percentile operation may be used as the error function to set an 80th percentile value for relative and absolute errors. This means that the calculation involves data values less than the 80th percentile value for relative errors and data values of the 80th percentile value for absolute errors, and the remaining values are removed or considered outliers. In this example, for data values that should be avoided from being removed, those data values must be less than the 80th percentile value for both relative and absolute errors. However, since the percentile thresholds for both relative and absolute errors can vary independently, in other embodiments, only one percentile threshold may be used.
[0034] In step 120, the model standard error and the coefficient of determination (r 2 The percentage change criteria for ) are identified. While the values of these statistics vary from model to model, the percentage change in the preceding iteration can be set provisionally, for example, 5 percent. These values can be used to terminate the iteration. Other termination criteria can be simply the number of iterations.
[0035] In step 130, optimization calculations are performed to generate model coefficients and predicted values for each facility.
[0036] In step 140, both relative and absolute errors for all facilities are calculated using equations (1) and (2).
[0037] In step 150, the error function with the threshold criteria identified in step 110 is applied to the data calculated in step 140 to determine the outlier threshold.
[0038] In step 160, the data is filtered to include only facilities where the relative error, absolute error, or both errors are less than the error threshold calculated in step 150, depending on the selected configuration.
[0039] In step 170, the optimization calculation is performed using the outlier-removed dataset.
[0040] In step 180, the standard error and r 2 The percentage change is compared to the criterion identified in step 120. If the percentage change is greater than the criterion, the process is repeated by returning to step 140. Otherwise, the iterative procedure ends in step 190, completing the resulting model calculated from this dynamic outlier bias reduction criterion. The model results are applied to all facilities, regardless of the state of previously removed or accepted data in their current iteration.
[0041] In other embodiments, the process begins with the selection of predetermined iteration parameters: (1) the absolute error and relative error percentile values to be used in the iteration process, (2) the coefficient of determination (r 2 (3) The improvement value of the standard error (also known as) and the standard error improvement value.
[0042] The process begins with either the original dataset, the actual data set, or at least one coefficient or factor used to calculate predicted values based on the original dataset. The coefficient or set of coefficients is applied to the original dataset to create a set of predicted values. The set of coefficients may include, but is not limited to, scalars, exponents, parameters, and periodic functions. The set of predicted data is then compared to the set of actual data. The standard error and coefficient of determination are calculated based on the difference between the predicted and actual data. Absolute and relative errors associated with each data point are used to remove outliers based on user-selected absolute and relative error percentiles. Data ranking is not necessary, as all data outside the range associated with the percentiles for absolute and / or relative errors are removed from the original dataset. The use of absolute and relative errors to filter data is illustrative and for illustrative purposes only, as this method can be performed for absolute or relative errors only, or for other functions.
[0043] The data associated with absolute and relative errors within the percentile range of user selection constitutes an outlier-removed dataset, with each iteration of the process having its own filtered dataset. This first outlier-removed dataset is used to determine predicted values compared to actual values. At least one coefficient is determined by optimizing the errors, and this coefficient is then used to generate predicted values based on the first outlier-removed dataset. The outlier-reduced coefficient acts as a mechanism for knowledge transfer from one iteration to the next.
[0044] After the first outlier-removed dataset is created, the standard errors and coefficient of determination are calculated and compared to the standard errors and coefficient of determination of the original dataset. The process stops if both the difference in standard errors and the difference in coefficient of determination are less than the respective improvement values. However, if at least one of the improvement criteria is not met, the process continues for another iteration. Using standard errors and coefficient of determination to check the iterative process is merely illustrative and illustrative, as such checks can be performed using standard errors alone, coefficients of determination alone, different statistical checks, or other performance termination criteria (such as the number of iterations).
[0045] If the first iteration fails to meet the improvement criteria, a second iteration is initiated by applying the first outlier-reduced data coefficients to the original data to determine the predicted values for the new set. In this case, the original data is processed again, and the absolute and relative errors for the data points, as well as the standard error and coefficient of determination values for the original data, are established while the coefficients from the first outlier-reduced dataset are being used. The data is then filtered to form a second outlier-reduced dataset, and coefficients based on this second outlier-reduced dataset are determined.
[0046] However, the second outlier-removed dataset is not necessarily a subset of the first outlier-removed dataset, but rather is associated with a second set of outlier-bias-reduced model coefficients, a second standard error, and a second coefficient of determination. Once these values are determined, the second standard error is compared to the first standard error, and the second coefficient of determination is compared to the first coefficient of determination.
[0047] The process terminates when the improvement (in standard error and coefficient of determination) exceeds the difference between these parameters. Otherwise, another iteration is started by processing the original data again. At this point, the second outlier-bias-reduced coefficients are used to process the original dataset and generate a new set of predicted values. A third outlier-removed dataset is created, optimized to determine a third set of outlier-bias-reduced coefficients through filtering based on the user-selected percentile values for absolute and relative errors. The process continues until error improvement or other termination criteria (such as a convergence criterion or a specific number of iterations) are met.
[0048] The output of this process is a set of coefficients or model parameters, where the coefficients or model parameters are mathematical values (or sets of values), such as, but not limited to, model predictions for comparison with data, the slope and intercept values of a linear equation, exponents, or polynomial coefficients. The output of dynamic outlier reduction is not its own output value, but rather a set of coefficients that modify the data to determine the output value.
[0049] In other embodiments illustrated in Figure 2, dynamic outlier reduction is applied as a data quality method to assess the consistency and accuracy of data to ensure that the data is suitable for a particular use. This method does not involve iterative steps for data quality operations. Other data quality methods can be used in conjunction with dynamic outlier reduction during this process. This method is applied to the arithmetic mean calculation of a given dataset. The data quality criterion is, for example, that consecutive data values are contained within the same range; that is, any values at excessively far apart intervals constitute inferior quality data. In this case, the error term consists of consecutive values of a function, and dynamic outlier reduction is applied to these error values.
[0050] In step 210, the initial data is listed in any order.
[0051] Step 220 constitutes a function or operation performed on the dataset. In this embodiment, the function and operation is ascending sorting of the data, followed by a continuous arithmetic mean calculation where each line corresponds to the mean of all data above that line.
[0052] Step 230 calculates relative and absolute errors from the data using consecutive values from the results of Step 220.
[0053] Step 240 allows the analyst to input the desired outlier removal error criterion (%). The quality criterion value is the result of the error calculation in Step 230 based on the data in Step 220.
[0054] Step 250 shows the dataset after filtering for data quality outliers. If the relative and absolute errors exceed the specific error criteria given in Step 240, specific values are removed.
[0055] Step 260 shows a comparison of the arithmetic mean calculation between the completed dataset and the outlier-removed dataset. This is the final step for the analyst to determine whether the identified outlier-removed data components are actually of inferior quality in all applicable mathematical or statistical calculations. Dynamic outlier reduction systems and methods eliminate the need for the analyst to directly remove data, and optimal implementation guidelines prompt the analyst to review and check the results for implementation validity.
[0056] In other embodiments illustrated in Figure 3, dynamic outlier reduction is applied as a data validation method to test the reasonable accuracy of a dataset to determine whether the data is suitable for a particular use. The method does not involve iterative steps for data validation operations. In this example, dynamic outlier reduction is applied to the calculation of the Pearson correlation coefficient between two datasets. The Pearson correlation coefficient is highly sensitive to values in a dataset that differ relatively from other data points. Validating a dataset against this statistic is important to ensure that the results represent what the majority of the data suggests, apart from the influence of extreme values. The data validation process in this example is that consecutive data values are contained within a specified range; that is, any values that are too far apart (e.g., outside the specified range) are considered inferior quality data. This is achieved by constructing a continuous error term for the function. By applying dynamic outlier reduction to these error values, the outlier-removed dataset becomes validated data.
[0057] In step 310, pairs of data are listed in any order.
[0058] Step 320 calculates the relative and absolute errors for each pair arranged in the dataset.
[0059] Step 330 allows the analyst to input the desired data authentication criteria. In this example, a 90% relative and absolute error threshold is selected. The quality criterion value in Step 330 is the resulting absolute and relative error percentile value for the data shown in Step 320.
[0060] Step 340 demonstrates the outlier removal process. This process removes potentially invalid data from the dataset using a criterion where both relative and absolute error values exceed the values corresponding to the user-selected percentile values entered in step 330. In practice, other error criteria can be used, so if multiple criteria are applied as shown in this example, any combination of error values can be applied to determine the outlier removal rules.
[0061] Step 350 calculates statistical results for the authenticated data and the raw data values. In this case, it is the Pearson correlation coefficient. These results are then examined for implementation validity by the analyst.
[0062] In other embodiments, dynamic outlier reduction is used to authenticate the entire dataset. Standard error improvements, coefficient of determination improvements, and absolute and relative error thresholds are selected, and the dataset is then filtered according to the error criteria. Even if the original dataset is of high quality, there will still be some data points with error values that fall outside the absolute and relative error thresholds. Therefore, it is important to determine whether any removal of data is necessary. If the outlier-removed dataset passes the standard error improvement and coefficient of determination improvement criteria after the first iteration, the original dataset is authenticated. This is because the filtered dataset generates standard errors and coefficients of determination that are too small to be considered significant (e.g., below the selected improvements).
[0063] In other embodiments, dynamic outlier reduction is used to provide insight into how each iteration of outlier removal affects the calculation. A graph or data table is provided so that the user can observe the progress of the outlier removal calculation as each iteration is performed. This stepwise approach allows the analyst to observe unique characteristics of the calculation that can add value and knowledge to the results. For example, the speed and nature of convergence demonstrate the impact of dynamic outlier reduction on calculating representative factors for multidimensional datasets.
[0064] As illustrated, we consider a linear regression calculation on a low-quality dataset of 87 records. The form of the equation to be regressiond is y = mx + b. Table 1 shows the results of the iterative process for 5 iterations. Notably, convergence is achieved in 3 iterations using a 95% relative and absolute error criterion. Changes in the regression coefficients can be observed. The dynamic outlier bias reduction method reduced the calculation dataset based on 79 records. A relatively low coefficient of determination (r) is observed. 2 =39%) is r 2 This indicates the need to test low (<95%) criteria to examine the additional outlier removal effect on statistics and calculated regression coefficients. [Table 1]
[0065] Table 2 shows the results of applying dynamic outlier bias reduction using 80% relative and absolute error criteria. Notably, a 15 percent point change in the outlier error criterion (from 95% to 80%) results in an additional 35% reduction in acceptable data (from 79 to 51 records). 2 This resulted in a 35 percent point increase (from 39% to 74%). To communicate the outlier-removed results to a broad audience and to provide insight into the effects of data variability in the analysis results, analysts can use graphs of the changes in the regression line along with the outlier-removed data and numerical results in Tables 1 and 2 of the analysis process. [Table 2]
[0066] As illustrated in Figure 4, one embodiment of the system used to carry out the method includes a computer system. The hardware consists of a processor 410 containing system memory 420 sufficient to perform the necessary numerical calculations. The processor 410 executes a computer program located in system memory 420 to carry out the method. A video and storage controller 430 is used to enable operation of a display 440. The system includes various data storage devices for data input. For example, a floppy disk unit 450, an internal / external disk drive 460, an internal CD / DVD 470, a tape unit 480, and other types of electronic storage media 490. The data storage devices described above are illustrative and illustrative only. These storage media are used to input datasets and outlier removal criteria into the system, store the outlier-removed datasets, store calculation factors, and store system-generated trend lines and trend line iteration graphs. Calculations can be performed by applying statistical software packages or from data entered in spreadsheet format using, for example, Microsoft Excel 100. The calculations are performed using customized software programs designed for enterprise-specific system implementation, or using commercially available software compatible with database and spreadsheet programs such as Excel. The system may also have an interface with a proprietary or public external storage medium 300 that interacts with other databases to provide data used in conjunction with the dynamic outlier bias reduction system and method calculations. Output devices may be telecommunication devices 510 that transmit system-generated graphs and reports, such as calculation worksheets, to administrative staff, printers 520, input devices 450, 460, 470, 480, 490, electronic storage media similar to those described above, and a proprietary storage database 530 via an intranet or the Internet. These output devices are illustrated and illustrative only.
[0067] As illustrated in Figures 5, 6A, 6B, 7A, 7B, 8A, and 8B, in one embodiment, dynamic outlier bias reduction can be used to quantitatively and qualitatively evaluate the quality of a dataset. This is based on the fact that the errors and correlations of the data values in the dataset are compared with the errors and correlations of a benchmark dataset consisting of random data values expanded from an appropriate range. In one embodiment, the errors can be specified to be the standard errors of the dataset. The correlations are the coefficient of determination (r) of the dataset. 2 ) can be specified to be. In other embodiments, the correlation can be specified to be Kendall's rank correlation coefficient, commonly referred to as Kendall's tau (τ) coefficient. In yet another embodiment, the correlation can be specified to be Spearman's rank correlation coefficient or Spearman's ρ (rho) coefficient. As described above, dynamic outlier bias reduction is used to systematically remove data values that are identified as outliers. It does not describe representatives of the underlying model or process. Typically, outliers are associated with a relatively small number of data values. However, in reality, datasets can be contaminated with spurious values or random noise without our knowledge. The graphs in Figures 5, 6A, 6B, 7A, 7B, 8A and 8B illustrate how dynamic outlier bias reduction systems and methods can be applied to identify situations where the underlying model is not supported by the data. Outlier reduction is performed by removing data values where the relative and / or absolute error calculated between the predictive model and the actual data values is greater than a bias criterion based on percentiles, such as 80%. This means that if the relative or absolute error percentile value is greater than the percentile threshold associated with the 80th percentile (where 80% of the data values have an error less than this value), the data value will be removed.
[0068] As illustrated in Figure 5, both a realistic model development dataset and a random-value dataset are compared, as these are typically models developed within the scope of the actual dataset. In practice, analysts typically have no prior knowledge of arbitrary dataset contamination; therefore, such understanding must be based on observing iterative results from several model calculations using dynamic outlier bias reduction systems and methods. Figure 5 illustrates an example of a model development calculation result for both datasets. The standard error, i.e., the amount of error that the model cannot explain, is represented by the coefficient of determination (%) or r, which indicates how much of the data variability is explained by the model. 2 The data is plotted against a specific point. The percentile value next to each point represents the bias criterion. For example, 90% indicates that data values with relative or absolute error values greater than the 90th percentile are removed from the model as outliers. This corresponds to removing 10% of the data values with the highest error in each iteration.
[0069] As illustrated in Figure 5, errors are reduced by increasing the bias criterion for both random and realistic dataset models. That is, the standard error and coefficient of determination are improved for both datasets. However, the standard error for random datasets is two to three times larger than that for realistic model datasets. Analysts can use an 80% coefficient of determination requirement as an acceptable level of precision for determining model parameters, for example. In Figure 5, the 80% r is calculated with a 70% bias criterion for random datasets and with an approximately 85% bias criterion for realistic data. 2This is achieved. However, the corresponding standard errors for random datasets are more than twice as large as those for realistic datasets. That is, by systematically performing model dataset analysis with different bias criteria and repeating the calculations with representative pseudo-datasets and plotting the results as shown in Figure 5, analysts can assess the acceptable bias criteria for a dataset (i.e., the acceptable percentage of removed data values), and thus the overall dataset quality. Furthermore, such systematic model dataset analysis can be used to automatically provide advice on the feasibility of datasets used in model development based on a configurable set of parameters. For example, in one embodiment in which a model is developed using dynamic outlier removal on a dataset, the error and correlation coefficient values for the model dataset and representative pseudo-dataset calculated under different bias criteria can be used to automatically provide advice on the feasibility of the dataset in supporting the developed model, and essentially the feasibility of the developed model in supporting the dataset.
[0070] As illustrated in Figure 5, observing the behavior of these model performance values for several cases provides a quantitative basis for determining whether the data values are representative of the process being modeled. For example, referring to Figure 5, the standard error for a realistic dataset at a 100% bias criterion (i.e., no bias reduction) approximately corresponds to the standard error for a random dataset at a 65% bias criterion (i.e., 35% of the data values with the highest error removed). This finding supports the conclusion that the data is uncontaminated.
[0071] In addition to the quantitative analysis described above, which is facilitated by the illustrative graph in Figure 5, dynamic outlier bias reduction can be used in subjective procedures to assist in the quality assessment of a dataset, even if they are not as powerful. This is done by plotting the model's predicted values against the actual target values given by the data, for both outliers and included outcomes.
[0072] Figures 6A and 6B illustrate the plots for 100% of both the realistic and random curves in Figure 5. The large scatter in Figure 6A is consistent with the inability of the model to fit any target value and consequently to intentional randomness. Figure 6B is consistent with and general to the actual data set, with the model prediction and actual values clustered around the line where the model prediction equals the actual target value (hereinafter referred to as the actual-prediction line).
[0073] Figures 7A and 7B illustrate the results from 70% of the data points in Figure 5 (i.e., 30% of the data has been removed as outliers). In Figures 7A and 7B, outlier bias reduction is shown to remove the points furthest from the actual-prediction line, but the large variability in model accuracy between Figures 7A and 7B indicates the process by which this dataset is modeled.
[0074] Figures 8A and 8B show the results from the 50% point in Figure 5 (i.e., 50% of the data has been removed as outliers). In this case, approximately half of the data is identified as outliers, and even with this much variability removed from the dataset, the model does not exactly describe the random dataset in Figure 8A. The general variability around the actual-prediction line is roughly the same as in Figures 6A and 7A, considering the removed data in each case. Figure 8B shows that when 50% of the variability is removed, the model was able to produce prediction results that closely matched the actual data. In addition to the performance criterion analysis shown in Figure 5, these types of visualization plots can be used by analysts to assess the quality of the actual dataset in their implementation of model deployment. Figures 5, 6A, 6B, 7A, 7B, 8A, and 8B illustrate visualization plots. Here, the analysis is based on performance criterion trends corresponding to various bias criteria. In other embodiments, the analysis may be based on other variables corresponding to bias criteria, such as model coefficient trends corresponding to various bias criteria chosen by the analyst.
[0075] Various embodiments include a system for reducing outlier bias in target variables measured for a facility. Figure 9 shows an example of such an embodiment. The system illustrated in Figure 9 includes a computer unit 1012 capable of processing a dataset, such as a dataset encompassing various performance measurements for an industrial facility. The computer unit 1012 includes a processor 1014 and a storage subsystem 1016, in which a computer program embodies the dynamic outlier bias removal method disclosed herein. The system 1010 includes an input unit 1018. The input unit 1018 further includes a measuring device 1020 that measures a given target variable and provides a corresponding dataset. The measuring device 1020 can be configured to measure any target variable of interest, such as, for example, the number of parts produced from an industrial plant facility per unit time, or the volume of refined material produced by a refining facility per unit time. In addition, multiple target variables can be measured simultaneously. In the illustrated embodiment, the measuring device 1020 includes a sensor 1022. Those skilled in the art will see that the scope of the present invention includes a variety of sensors used to measure various physical attributes of a substance and / or components produced by or used in an industrial facility. Examples include sensors capable of detecting and quantifying chemical substances such as greenhouse gas emissions. In addition, those skilled in the art will see that the measurement of a target variable of interest includes optional means of data collection, reception, measurement, storage, and processing. Target variables, datasets, and data can include all types of data, including but not limited to industrial process data, computer system data, financial data, economic data, stock, bond, and futures data, internet search data, security data, human identification data such as voice, cloud data, big data, and insurance data. The scope and implications of this disclosure and the present invention are not limited to such types of target variables, datasets, or data. Those skilled in the art will also see that sensors and measuring devices are or may include computers, computer systems, and processors. Furthermore, system 1010 includes an output unit 1024 capable of outputting processed data.The output device includes a monitor, printer, or transmitting device (not shown).
[0076] In one embodiment, system 1010 activates sensor 1022. Sensor 1022 then detects and quantifies a given compound, such as carbon dioxide. Detection and quantification can be performed continuously or within discrete time steps. A dataset is generated after each measurement is completed, stored in storage subsystem 1016, and input to computer unit 1012. The dataset is processed by a dynamic outlier removal computer program stored in storage subsystem 1016 and terminated according to various embodiments of the method disclosed herein. Once the computer program has finished processing the data, the processed data is output by output unit 1024. In embodiments where output unit 1024 is a monitor or printer, the results are visualized in a diagram. In one embodiment where output unit 1024 includes a transmitter, the processed data is sent to a central database or control center, where the data is further processed (not shown). Thus, systems according to the various embodiments of the disclosure provide a powerful tool for comparing different facilities within a company or within a technical field in an automated manner with reduced outlier bias.
[0077] In a preferred embodiment, the measuring device 1020 includes one or more sensors for detecting and quantifying chemical substances. Due to global warming, greenhouse gas emissions from facilities are becoming an increasingly important target variable. Facilities that emit small amounts of greenhouse gases are ranked better than facilities that emit large amounts. However, the overall productivity of the latter is better. Greenhouse gases include, for example, carbon dioxide (CO2), ozone (O3), water vapor (H2O), hydrofluorocarbons (HFCs), perfluorocarbons (PFCs), chlorofluorocarbons (CFCs), sulfur hexafluoride (SF6), methane (CH4), nitrous oxide (N2O), carbon monoxide (CO), and nitrogen oxides (NO2O). xThese are ) and non-methane volatile organic compounds (NMVOCs). Automated detection and quantification of these compounds can be used to develop industrial standards for predetermined permissible emissions of greenhouse gases. However, the application of dynamic outlier bias removal results in the removal of outliers caused by abnormal production conditions, such as operational errors or even accidents. That is, accurate and meaningful standard development can be obtained by using the various embodiments disclosed herein. Once the industrial standards are developed, the system is used to compare emissions with those standards.
[0078] Those skilled in the art will further see that the scope of the present invention includes applications of various embodiments of the disclosure for reducing outlier bias in target variables related to financial instruments such as equity securities (e.g., common stock) or derivative contracts (e.g., forwards, futures, options, swaps, etc.). For example, in one embodiment, system 1010 includes an input unit 1018 that receives data related to a financial instrument such as common stock and provides a corresponding dataset. The target variable may be the stock price. Furthermore, variables related to the target variable can be determined using various well-known methods for evaluating financial instruments, such as discounted cash flow analysis. Such related variables may include related dividends, retained earnings, or cash flows, earnings per share, price-to-earnings ratio, or growth rate, etc. Once a database of target and related variable values is formed, the various embodiments of dynamic outlier bias removal disclosed herein can be applied to the database to obtain an accurate model for evaluating financial instruments.
[0079] The above disclosure and description of preferred embodiments of the present invention are illustrative and exemplary, and those skilled in the art will understand that various modifications can be made in detail of the exemplary systems and methods without departing from the scope of the invention.
Claims
1. A computer implementation method for reducing outlier bias, The processor, The steps include determining a target variable for the facility, A step of identifying a plurality of variables for the facility, wherein the plurality of variables include a plurality of direct variables for the facility that affect the target variable, and a plurality of transformed variables for the facility, each of which is a function of at least one of the direct variables that affects the target variable. A step of obtaining a dataset for the facility that includes the values of the plurality of variables, The steps include selecting an initial set of multiple actual values for the target variable and one or more model coefficients, The steps include generating a plurality of model predicted values for the target variable based on the dataset and the initial set of one or more model coefficients, The steps include determining a plurality of error values based on the plurality of model predicted values and the plurality of actual values of the target variable, The steps include determining one or more error thresholds based on the aforementioned plurality of error values and one error criterion, The steps include: removing data with error values greater than or equal to a corresponding error threshold from the aforementioned dataset to generate an outlier-removed dataset; The steps include determining one or more new model coefficients based on the outlier-removed dataset and one or more model coefficients from the previous iteration, The steps include generating a plurality of new model prediction values based on the dataset and the one or more new model coefficients, A computer implementation method comprising the steps of repeating at least the determination of the error value, the determination of the error threshold, the generation of the outlier-removed dataset, the determination of the new model coefficients, and the generation of the new model predicted value, using the one or more new model coefficients in place of the one or more model coefficients in the previous iteration until one performance termination criterion is met.
2. The computer implementation method according to claim 1, wherein the error value includes at least one of a relative error and an absolute error.
3. The computer implementation method according to claim 2, wherein the plurality of error values include a plurality of relative error values and a plurality of absolute error values.
4. With reference number m, the relative error value is given by: relative error m = ((predicted value m - actual value m) / actual value m) 2 The absolute error value is determined by the following formula: absolute error m = (predicted value m - actual value m) 2 The computer implementation method according to claim 3, determined by the method described above.
5. The computer implementation method according to claim 2, wherein the step of determining one or more error thresholds includes determining one or more percentile thresholds using percentile operations on the error value.
6. The aforementioned multiple error values include multiple relative error values and multiple absolute error values, The computer implementation method according to claim 5, wherein the percentile threshold for the relative error value and the percentile threshold for the absolute error value can be set independently, and the outlier-removed dataset includes data in which both the relative error value and the absolute error value are smaller than the corresponding percentile threshold.
7. The computer implementation method according to claim 6, wherein the percentile threshold for the relative error value and the percentile threshold for the absolute error value each correspond to the 80th percentile value.
8. The computer implementation method according to claim 1, wherein the step of determining the one or more new model coefficients includes using the outlier-removed dataset and determining the one or more new model coefficients by at least one of a linear optimization model and a nonlinear optimization model to reduce the error between the model predicted values and the actual values.
9. The computer implementation method according to claim 1, wherein each iteration in the repeated step is performed using the respective outlier-removed dataset generated in that iteration.
10. The computer implementation method according to claim 1, further comprising determining a model performance value for each iteration, which includes at least one of the standard error and the coefficient of determination, wherein the performance termination criterion is based on the change in the model performance value between iterations.
11. The aforementioned model performance value includes both the standard error and the coefficient of determination. The aforementioned performance termination criteria include the standard error termination value and the coefficient of determination termination value, The computer implementation method according to claim 10, wherein the performance termination criterion is satisfied when the standard error termination value is greater than the difference between the first standard error and the second standard error, and the coefficient of determination termination value is greater than the difference between the first coefficient of determination and the second coefficient of determination.
12. A system that reduces outlier bias, A server including a processor and storage subsystem, The storage subsystem stores a database containing a dataset for the facility, The computer program stored in the aforementioned storage subsystem, Equipped with, When the computer program is executed by the processor, the processor will: To determine one target variable for the aforementioned facility, Identifying a plurality of variables relating to the facility, wherein the plurality of variables include a plurality of direct variables relating to the facility that affect the target variable, and a plurality of transformed variables relating to the facility, each of which is a function of at least one of the direct variables that affects the target variable. Using the values of the multiple variables included in the dataset, the multiple actual values of the target variable, and an initial set of one or more model coefficients, multiple model predicted values for the target variable are generated. Determining multiple error values based on the multiple model predicted values and the multiple actual values of the target variable, Determining one or more error thresholds based on the aforementioned multiple error values and one error criterion, From the aforementioned dataset, data with error values exceeding a corresponding error threshold are removed to generate an outlier-removed dataset. Determine one or more new model coefficients based on the outlier-removed dataset and one or more model coefficients from the previous iteration, The process involves generating multiple new model prediction values based on the aforementioned dataset and the one or more new model coefficients, A system including an instruction to repeat, at least, the determination of the error value, the determination of the error threshold, the generation of the outlier-removed dataset, the determination of the new model coefficients and the generation of the new model prediction value, using the one or more new model coefficients in place of the one or more model coefficients in the previous iteration, until one performance termination criterion is met.
13. The system according to claim 12, wherein the error value includes at least one of a relative error and an absolute error.
14. The aforementioned multiple error values include multiple relative error values and multiple absolute error values. With reference number m, the relative error value is given by: relative error m = ((predicted value m - actual value m) / actual value m) 2 The absolute error value is determined by the following formula: absolute error m = (predicted value m - actual value m) 2 The system according to claim 13, determined by...
15. The system according to claim 13, wherein the one or more error thresholds include one or more percentile thresholds determined using percentile operations on the error value.
16. The aforementioned multiple error values include multiple relative error values and multiple absolute error values, The system according to claim 15, wherein the percentile threshold for the relative error value and the percentile threshold for the absolute error value can be set independently, and the outlier-removed dataset includes data in which both the relative error value and the absolute error value are smaller than the corresponding percentile threshold.
17. The system according to claim 12, wherein the instruction causes the processor to use the outlier-removed dataset to determine one or more new model coefficients using at least one of a linear optimization model and a nonlinear optimization model to reduce the error between the model predicted values and the actual values.
18. The system according to claim 12, wherein each iteration in the aforementioned repetition is performed using the respective outlier-removed dataset generated in that iteration.
19. The system according to claim 12, wherein the instruction further causes the processor to determine a model performance value for each iteration, which includes at least one of the standard error and the coefficient of determination, and the performance termination criterion is based on the change in the model performance value between iterations.
20. The aforementioned model performance value includes both the standard error and the coefficient of determination. The aforementioned performance termination criteria include the standard error termination value and the coefficient of determination termination value, The system according to claim 19, wherein the performance termination criterion is satisfied when the standard error termination value is greater than the difference between the first standard error and the second standard error, and the coefficient of determination termination value is greater than the difference between the first coefficient of determination and the second coefficient of determination.
Citation Information
Patent Citations
Adapting method for engine control parameter and its system
JP2004068729A
Integrated circuit device abnormality detection apparatus, method and program
JP2008166644A
Network performance prediction system, network performance prediction method, and program
JP2009253362A
Working hour estimation device, method, and program
JP2010250674A
Stock portfolio and method
US20050131794A1
