A data cleaning method for a data-driven optimization system

Abstract

The present invention relates to the technical field of electrical digital data processing, and particularly relates to a data cleaning method for a data-driven optimization system. The method includes: obtaining various production data of different categories in an SCR flue gas denitration system; determining the importance of each production data for the outlet nitrogen oxide concentration during the SCR flue gas denitration process; establishing a sliding window for each production data, and using the importance to determine the box plot of each production data; using the box plot to determine the first set of potential abnormal data points of each production data, and using rules to determine the second set of potential abnormal data points of each production data; determining the intersection of abnormal data points between the first set and the second set, and using the intersection of abnormal data points to perform data cleaning on each production data; through the data cleaning method of the data-driven optimization system of the present invention, the accuracy of abnormal data recognition can be improved and the generalization ability can be improved.

Description

Technical Field

[0001] The present invention relates to the technical field of electric digital data processing, and particularly relates to a data cleaning method for a data-driven optimization system. Background Art

[0002] In order to solve the related environmental problems caused by coal-fired flue gas emissions, among many flue gas denitrification technologies, the SCR (Selective Catalytic Reduction) denitrification system has been widely used in coal-fired power plants due to its advantages such as mature technology, convenient installation, high denitrification efficiency, and almost no secondary pollution. The SCR flue gas denitrification process is a complex physical and chemical process with characteristics such as non-linearity and time lag. For these complex processes, it is difficult to describe them with a mechanism model.

[0003] Currently, a new system optimization method has been created through data-driven optimization technology. By using a large amount of production data generated by the control system, data engineers have achieved data-driven optimization in the SCR flue gas denitrification process by means of data science techniques. When performing data-driven optimization, a large amount of production data generated by the control system is first collected. When the collected production data contains outliers, due to the large difference between outliers and normal values, it will greatly reduce the generalization ability of the model and increase the burden of data processing, resulting in a decrease in analysis efficiency. Therefore, data cleaning is particularly important for data-driven optimization technology.

[0004] Existing data cleaning methods usually use box plots to detect outliers in data, judge abnormal data through quartiles, and then achieve data cleaning. However, the production data in the SCR flue gas denitrification process has the characteristics of non-linearity and time lag, which leads to uneven distribution of production data in time. And due to the influence of time lag, the current data may be affected by historical data, ultimately resulting in inaccurate identification of abnormal data and affecting the effect of data cleaning. Summary of the Invention

[0005] In order to solve the technical problems of inaccurate identification of abnormal data in the current SCR flue gas denitrification system and poor data cleaning effect, the purpose of the present invention is to provide a data cleaning method for a data-driven optimization system, and the specific technical solution adopted is as follows:

[0006] The present invention provides a data cleaning method for a data-driven optimization system, and the method includes:

[0007] Obtain various production data of different categories in the SCR flue gas denitrification system;

[0008] Determine the importance of various production data for the outlet nitrogen oxide concentration during the SCR flue gas denitrification process;

[0009] Establish a sliding window for each production data, and use the importance to determine the box plot of each production data;

[0010] Use the box plot to determine the first set of potential abnormal data points of each production data, and use rules to determine the second set of potential abnormal data points of each production data;

[0011] Determine the intersection of abnormal data points between the first set and the second set, and use the intersection of abnormal data points to clean the data of each production data;

[0012] Among them, the window represents a data set, and the sliding window represents a variable data set with movable time series.

[0013] Furthermore, each production data includes the nitrogen oxide concentration in the reactor outlet flue, and the nitrogen oxide concentration in the reactor outlet flue is used as the control data;

[0014] The steps of determining the importance of various production data for the outlet nitrogen oxide concentration during the SCR flue gas denitrification process include:

[0015] Determine other production data in the SCR flue gas denitrification system except the nitrogen oxide concentration in the reactor outlet flue, create a target matching window on the other production data, and create a search window on the control data;

[0016] Determine each matched window intercepted with the time series in the search window, and determine the target matched window with the highest matching degree with the target matching window in each matched window;

[0017] Determine the first correlation degree between the target matching window and the target matched window;

[0018] Determine the second correlation degree between the nitrogen oxide concentration in the reactor outlet flue and the outlet nitrogen oxide concentration;

[0019] Use the first correlation degree and the second correlation degree to determine the importance of each production data for the outlet nitrogen oxide concentration.

[0020] Furthermore, the steps of determining the first correlation degree between the target matching window and the target matched window include:

[0021] Determine the matching degree and time interval between the target matching window and the target matched window;

[0022] Use the matching degree and the time interval to determine the first correlation degree between the target matching window and the target matched window.

[0023] Further, the steps of establishing a sliding window for each production data include:

[0024] Determine the initial size of the sliding window for each production data to obtain an initial sliding window;

[0025] Determine the skewness of the data point distribution in the initial sliding window;

[0026] According to the skewness, adjust the size of the initial sliding window to make it meet the preset normal distribution condition, and obtain the sliding window for each production data.

[0027] Further, the steps of determining the initial size of the sliding window for each production data to obtain an initial sliding window include:

[0028] Obtain the historical average duration of each denitrification treatment of the SCR flue gas denitrification system and the acquisition frequency of each production data;

[0029] Use the historical average duration and the acquisition frequency to calculate the initial size of the sliding window.

[0030] Further, the steps of adjusting the size of the initial sliding window according to the skewness to make it meet the preset normal distribution condition and obtain the sliding window for each production data include:

[0031] Use the skewness to calculate the reasonableness of the time length of the initial sliding window;

[0032] If the reasonableness is greater than the preset reasonable threshold, it is determined that the initial sliding window meets the preset normal distribution condition;

[0033] If the reasonableness is less than or equal to the preset reasonable threshold, expand the size of the initial sliding window until the reasonableness is greater than the preset reasonable threshold.

[0034] Further, the steps of establishing a sliding window for each production data and using the importance to determine the box plot of each production data include:

[0035] Determine the upper and lower quartiles and the median in the box plot corresponding to the sliding window;

[0036] Use the importance to determine the abnormal upper and lower limits in the box plot corresponding to the sliding window;

[0037] Use the upper and lower quartiles, the median, and the abnormal upper and lower limits to determine the box plot of the sliding window;

[0038] Summarize the box plots of the sliding windows to obtain the box plots of each production data;

[0039] Among them, the higher the importance, the closer the abnormal upper and lower limits are, and vice versa, the farther the abnormal upper and lower limits are.

[0040] Furthermore, the steps of determining the abnormal upper and lower limits in the box plot corresponding to the sliding window by using the importance include:

[0041] Using the importance, calculate the upper and lower limit adjustment coefficients of the box plot corresponding to the sliding window;

[0042] Using the upper and lower quartiles in the box plot corresponding to the sliding window, calculate the inner limit range of the box plot;

[0043] Using the upper quartile, the inner limit range and the upper and lower limit adjustment coefficients, calculate the abnormal upper limit in the box plot;

[0044] Using the lower quartile, the inner limit range and the upper and lower limit adjustment coefficients, calculate the abnormal lower limit in the box plot.

[0045] Furthermore, the steps of performing data cleaning on each production data by using the intersection of the abnormal data points include:

[0046] Determine the number of abnormal times when any target data point in the intersection of the abnormal data points is identified as an abnormal data, and the number of window of the sliding window containing the target data point;

[0047] Using the number of abnormal times, the number of windows and the total number of sliding windows corresponding to each production data, calculate the abnormal performance degree of the target data point;

[0048] If the abnormal performance degree is greater than the preset abnormal threshold, the target data point is marked as an abnormal point;

[0049] Perform data cleaning on all abnormal data marked as abnormal points in each production data.

[0050] Furthermore, after the step of calculating the abnormal performance degree of the target data point, the method further includes:

[0051] Determine the reasonable degree of the sliding window corresponding to the target data point;

[0052] Use the reasonable degree to correct the abnormal performance degree to obtain the corrected abnormal performance degree.

[0053] The present invention has the following beneficial effects:

[0054] The purpose of the present invention is to combine Clean the production data in the SCR flue gas denitrification process by using the box plot rule and other rules. Since the production data has the characteristics of non-linearity and time lag, obtain abnormal data in different dimensions to improve the effect of data cleaning and avoid deleting normal data.

[0055] This invention can combine the box plot rule and other rules to detect abnormal points in the production data during the SCR flue gas denitrification production process. When constructing the box plot, combine other rules to make the distribution of data points in the sliding window more reasonable. When identifying abnormal points, improve the accuracy of identification and generalization ability by obtaining the intersection of abnormal point sets and the intersection of abnormal points in different windows. At the same time, deeply obtain the manifestation degree of abnormal points according to the performance of abnormal points in different sliding windows and the distribution of data points in the window, and improve the accuracy of identifying abnormal data points. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] In order to more clearly illustrate the technical solutions and advantages in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the following drawings are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0057] Figure 1 It is a flowchart of the steps of a data cleaning method for a data-driven optimization system provided by an embodiment of the present invention;

[0058] Figure 2 It is a detailed flowchart of step S2 in a data cleaning method for a data-driven optimization system provided by an embodiment of the present invention;

[0059] Figure 3 It is a detailed flowchart of step S3 in a data cleaning method for a data-driven optimization system provided by another embodiment of the present invention;

[0060] Figure 4 It is a detailed flowchart of step S5 in a data cleaning method for a data-driven optimization system provided by an embodiment of the present invention;

[0061] Figure 5 It is a schematic structural diagram of the hardware operating environment of a data cleaning device for a data-driven optimization system involved in the embodiment solution of the present invention;

[0062] Figure 6 It is a schematic framework diagram of a data cleaning device for a data-driven optimization system involved in the embodiment solution of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0063] To further elaborate on the technical means and effects adopted by the present invention to achieve the intended invention purpose, the following specifically describes, in conjunction with the accompanying drawings and preferred embodiments, a data cleaning method for a data-driven optimization system proposed according to the present invention, including its specific implementation manner, structure, features, and effects. In the following description, different "one embodiment" or "another embodiment" do not necessarily refer to the same embodiment. In addition, specific features, structures, or characteristics in one or more embodiments can be combined in any suitable form.

[0064] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the technical field to which the present invention belongs.

[0065] The following specifically describes the specific solution of a data cleaning method for a data-driven optimization system provided by the present invention in conjunction with the accompanying drawings.

[0066] Embodiment 1:

[0067] For the data cleaning method of a data-driven optimization system provided by the present invention, please refer to Figure 1 , which shows the flowchart of the steps of the data cleaning method for the data-driven optimization system provided by an embodiment of the present invention.

[0068] The method includes:

[0069] Step S1, obtaining various production data of different categories in the SCR flue gas denitrification system;

[0070] In this embodiment, the specific scenario targeted is:

[0071] In the process of data cleaning for the production data in the SCR flue gas denitrification process, since the chemical reaction in the treatment process requires a certain time to reach chemical equilibrium, there is a lag between the actual chemical reaction rate and the theoretical prediction. At the same time, after the chemical reagent is initially put in, it takes a certain time for transportation and diffusion, further resulting in a lag in the change of production data.

[0072] To sum up, the time stamps corresponding to the operations do not exactly correspond to the characteristics shown by the production data, resulting in the algorithm possibly identifying the data with normal changes as data with abnormal changes during the cleaning process, and then wrongly deleting or modifying them. At the same time, since the production data itself is a non-linear changing data, the algorithm cannot distinguish between normal and abnormal data according to the law of data change, further increasing the difficulty of cleaning. Therefore, in order to achieve data-driven optimization of the SCR flue gas denitrification system, a method that can more accurately clean the production data in the SCR flue gas denitrification process is required.

[0073] The production data mainly involved in the SCR denitration system include: nitrogen oxide content, oxygen content, ammonia content, temperature, flow rate, pressure, etc. Corresponding sensors are set according to the required data types to collect production data. The specific production data is shown in Table 1 below:

[0074] Table 1:

[0075]

[0076] It should be noted that the header of Table 1 includes: serial number, variable name, unit, and variable type. The serial number in this table represents the arrangement serial number of different variable names in Table 1. Different variable names represent different categories of production data. As shown above, there are 26 items of production data, but it is not limited to these 26 items, and it can be more or less than 26 items. The unit is the unit of each production data, and the variable type is used to distinguish whether different items of production data belong to controllable variables or uncontrollable variables.

[0077] As for how to obtain the above-mentioned various production data of different categories in the SCR flue gas denitration system, it can include the following links:

[0078] 1. First, install the required sensors at the corresponding positions of the SCR flue gas denitration system (referred to as "the system" for short). For example, install nitrogen oxide sensors, oxygen sensors, ammonia sensors, etc. at the inlet and outlet of the SCR reactor;

[0079] 2. Check the sensors on the SCR flue gas denitration system before the denitration work to ensure the accuracy of the sensors during data collection;

[0080] 3. Subsequently, set the acquisition parameters for the sensors to collect data, including the data acquisition frequency. For example, it is stipulated that the data acquisition frequency of all sensors is 5 seconds / time;

[0081] 4. Start the SCR flue gas denitration system and the sensors, and introduce the flue gas generated by coal block combustion into the SCR flue gas denitration system;

[0082] 5. Use the sensors to collect the production data during the operation of the system, and store the collected data in the designated area of the system.

[0083] So far, the collection of production data during the SCR flue gas denitration process is completed.

[0084] Step S2, determine the importance of each production data for the outlet nitrogen oxide concentration during the SCR flue gas denitration process;

[0085] In the process of data cleaning for the production data of SCR flue gas denitrification, the importance of each production data is different, and the required precision for data cleaning is also different. Based on the operating purpose of the system, it can be known that "NOx in the reactor outlet flue" (the concentration of nitrogen oxides in the reactor outlet flue) has a relatively high importance for the outlet nitrogen oxide concentration. Taking this as a reference, it is denoted as , therefore, by obtaining the correlation between different production data and "NOx in the reactor outlet flue", the importance of other different production data for the outlet nitrogen oxide concentration can be quantified. Here, the importance can be considered as the degree of influence on the outlet nitrogen oxide concentration.

[0086] Please refer to Figure 2 , in a specific embodiment, each production data includes the concentration of nitrogen oxides in the reactor outlet flue, and the concentration of nitrogen oxides in the reactor outlet flue is used as the control data; the step S2 includes:

[0087] Step S20, determine other production data in the SCR flue gas denitrification system except the concentration of nitrogen oxides in the reactor outlet flue, create a target matching window on the other production data, and create a search window on the control data;

[0088] Step S21, determine each matched window intercepted by the search window sliding with time sequence, and determine the target matched window with the highest matching degree with the target matching window in each matched window;

[0089] Step S22, determine the first correlation degree between the target matching window and the target matched window;

[0090] Specifically, the step S22 includes:

[0091] Determine the matching degree and time interval between the target matching window and the target matched window;

[0092] Utilize the matching degree and the time interval to determine the first correlation degree between the target matching window and the target matched window.

[0093] Step S23, determine the second correlation degree between the concentration of nitrogen oxides in the reactor outlet flue and the outlet nitrogen oxide concentration;

[0094] Step S24, utilize the first correlation degree and the second correlation degree to determine the importance of each production data for the outlet nitrogen oxide concentration.

[0095] 1. First, select any one dimension from the required data list in Table 1 above , representing one other production data, and judge the production node type of this data, that is, input-end data, output-end data, and process data (temperature, load, etc.);

[0096] 2. Subsequently, according to the production node type of the selected production data, a matching window or a search window can be selected to be constructed on the time-series data set of the production data; it should be noted that since each production data is continuously collected over time, its data set form can be a time-series data set, that is, each data in each production data corresponds to its collection time. The window can be considered as a part of the data set intercepted from the complete time-series data set. The matching window and the search window are relative concepts, that is, the most matching window to be matched is found from the search window, which also means that the time length of the search window, that is, the size of the search window, is generally larger than the size of the matching window.

[0097] According to the characteristics of the time lag of production data, a matching window is constructed when the selected type is the input end, and conversely, a search window is constructed when the type is the output end or process data. The "NOx in the reactor outlet flue" is constructed as a search window. In the following text, a matching window is created for other production data, and the "NOx in the reactor outlet flue" is constructed as a search window to illustrate this embodiment.

[0098] 3. Construct a fixed window (target matching window) on the time-series data set, and stipulate that the length of the target matching window is , where represents the historical average duration of a denitrification treatment;

[0099] 4. Further, according to the length of the fixed window, a corresponding window (search window) is set on the control data dimension (here it can refer to "NOx in the reactor outlet flue"), and the size of the search window is stipulated as ;

[0100] It should be noted that the time starting points of the matching window and the search window are the same, and there is also a window (window to be matched) with the same size as the matching window in the search window, which is used for similarity analysis with the matching window.

[0101] 5. Subsequently, calculate the matching degree (matching coefficient) between the target matching window and the search window (Pearson correlation coefficient);

[0102] 6. Move the window to be matched in the search window along with the time series. For example, set the sliding step size of the window to 10 data points (which can be customized), and calculate the matching degree again ; It is relatively easy to understand that, for example, the data starting point of the first window to be matched corresponds to the data starting point of the search window, that is, the first data point, and the data starting point of the second window to be matched can correspond to the 11th data point of the search window. Here, the order is in accordance with the time series, that is, the sequence of collection time.

[0103] 7. Repeat the above steps until the matching of the search window is completed. Screen the target matched window with the highest matching degree among them with the target matching window, mark the time corresponding to the target matched window and obtain the time interval between it and the target matching window. The calculation of the time interval can be the acquisition time difference corresponding to the data start point or the respective data end points between the matching window and the matched window, that is, the data lag time ;

[0104] 8. Match each production data in the required data list with "NOx in the reactor outlet flue" to obtain the lag time of all data dimensions (all different categories of production data) ;

[0105] 9. Further, calculate the correlation between any production data and . The shorter the lag time and the higher the matching degree between the target matching window and the target matched window , the higher the correlation (the first correlation degree) ;

[0106] ;

[0107] 10. According to the correlation between other different production data and "NOx in the reactor outlet flue" , obtain the importance of different production data to the outlet nitrogen oxide concentration :

[0108] ;

[0109] Among them, represents the proportion of the NOx concentration in the reactor outlet flue in the total emissions, that is, the correlation (the second correlation degree) between "NOx in the reactor outlet flue" and the outlet nitrogen oxide concentration

[0110] Finally, obtain the importance of different production data to the outlet nitrogen oxide concentration .

[0111] Step S3, establish a sliding window for each production data and use the importance to determine the box plot of each production data

[0112] Among them, the sliding window represents a variable data set of movable time series. That is, the time length of the window, that is, the window size, can be a variable data set. It is relatively easy to understand that the larger the time length, the more data points correspond. Each data point in each production data can correspond to establish a corresponding sliding window, so each data point can be included in multiple sliding windows

[0113] Please refer toFigure 3 , in a specific embodiment, the step S3 includes:

[0114] Step S30, determine the initial size of the sliding window for each production data to obtain an initial sliding window;

[0115] Specifically, the step S30 includes:

[0116] Obtain the historical average duration of each denitrification treatment of the SCR flue gas denitrification system and the acquisition frequency of collecting each production data;

[0117] Use the historical average duration and the acquisition frequency to calculate the initial size of the sliding window.

[0118] Step S31, determine the skewness of the data point distribution in the initial sliding window;

[0119] Step S32, adjust the size of the initial sliding window according to the skewness to make it meet the preset normal distribution condition, and obtain the sliding window of each production data.

[0120] Specifically, the step S32 includes:

[0121] Use the skewness to calculate the rationality of the time length of the initial sliding window;

[0122] If the rationality is greater than the preset rationality threshold, it is determined that the initial sliding window meets the preset normal distribution condition;

[0123] If the rationality is less than or equal to the preset rationality threshold, expand the size of the initial sliding window until the rationality is greater than the preset rationality threshold.

[0124] For step S3 and each of the above specific steps:

[0125] Since each production data of the SCR denitrification system has the characteristics of non-linearity and time lag, during the data cleaning process, it is not possible to directly detect outliers in the overall data sample. In order to eliminate the time lag characteristics of the time series data, a sliding window is set for each production data to reduce the impact of historical data samples on data cleaning. At the same time, since The rule applies to the scenario where the data sample (the data sample mentioned in the embodiment is also the historical data set of each production data) follows a normal distribution. Therefore, when dividing the sliding window, it is also necessary to ensure that the data set of each production data presents a normal distribution as much as possible. The specific construction steps of the sliding window are as follows:

[0126] 1. First, randomly select a production data, and use the moving average algorithm to supplement the missing values in this data sample. That is, for the positions where there are missing data, calculate the mean of multiple (for example, five) data points on the left and right of the missing data point, and use the mean as the data of this missing data point;

[0127] 2. Subsequently, according to the acquisition frequency of the sensor and the historical average duration of the first denitration treatment (in seconds), set the initial size (time length) of the sliding window ;

[0128] ;

[0129] 3. Further, perform statistics on the data in the current sliding window (which can be the initial sliding window or the sliding window after adjusting the time length of the initial sliding window), and construct a two-dimensional coordinate system (the horizontal axis is the data value size, and the vertical axis is the number of data points);

[0130] 4. Calculate the mean value of the data values in the current sliding window and the skewness of the data point distribution (both the mean and skewness here are existing mathematical concepts and will not be elaborated too much here);

[0131] It should be noted that when (approaching) indicates that the data distribution in the current sliding window is symmetric, that is, its data points are normally distributed. On the contrary, if it is less than 0, it is left-skewed (the left tail is longer), and if it is greater than 0, it is right-skewed (the right tail is longer).

[0132] 5. When the absolute value of the skewness of the data points in the sliding window is smaller, the normal distribution of the data in the current sliding window is better, and thus the length of the current sliding window is more reasonable. Calculate the reasonable degree of the time length of the current sliding window ;

[0133] ;

[0134] 6. When the reasonable degree is greater than the preset reasonable threshold (which can be set according to actual needs), for example, when it is considered that the data points in the current sliding window are normally distributed. Otherwise, expand the time length of the current sliding window. For example, it can be stipulated that the sliding window expands by 10 data points at a time;

[0135] It should be noted that the maximum time length of the sliding window can be limited to . If when the time length of the sliding window expands to the upper limit and no suitable sliding window is found, the window with the largest reasonable degree is the finally adjusted sliding window of this sliding window.

[0136] Further, in one embodiment, step S3 includes:

[0137] Determine the upper and lower quartiles and the median in the box plot corresponding to the sliding window;

[0138] Use the importance level to determine the abnormal upper and lower limits in the box plot corresponding to the sliding window;

[0139] Use the upper and lower quartiles, the median, and the abnormal upper and lower limits to determine the box plot of the sliding window;

[0140] Summarize the box plots of the sliding windows to obtain the box plots of each production data;

[0141] Among them, the higher the importance level, the closer the abnormal upper and lower limits are, and vice versa, the farther the abnormal upper and lower limits are.

[0142] Among them, the step of using the importance level to determine the abnormal upper and lower limits in the box plot corresponding to the sliding window includes:

[0143] Use the importance level to calculate the upper and lower limit adjustment coefficients of the box plot corresponding to the sliding window;

[0144] Use the upper and lower quartiles in the box plot corresponding to the sliding window to calculate the inner limit range of the box plot;

[0145] Use the upper quartile, the inner limit range, and the upper and lower limit adjustment coefficients to calculate the abnormal upper limit in the box plot;

[0146] Use the lower quartile, the inner limit range, and the upper and lower limit adjustment coefficients to calculate the abnormal lower limit in the box plot.

[0147] For step S3 and each of the above specific steps:

[0148] When using a box plot to detect abnormal points in data over a period of time, since the importance levels of different production data in the required data list for the outlet nitrogen oxide concentration are different, different tolerances (corresponding to the coefficient ) can be selected for different production data when obtaining potential abnormal data points, so as to improve the richness of data points during the data cleaning process. The specific steps are as follows:

[0149] Limit the coefficients of the abnormal upper and lower limits of the box plot according to the importance levels of different production data , the higher the importance level, the closer the abnormal upper and lower limits are, and vice versa, the farther the abnormal upper and lower limits are;

[0150] Obtain the upper and lower quartiles of the box plot corresponding to the current sliding window 、 and the median ;

[0151] Calculate the inner limit range of the box plot, that is, the inner limit range ;

[0152] Obtain the upper and lower limits of outliers of the box plot 、 , where .

[0153] ;

[0154] ;

[0155] It should be noted that the interval between the upper and lower limits of outliers is not less than the inner limit range of the box plot .<s

[0156] Thus, the box plots of each production data are obtained

[0157] Step S4, use the box plot to determine the first set of potential outlier data points of each production data, and use < / s rules to determine the second set of potential outlier data points of each production data;

[0158] Step S5, determine the intersection of outlier data points between the first set and the second set, and use the intersection of outlier data points to perform data cleaning on each production data

[0159] Screen all potential outlier data points in the current sliding window according to the box plot rules to obtain the set of potential outlier data points (the first set);

[0160] At the same time, according to rules, identify outlier data points in the data points of this sliding window to obtain the set of potential outlier data points (the second set);

[0161] Compare the set of potential outlier data points with , and obtain the intersection e of its outlier data points;

[0162] Repeat the above steps to obtain the intersection e of each outlier data point of the production data;

[0163] It should be noted that since multiple sliding windows will be constructed during the process of adjusting the time length according to the reasonable degree, there are also multiple intersections e of outlier data points, and due to the different distributions of data points within the sliding window, the outlier data points are also different

[0164] Finally, by comparing and combining the intersection e of the abnormal data points of all sliding windows, the intersection of the abnormal data points of all production data items is obtained. .

[0165] Furthermore, the intersection of the abnormal data points is used to clean the production data items.

[0166] Please refer to Figure 4 , in a specific embodiment, the step S5 includes:

[0167] Step S50: Determine the number of times an abnormal data point is identified as abnormal for any target data point in the intersection of the abnormal data points and the number of windows of the sliding window containing the target data point;

[0168] Step S51: Calculate the abnormal performance degree of the target data point by using the number of abnormal times, the number of windows, and the total number of sliding windows corresponding to each production data item;

[0169] Step S52: If the abnormal performance degree is greater than the preset abnormal threshold, the target data point is marked as an abnormal point;

[0170] Step S53: Clean all the abnormal data marked as abnormal points in each production data item.

[0171] Through the above embodiments, the sets and intersections of the abnormal data points of each production data item are obtained. At this time, according to the correlation of different production data, the potential abnormal data points are cleaned. The specific steps are as follows:

[0172] 1. First, screen the intersection of the abnormal data points of the production data, and count the number of times the q-th data point is identified as abnormal as the target data point and the number of windows of the sliding window containing this data point ;

[0173] 2. Further, calculate the abnormal performance degree of the q-th data point :

[0174] ;

[0175] where m represents the total number of sliding windows.

[0176] 3. Set a preset abnormal threshold as needed to screen the potential abnormal data points. For example, when the abnormal performance degree of the q-th data point it is considered an abnormal point and it is marked;

[0177] 4. Repeat the screening steps to obtain all the abnormal data points in the current data dimension;

[0178] 5. Clean the obtained abnormal data points, for example, modify the outliers through a moving average algorithm, etc.;

[0179] 6. Clean all the production data in the SCR flue gas denitrification process through the above steps.

[0180] Thus, the data cleaning of each feature in the SCR flue gas denitrification production process is completed.

[0181] In another embodiment, after the step of calculating the abnormal performance degree of the target data point, the method further includes:

[0182] Determine the reasonable degree of the sliding window corresponding to the target data point;

[0183] Use the reasonable degree to correct the abnormal performance degree to obtain the corrected abnormal performance degree.

[0184] In this embodiment, for the reasonable degree of the sliding window where each target data point (potential abnormal data point) identified as an abnormal data is located is determined. When the reasonable degree is higher, the potential abnormal data points obtained according to the rule perform better, that is, the effect of identifying abnormal data is better, and further correct the abnormal performance degree of the q-th data point :

[0185] ;

[0186] where represents the corrected abnormal performance degree.

[0187] It should be noted that when a potential abnormal data point appears in multiple sliding windows, the mean value of the obtained reasonable degree is used as the reasonable degree .

[0188] The present invention can detect abnormal points in the production data of the SCR flue gas denitrification process by combining the box plot rule and the rule. When constructing the box plot, the rule is combined to make the distribution of data points in the sliding window more reasonable. When identifying abnormal points, the intersection of the abnormal point sets and the intersection of abnormal points in different windows are obtained to improve the accuracy of identification and the generalization ability. At the same time, according to the performance of abnormal points in different sliding windows and the distribution of data points in the window, the performance degree of abnormal points is deeply obtained to improve the accuracy of identifying abnormal data points.

[0189] In addition, for the application effects of the data-driven optimization model in the above various embodiments of the present invention, as follows:

[0190] By using the historical data optimized by cleaning to establish and optimize the model, it is possible to better solve problems such as multi-coupling, non-linearity, and large lag among data, so as to discover the relationship between data and target tags. Then, through optimization, when the working condition is stable, reverse control point setting variables are carried out to achieve the purpose of energy conservation.

[0191] The SCR modeling optimization system has a total of 2 models, the outlet NOx concentration prediction model and the ammonia injection flow prediction model; among them, the outlet NOx concentration prediction model is divided into side A, side B, and the total outlet; the ammonia injection flow prediction model is divided into side A and side B. The above models are trained and optimized on historical data, and the trained models are saved for optimization.

[0192] The opening degree of the ammonia injection valve is used as the decision variable in the optimization algorithm, that is, the set point variable that needs to be adjusted by data driving in the actual working condition. The optimization goal is that the mean is less than the environmental protection limit value. Determining the set point variable under the current working condition is the final optimization result. However, changing the set point variable will cause the change of the working condition. In order to maintain the stability of the working condition, constraint conditions need to be added. Through historical data analysis, the set point variable is optimized within a reasonable range, and the output is the optimal opening degree of the ammonia injection valve.

[0193] Embodiment 2:

[0194] The embodiment of the present invention also proposes a data cleaning device for a data-driven optimization system. The data cleaning device of the data-driven optimization system can be a data processing device such as a computer or a server.

[0195] As Figure 5 shown, Figure 5 It is a schematic structural diagram of the hardware operating environment of the data cleaning device of the data-driven optimization system involved in the embodiment of the present invention.

[0196] As Figure 5As shown, the data cleaning device of the data-driven optimization system may include: a processor 1001, such as a CPU, a network interface 1004, a user interface 1003, a memory 1005, and a communication bus 1002. Among them, the communication bus 1002 is used to achieve connection communication between these components. The user interface 1003 may include a display and an input unit such as a control panel. Optionally, the user interface 1003 may also include a standard wired interface and a wireless interface. The network interface 1004 may optionally include a standard wired interface and a wireless interface (such as a WIFI interface). The memory 1005 may be a high-speed RAM memory or a stable memory (non-volatile memory), such as a disk memory. Optionally, the memory 1005 may also be a storage device independent of the aforementioned processor 1001. As a computer storage medium, the memory 1005 may include a data cleaning program of the data-driven optimization system.

[0197] Those skilled in the art can understand that Figure 5 the hardware structure shown in does not constitute a limitation on the device, and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0198] Continuing to refer to Figure 5 , Figure 5 the memory 1005 as a computer-readable storage medium in may include an operating system, a user interface module, a network communication module, and a data cleaning program of the data-driven optimization system.

[0199] In Figure 5 , the network communication module is mainly used to connect to the server and can communicate with the server for data; while the processor 1001 can call the data cleaning program of the data-driven optimization system stored in the memory 1005 and execute the steps in each of the above embodiments.

[0200] Based on the hardware structure of the data cleaning device of the above data-driven optimization system, various embodiments for implementing the data cleaning of the data-driven optimization system of the present invention are realized.

[0201] In addition, the present invention also provides a data cleaning device for a data-driven optimization system. Please refer to Figure 6 , the data cleaning device for the data-driven optimization system includes:

[0202] a data acquisition module A10, configured to obtain various production data of different categories in the SCR flue gas denitrification system;

[0203] The box plot analysis module A20 is used to determine the importance of various production data for the outlet nitrogen oxide concentration during the SCR flue gas denitrification process; establish a sliding window for each production data, and use the importance to determine the box plot of each production data;

[0204] The data identification module A30 is used to determine the first set of potential abnormal data points of each production data by using the box plot, and use rules to determine the second set of potential abnormal data points of each production data;

[0205] The data screening module A40 is used to determine the intersection of abnormal data points between the first set and the second set, and use the intersection of abnormal data points to perform data cleaning on each production data;

[0206] Furthermore, the box plot analysis module A20 is further used for:

[0207] Determine other production data in the SCR flue gas denitrification system except the nitrogen oxide concentration in the reactor outlet flue, create a target matching window on the other production data, and create a search window on the control data;

[0208] Determine each matched window intercepted by chronological sliding in the search window, and determine the target matched window with the highest matching degree with the target matching window in each matched window;

[0209] Determine the first correlation degree between the target matching window and the target matched window;

[0210] Determine the second correlation degree between the nitrogen oxide concentration in the reactor outlet flue and the outlet nitrogen oxide concentration;

[0211] Use the first correlation degree and the second correlation degree to determine the importance of each production data for the outlet nitrogen oxide concentration.

[0212] Furthermore, the box plot analysis module A20 is further used for:

[0213] Determine the matching degree and time interval between the target matching window and the target matched window;

[0214] Use the matching degree and the time interval to determine the first correlation degree between the target matching window and the target matched window.

[0215] Furthermore, the box plot analysis module A20 is further used for:

[0216] Determine the initial size of the sliding window of each production data to obtain an initial sliding window;

[0217] Determine the skewness of the data point distribution in the initial sliding window;

[0218] Adjust the size of the initial sliding window according to the skewness so that it meets the preset normal distribution condition, and obtain the sliding window of each production data.

[0219] Furthermore, the box plot analysis module A20 is further configured to:

[0220] Calculate the reasonableness of the time length of the initial sliding window by using the skewness;

[0221] If the reasonableness is greater than the preset reasonable threshold, it is determined that the initial sliding window meets the preset normal distribution condition;

[0222] If the reasonableness is less than or equal to the preset reasonable threshold, expand the size of the initial sliding window until the reasonableness is greater than the preset reasonable threshold.

[0223] Furthermore, the box plot analysis module A20 is further configured to:

[0224] Determine the upper and lower quartiles and the median in the box plot corresponding to the sliding window;

[0225] Determine the abnormal upper and lower limits in the box plot corresponding to the sliding window by using the importance;

[0226] Determine the box plot of the sliding window by using the upper and lower quartiles, the median, and the abnormal upper and lower limits;

[0227] Summarize the box plots of the sliding windows to obtain the box plots of each production data;

[0228] Furthermore, the box plot analysis module A20 is further configured to:

[0229] Calculate the upper and lower limit adjustment coefficients of the box plot corresponding to the sliding window by using the importance;

[0230] Calculate the inner limit range of the box plot by using the upper and lower quartiles in the box plot corresponding to the sliding window;

[0231] Calculate the abnormal upper limit in the box plot by using the upper quartile, the inner limit range, and the upper and lower limit adjustment coefficients;

[0232] Calculate the abnormal lower limit in the box plot by using the lower quartile, the inner limit range, and the upper and lower limit adjustment coefficients.

[0233] Furthermore, the box plot analysis module A20 is further configured to:

[0234] Obtain the historical average duration of each denitrification treatment of the SCR flue gas denitrification system and the acquisition frequency of collecting each production data;

[0235] The initial size of the sliding window is calculated using the historical average duration and the acquisition frequency.

[0236] Furthermore, the data screening module A40 is further configured to:

[0237] Determine the number of times any target data point in the intersection of the abnormal data points is identified as abnormal data and the number of sliding windows that include the target data point;

[0238] Calculate the abnormal performance degree of the target data point using the number of abnormalities, the number of windows, and the total number of sliding windows corresponding to each production data;

[0239] If the abnormal performance degree is greater than the preset abnormal threshold, the target data point is marked as an abnormal point;

[0240] All abnormal data marked as abnormal points in various production data are cleaned.

[0241] Furthermore, the data screening module A40 is further configured to:

[0242] Determining the rationality of the sliding window corresponding to the target data point;

[0243] The abnormal performance degree is corrected using the reasonable degree to obtain a corrected abnormal performance degree.

[0244] The specific implementation of the data cleaning device of the data-driven optimization system of the present invention is basically the same as the various embodiments of the data cleaning method of the data-driven optimization system described above, and will not be repeated here.

[0245] In addition, the present invention further provides a computer-readable storage medium. The computer-readable storage medium of the present invention stores a data cleaning program for a data-driven optimization system, wherein when the data cleaning program for the data-driven optimization system is executed by a processor, the steps of the data cleaning method for the data-driven optimization system are implemented as described above.

[0246] Among them, the method implemented when the data cleaning program of the data-driven optimization system is executed can refer to the various embodiments of the data cleaning method of the data-driven optimization system of the present invention, and will not be repeated here.

[0247] It should be noted that the order in which the embodiments of the present invention are described above is for illustrative purposes only and does not necessarily represent the superiority or inferiority of the embodiments. The processes depicted in the accompanying drawings do not necessarily require the specific order or sequential order shown to achieve the desired results. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0248] Each embodiment in this specification is described in a progressive manner. For the same or similar parts among the embodiments, reference can be made to each other, and the key point of each embodiment is to illustrate the differences from other embodiments.

[0249] Those skilled in the art should understand that the embodiments of the present invention can be provided as methods, devices, or computer program products. Therefore, the present invention can be implemented in the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present invention can be implemented in the form of a computer program product on one or more computer-usable storage media (including but not limited to disk memories, CD-ROMs, optical memories, etc.) containing computer-usable program codes.

[0250] The above are only the preferred embodiments of the present invention, and do not limit the protection scope of the present invention. Any equivalent structural transformation made using the content of the specification and drawings of the present invention under the inventive concept of the present invention, or any direct / indirect application in other related technical fields, is included in the protection scope of the present invention.

Claims

1. A data cleaning method for a data-driven optimization system, characterized in that: The method comprises: Obtain various types of production data in the SCR flue gas denitrification system; Determine the importance of various production data to the outlet nitrogen oxide concentration during the SCR flue gas denitrification process; Establishing a sliding window for each production data, and determining a box plot of each production data using the importance; The box plot is used to determine the first set of potential abnormal data points of each production data, and the The rule determines a second set of potentially abnormal data points for each production data item; Determine an intersection of abnormal data points between the first set and the second set, and perform data cleaning on each production data using the intersection of abnormal data points; Wherein, the window represents a data set, and the sliding window represents a variable data set of movable time series; The various production data include the concentration of nitrogen oxides in the flue gas at the reactor outlet, and the concentration of nitrogen oxides in the flue gas at the reactor outlet is used as control data; The step of determining the importance of various production data to the outlet nitrogen oxide concentration during the SCR flue gas denitrification process includes: Determine other production data in the SCR flue gas denitrification system except for the concentration of nitrogen oxides in the flue gas at the reactor outlet, create a target matching window on the other production data, and create a search window on the reference data; Determine each matched window intercepted by sliding in sequence in the search window, and determine a target matched window with the highest matching degree among each matched window; Determining a first correlation between the target matching window and the target matched window; determining a second correlation between the flue gas nitrogen oxide concentration at the reactor outlet and the outlet nitrogen oxide concentration; Determining the importance of each production data to the outlet nitrogen oxide concentration by using the first correlation and the second correlation; The step of establishing a sliding window for each production data includes: Determining an initial size of a sliding window for each production data to obtain an initial sliding window; determining the skewness of the distribution of data points in the initial sliding window; According to the skewness, the size of the initial sliding window is adjusted to make it conform to a preset normal distribution condition, thereby obtaining a sliding window of each production data; The step of establishing a sliding window for each production data and determining a box plot of each production data using the importance includes: Determine the upper and lower quartiles and median of the box plot corresponding to the sliding window; Using the importance, determine the upper and lower limits of anomalies in the box plot corresponding to the sliding window; Use the upper and lower quartiles, median, and upper and lower anomaly limits to determine the box plot of the sliding window; Summarize the box plots of the sliding window to obtain the box plots of each production data; The higher the importance, the closer the upper and lower limits of the anomaly are, and vice versa.

2. The data cleaning method of the data-driven optimization system according to claim 1, characterized in that: The step of determining a first correlation between the target matching window and the target matched window includes: Determine the matching degree and time interval between the target matching window and the target matched window; A first correlation between the target matching window and the target matched window is determined using the matching degree and the time interval.

3. The data cleaning method of the data-driven optimization system according to claim 1, characterized in that: The step of determining an initial size of a sliding window for each production data to obtain an initial sliding window includes: Obtain the historical average duration of each denitrification treatment of the SCR flue gas denitrification system and the frequency of collecting various production data; The initial size of the sliding window is calculated using the historical average duration and the acquisition frequency.

4. The data cleaning method of the data-driven optimization system according to claim 1, characterized in that: The step of adjusting the size of the initial sliding window according to the skewness to make it conform to a preset normal distribution condition, and obtaining a sliding window for each production data includes: Calculating the rationality of the time length of the initial sliding window using the skewness; If the rationality is greater than a preset rationality threshold, it is determined that the initial sliding window meets the preset normal distribution condition; If the reasonableness is less than or equal to a preset reasonable threshold, the size of the initial sliding window is expanded until the reasonableness is greater than the preset reasonable threshold.

5. The data cleaning method of the data-driven optimization system according to claim 1, characterized in that: The step of determining the upper and lower limits of anomalies in the box plot corresponding to the sliding window using the importance includes: Using the importance, calculating the upper and lower limit adjustment coefficients of the box plot corresponding to the sliding window; The inner limit of the box plot is calculated using the upper and lower quartiles of the box plot corresponding to the sliding window; The upper quartile, the inner limit range and the upper and lower limit adjustment coefficients are used to calculate the upper limit of the abnormality in the box plot; The lower quartile, the inner limit range, and the upper and lower limit adjustment coefficients are used to calculate the lower abnormal limit in the box plot.

6. The data cleaning method of the data-driven optimization system according to claim 1, characterized in that: The step of performing data cleaning on each production data using the intersection of the abnormal data points includes: Determine the number of times any target data point in the intersection of the abnormal data points is identified as abnormal data and the number of sliding windows that include the target data point; Calculate the abnormal performance degree of the target data point using the number of abnormalities, the number of windows, and the total number of sliding windows corresponding to each production data; If the abnormal performance degree is greater than the preset abnormal threshold, the target data point is marked as an abnormal point; All abnormal data marked as abnormal points in various production data are cleaned.

7. The data cleaning method of the data-driven optimization system according to claim 6, characterized in that: After the step of calculating the abnormal performance degree of the target data point, the method further includes: Determine the rationality of the sliding window corresponding to the target data point. , Represents the skewness of the distribution of data points in the sliding window; The abnormal performance degree is corrected using the reasonable degree to obtain a corrected abnormal performance degree.

Images