Data cleaning method of data-driven optimization system

Abstract

The invention relates to the technical field of electrical digital data processing, in particular to a data cleaning method for a data-driven optimization system, and the method comprises the steps: obtaining different types of production data in an SCR flue gas denitration system; determining the importance degree of each piece of production data on the concentration of the nitric oxide at the outlet in the SCR flue gas denitration process; establishing a sliding window for each piece of production data, and determining a box plot of each piece of production data by using the importance degree; determining a first set of potential abnormal data points of each piece of production data by using a box plot, and determining a second set of potential abnormal data points of each piece of production data by using a rule; determining an abnormal data point intersection between the first set and the second set, and performing data cleaning on each production data by using the abnormal data point intersection; by means of the data cleaning method of the data-driven optimization system, the accuracy degree of abnormal data identification can be improved, and the generalization ability can be improved.

Description

Technical Field

[0001] The present invention relates to the technical field of electronic digital data processing, and in particular to a data cleaning method for a data driven optimization system. Background Art

[0002] In order to solve the environmental problems caused by coal-fired flue gas emissions, among the many flue gas denitrification technologies, the SCR (Selective Catalytic Reduction) denitrification system has been widely used in coal-fired power plants due to its mature technology, easy installation, high denitrification efficiency, and almost no secondary pollution. The SCR flue gas denitrification process is a complex physical and chemical process with characteristics such as nonlinearity and time lag. It is difficult to describe these complex processes using mechanism models.

[0003] At present, a new system optimization method has been created through data-driven optimization technology. Using the large amount of production data generated by the control system, data engineers use data science technology to achieve data-driven optimization in the SCR flue gas denitrification process. When performing data-driven optimization, a large amount of production data generated by the control system will first be collected. If the collected production data contains abnormal values, the large difference between abnormal values ​​and normal values ​​will greatly reduce the generalization ability of the model, increase the burden of data processing, and reduce the analysis efficiency. Therefore, data cleaning is particularly important for data-driven optimization technology.

[0004] Existing data cleaning methods usually use box plots to detect outliers in data, and use quartiles to judge abnormal data to achieve data cleaning. However, the production data in the SCR flue gas denitrification process has nonlinear and time-lag characteristics, which may lead to uneven distribution of production data in time, and due to the influence of time lag, current data may be affected by historical data, which will eventually lead to inaccurate identification of abnormal data and affect the effect of data cleaning. Summary of the invention

[0005] In order to solve the current technical problems that the recognition of abnormal data in the SCR flue gas denitrification system is not accurate enough and the data cleaning effect is poor, the purpose of the present invention is to provide a data cleaning method for a data-driven optimization system. The technical solution adopted is as follows: The present invention provides a data cleaning method for a data-driven optimization system, the method comprising: Obtain various types of production data in the SCR flue gas denitrification system; Determine the importance of various production data to the outlet nitrogen oxide concentration during the SCR flue gas denitrification process; Establishing a sliding window for each production data, and determining a box plot of each production data using the importance; The box plot is used to determine a first set of potential abnormal data points for each production data, and The rule determines a second set of potentially abnormal data points for each production data; Determine an intersection of abnormal data points between the first set and the second set, and use the intersection of abnormal data points to perform data cleaning on each production data; The window represents a data set, and the sliding window represents a variable data set of movable time series.

[0006] Furthermore, various production data include the concentration of nitrogen oxides in the flue gas at the reactor outlet, and the concentration of nitrogen oxides in the flue gas at the reactor outlet is used as control data; The steps to determine the importance of various production data to the outlet nitrogen oxide concentration in the SCR flue gas denitrification process include: Determine other production data in the SCR flue gas denitrification system except for the concentration of nitrogen oxides in the flue gas at the reactor outlet, create a target matching window on the other production data, and create a search window on the control data; Determine each matched window captured by sliding in the search window in a sequential manner, and determine a target matched window with the highest matching degree with the target matching window among each matched window; Determine a first correlation between the target matching window and the target matched window; Determining a second correlation between the reactor outlet flue gas nitrogen oxide concentration and the outlet nitrogen oxide concentration; The importance of each production data to the outlet nitrogen oxide concentration is determined by using the first correlation and the second correlation.

[0007] Furthermore, the step of determining a first correlation between the target matching window and the target matched window includes: Determine the matching degree and time interval between the target matching window and the target matched window; A first correlation between the target matching window and the target matched window is determined by using the matching degree and the time interval.

[0008] Furthermore, the step of establishing a sliding window for each production data includes: Determine the initial size of the sliding window of each production data to obtain an initial sliding window; Determining the skewness of the distribution of data points in the initial sliding window; According to the skewness, the size of the initial sliding window is adjusted to make it meet the preset normal distribution conditions, and a sliding window of each production data is obtained.

[0009] Furthermore, the step of determining the initial size of the sliding window of each production data to obtain the initial sliding window includes: Obtain the historical average duration of each denitration treatment of the SCR flue gas denitrification system and the frequency of collecting various production data; The initial size of the sliding window is calculated using the historical average duration and the acquisition frequency.

[0010] Furthermore, according to the skewness, the size of the initial sliding window is adjusted to make it meet the preset normal distribution condition, and the step of obtaining the sliding window of each production data includes: Using the skewness, calculating the rationality of the time length of the initial sliding window; If the rationality is greater than a preset rationality threshold, it is determined that the initial sliding window meets the preset normal distribution condition; If the reasonable degree is less than or equal to a preset reasonable threshold, the size of the initial sliding window is enlarged until the reasonable degree is greater than the preset reasonable threshold.

[0011] Furthermore, the step of establishing a sliding window for each production data and determining a box plot of each production data using the importance includes: Determine the upper and lower quartiles and median in the box plot corresponding to the sliding window; Using the importance, determine the upper and lower limits of the abnormality in the box plot corresponding to the sliding window; Use the upper and lower quartiles, median, and upper and lower abnormal limits to determine the box plot of the sliding window; Summarize the box plots of the sliding window to obtain the box plots of each production data; The higher the importance, the closer the upper and lower limits of the abnormality are, and vice versa.

[0012] Further, the step of determining the upper and lower limits of the abnormality in the box plot corresponding to the sliding window by using the importance includes: Using the importance, calculating the upper and lower limit adjustment coefficients of the box plot corresponding to the sliding window; The inner limit range of the box plot is calculated by using the upper and lower quartiles in the box plot corresponding to the sliding window; The upper quartile, the inner limit range and the upper and lower limit adjustment coefficients are used to calculate the upper limit of the abnormality in the box plot; The lower quartile, the inner limit range and the upper and lower limit adjustment coefficients are used to calculate the abnormal lower limit in the box plot.

[0013] Furthermore, the step of using the intersection of the abnormal data points to clean the various production data includes: Determine the number of abnormal times that any target data point in the intersection of the abnormal data points is identified as abnormal data and the number of sliding windows that include the target data point; The abnormal performance degree of the target data point is calculated by using the number of abnormalities, the number of windows and the total number of sliding windows corresponding to each production data; If the abnormal performance degree is greater than a preset abnormal threshold, the target data point is marked as an abnormal point; All abnormal data marked as abnormal points in various production data are cleaned.

[0014] Furthermore, after the step of calculating the abnormal performance degree of the target data point, the method further includes: Determining the rationality of the sliding window corresponding to the target data point; The abnormal performance degree is corrected using the reasonable degree to obtain a corrected abnormal performance degree.

[0015] The present invention has the following beneficial effects: The purpose of the present invention is to combine The production data in the SCR flue gas denitrification process is cleaned by using rules and box plot rules. Since the production data has the characteristics of nonlinearity and time lag, the abnormal data is obtained in different dimensions to improve the effect of data cleaning and avoid the deletion of normal data.

[0016] The present invention can combine the box plot rule and Rules, detect abnormal points in the production data of SCR flue gas denitrification production process, and combine them when constructing the box plot The rule makes the distribution of data points in the sliding window more reasonable. When identifying abnormal points, the intersection of the abnormal point set and the intersection of abnormal points in different windows are obtained to improve the accuracy of identification and generalization ability. At the same time, according to the performance of abnormal points in different sliding windows and the distribution of data points in the window, the performance of abnormal points is deeply obtained to improve the accuracy of identifying abnormal data points. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions and advantages in the embodiments of the present invention or the prior art, the drawings required for use in the embodiments or the prior art descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0018] Figure 1 A flowchart of a data cleaning method for a data-driven optimization system provided by an embodiment of the present invention; Figure 2 A detailed flow chart of step S2 in a data cleaning method for a data-driven optimization system provided by an embodiment of the present invention; Figure 3 A detailed flow chart of step S3 in a data cleaning method for a data-driven optimization system provided by another embodiment of the present invention; Figure 4 A detailed flow chart of step S5 in a data cleaning method for a data-driven optimization system provided by an embodiment of the present invention; Figure 5 A schematic diagram of the structure of the hardware operating environment of a data cleaning device of a data-driven optimization system involved in an embodiment of the present invention; Figure 6 It is a schematic diagram of the framework structure of a data cleaning device of a data-driven optimization system involved in an embodiment of the present invention. DETAILED DESCRIPTION

[0019] In order to further explain the technical means and effects adopted by the present invention to achieve the predetermined invention purpose, the data cleaning method of a data-driven optimization system proposed by the present invention, its specific implementation method, structure, features and effects are described in detail as follows in combination with the accompanying drawings and preferred embodiments. In the following description, different "one embodiment" or "another embodiment" does not necessarily refer to the same embodiment. In addition, specific features, structures or characteristics in one or more embodiments may be combined in any suitable form.

[0020] Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.

[0021] The following is a detailed description of a data cleaning method for a data-driven optimization system provided by the present invention in conjunction with the accompanying drawings.

[0022] Embodiment 1: For a data cleaning method for a data-driven optimization system provided by the present invention, please refer to Figure 1 , which shows a flow chart of the steps of a data cleaning method of a data-driven optimization system provided by an embodiment of the present invention.

[0023] The method comprises: Step S1, obtaining various types of production data in the SCR flue gas denitrification system; In this embodiment, the specific scenarios targeted are: In the process of data cleaning of the production data in the SCR flue gas denitrification process, since the chemical reaction in the treatment process takes a certain amount of time to reach chemical equilibrium, there is a lag between the actual chemical reaction rate and the theoretical prediction. At the same time, the chemical reagents require a certain amount of time to be transported and diffused after they are initially put into use, which further leads to a lag in the changes in production data.

[0024] In summary, the timestamp corresponding to the operation does not completely correspond to the characteristics of the production data, which may cause the algorithm to identify the normally changing data as abnormally changing data during cleaning, and then mistakenly delete or modify it. At the same time, since the production data itself is a nonlinear changing data, this makes it impossible for the algorithm to distinguish whether the data is normal or abnormal according to the law of data change, further increasing the difficulty of cleaning. Therefore, in order to achieve data-driven optimization of the SCR flue gas denitrification system, a method is needed to more accurately achieve the cleaning of the production data of the SCR flue gas denitrification process.

[0025] The main production data involved in the SCR denitration system include: nitrogen oxide content, oxygen content, ammonia content, temperature, flow, pressure, etc. According to the required data type, the corresponding sensor is set and the production data is collected. The specific production data included is shown in Table 1 below: Table 1

[0026] It should be noted that the header of Table 1 includes: serial number, variable name, unit and variable type. The serial number in the table indicates the arrangement serial number of different variable names in Table 1. Different variable names represent different categories of production data. As shown above, there are 26 production data, but it is not limited to these 26 items. There can be more or less than 26 items. The unit is the unit of each production data. The variable type is used to determine whether different production data items are controllable variables or uncontrollable variables.

[0027] As for how to obtain the above-mentioned production data of different categories in the SCR flue gas denitrification system, the following links can be included: 1. First, install the required sensors at the corresponding positions of the SCR flue gas denitrification system (hereinafter referred to as the "system"), such as installing nitrogen oxide sensors, oxygen sensors, ammonia sensors, etc. at the inlet and outlet of the SCR reactor; 2. Check the sensors on the SCR flue gas denitrification system before denitrification work to ensure the accuracy of the sensors during data collection; 3. Then, set the data collection parameters of the sensor, including the data collection frequency, for example, set the data collection frequency of all sensors to 5 seconds / time; 4. Start the SCR flue gas denitrification system and sensor, and introduce the flue gas generated by the combustion of coal into the SCR flue gas denitrification system; 5. Use sensors to collect production data during the system operation process and store the collected data in a designated area of ​​the system.

[0028] At this point, the collection of production data during the SCR flue gas denitrification process is completed.

[0029] Step S2, determining the importance of various production data to the outlet nitrogen oxide concentration during the SCR flue gas denitrification process; In the process of data cleaning of SCR flue gas denitrification production data, the importance of each production data is different, and the accuracy required for data cleaning is different. Based on the operation purpose of the system, it can be known that "reactor outlet flue NOx" (reactor outlet flue nitrogen oxide concentration) has a higher importance to the outlet nitrogen oxide concentration, which is used as a reference and recorded as Therefore, by obtaining the correlation between different production data and "reactor outlet flue NOx", the importance of other different production data to the outlet nitrogen oxide concentration can be quantified. The importance here can be considered as the degree of influence on the outlet nitrogen oxide concentration.

[0030] Please refer to Figure 2 In a specific embodiment, the various production data include the concentration of nitrogen oxides in the flue gas at the reactor outlet, and the concentration of nitrogen oxides in the flue gas at the reactor outlet is used as the reference data; the step S2 includes: Step S20, determining other production data in the SCR flue gas denitration system except the concentration of nitrogen oxides in the flue gas at the reactor outlet, creating a target matching window on the other production data, and creating a search window on the control data; Step S21, determining each matched window intercepted by sliding in the search window in sequence, and determining a target matched window with the highest matching degree with the target matching window among each matched window; Step S22, determining a first correlation between the target matching window and the target matched window; Specifically, the step S22 includes: Determine the matching degree and time interval between the target matching window and the target matched window; A first correlation between the target matching window and the target matched window is determined by using the matching degree and the time interval.

[0031] Step S23, determining a second correlation between the flue gas nitrogen oxide concentration at the reactor outlet and the outlet nitrogen oxide concentration; Step S24: using the first correlation and the second correlation, determine the importance of each production data to the outlet nitrogen oxide concentration.

[0032] 1. First, select one dimension from the required data list in Table 1 above , represents an item of other production data, and determines the production node type of the data, i.e., input data, output data, and process data (temperature, load, etc.); 2. Then, according to the production node type of the selected production data, you can choose to build a matching window or a search window on the time series data set of the production data; it should be noted that since each production data is collected continuously over time, its data set form can be a time series data set, that is, each data in each production data corresponds to its collection time. The window can be considered as a part of the data set intercepted from the complete time series data set. The matching window and the search window are relative concepts, that is, finding the matched window that best matches the matching window from the search window, which also means that the time length of the search window, that is, the size of the search window, is generally larger than the size of the matching window.

[0033] According to the time lag characteristics of production data, when the selected type is the input end, a matching window is constructed, otherwise when the type is the output end or process data, a search window is constructed, and "reactor outlet flue NOx" is constructed as a search window. In the following, other production data are used to create matching windows, and "reactor outlet flue NOx" is used to construct a search window to illustrate this embodiment.

[0034] 3. Construct a fixed window (target matching window) on the time series data set, and specify the length of the target matching window as ,in Indicates the historical average duration of a denitrification treatment; 4. Further, according to the length of the fixed window, a corresponding window (search window) is set on the reference data dimension (here it can refer to "reactor outlet flue NOx"), and the search window size is specified as ; It should be noted that the matching window and the search window have the same time starting point, and there is also a window (matched window) of the same size as the matching window in the search window, which is used for similarity analysis with the matching window.

[0035] 5. Then, calculate the matching degree (matching degree) between the target matching window and the search window (Pearson correlation coefficient); 6. Move the matched window in the search window in sequence, for example, set the sliding step of the window to 10 data points (customizable), and calculate the matching degree again ; It is relatively easy to understand. For example, the data starting point of the first matched window corresponds to the data starting point of the search window, that is, the first data point, and the data starting point of the second matched window corresponds to the 11th data point of the search window. The order here is in time sequence, that is, the order of collection time.

[0036] 7. Repeat the above steps until the search window is matched, select the target matched window with the highest matching degree with the target matching window, mark the time corresponding to the target matched window and obtain the time interval between it and the target matching window. The time interval can be calculated as the acquisition time difference corresponding to the data starting point or the respective data end point between the matching window and the matched window, that is, the data lag time ; 8. Match each production data in the required data list with the "reactor outlet flue NOx" to obtain the lag time of all data dimensions (all different categories of production data) ; 9. Further, calculate any production data and Correlation, lag time The shorter the target matching window is, the better the match between the target matching window and the target matched window is. The higher the correlation (first correlation) The higher; ; 10. Based on the correlation between other different production data and "reactor outlet flue NOx" , to obtain the importance of different production data on the outlet NOx concentration : ; in, It indicates the proportion of NOx concentration at the reactor outlet flue to the total emissions, that is, the correlation between "NOx at the reactor outlet flue" and the outlet nitrogen oxide concentration (the second correlation).

[0037] Finally, the importance of different production data on the outlet NOx concentration is obtained. .

[0038] Step S3, establishing a sliding window for each production data, and determining a box plot of each production data using the importance; The sliding window represents a variable data set of movable time series. That is, the time length of the window, i.e., the data set whose window size can be changed. It is easy to understand that the longer the time length, the more data points it corresponds to. Each data point in each production data can establish a corresponding sliding window, so each data point can be included in multiple sliding windows.

[0039] Please refer to Figure 3 In a specific embodiment, the step S3 includes: Step S30, determining the initial size of the sliding window of each production data to obtain an initial sliding window; Specifically, the step S30 includes: Obtain the historical average duration of each denitration treatment of the SCR flue gas denitrification system and the frequency of collecting various production data; The initial size of the sliding window is calculated using the historical average duration and the acquisition frequency.

[0040] Step S31, determining the skewness of the distribution of data points in the initial sliding window; Step S32, according to the skewness, adjusting the size of the initial sliding window to make it meet the preset normal distribution condition, and obtaining the sliding window of each production data.

[0041] Specifically, the step S32 includes: Using the skewness, calculating the rationality of the time length of the initial sliding window; If the rationality is greater than a preset rationality threshold, it is determined that the initial sliding window meets the preset normal distribution condition; If the reasonable degree is less than or equal to a preset reasonable threshold, the size of the initial sliding window is enlarged until the reasonable degree is greater than the preset reasonable threshold.

[0042] For step S3 and the above specific steps: Since the production data of the SCR denitrification system are all nonlinear and time-lag, it is not possible to directly detect abnormal points in the overall data samples during data cleaning. In order to eliminate the time-lag characteristics of time series data, a sliding window is set for each production data to reduce the impact of historical data samples on data cleaning. The rule is applicable to the scenario where the data samples (the data samples mentioned in the embodiment are the data sets of each production data history) are normally distributed. Therefore, when dividing the sliding window, it is also necessary to ensure that the data set of each production data is normally distributed as much as possible. The specific steps for constructing the sliding window are: 1. First, select any production data and use the sliding average algorithm to supplement the missing values ​​of the data sample. That is, at the location where there is missing data, calculate the mean of multiple (for example, five) data points on the left and right of the missing data point, and use the mean as the data of this missing data point; 2. Then, according to the sensor's acquisition frequency And the historical average duration of a denitrification treatment (in seconds), set the initial size of the sliding window (time length) ; ; 3. Further, statistics are collected on the data in the current sliding window (which may be the initial sliding window or the sliding window after the initial sliding window is adjusted in time length) to construct a two-dimensional coordinate system (the horizontal axis is the size of the data value, and the vertical axis is the number of data points); 4. Calculate the mean of the data values ​​in the current sliding window and the skewness of the distribution of data points (The mean and skewness here are existing mathematical concepts and will not be elaborated on here); It should be noted that when When (approaches), it means that the data distribution of the current sliding window is symmetrical, that is, its data points are normally distributed. Otherwise, if it is less than 0, it is left-skewed (the left tail is longer), and if it is greater than 0, it is right-skewed (the right tail is longer).

[0043] 5. When the absolute value of the skewness of the sliding window data point is smaller, the normal distribution of the data in the current sliding window is better, and the length of the current sliding window is more reasonable. Calculate the reasonableness of the current sliding window time length ; ; 6. When the reasonableness is greater than the preset reasonable threshold (which can be set according to actual needs), for example, When the data points in the current sliding window are normally distributed, it is considered that the data points in the current sliding window are normally distributed. Otherwise, the time length of the current sliding window is expanded. For example, the sliding window can be expanded by 10 data points at a time. It should be noted that the maximum time length of the sliding window can be limited to If the time length of the sliding window is extended to the upper limit and no suitable sliding window is found, the reasonable degree The largest window is the sliding window after the sliding window is finally adjusted.

[0044] Furthermore, in one embodiment, step S3 includes: Determine the upper and lower quartiles and median in the box plot corresponding to the sliding window; Using the importance, determine the upper and lower limits of the abnormality in the box plot corresponding to the sliding window; Use the upper and lower quartiles, median, and upper and lower abnormal limits to determine the box plot of the sliding window; Summarize the box plots of the sliding window to obtain the box plots of each production data; The higher the importance, the closer the upper and lower limits of the abnormality are, and vice versa.

[0045] The step of determining the upper and lower limits of the abnormality in the box plot corresponding to the sliding window by using the importance includes: Using the importance, calculating the upper and lower limit adjustment coefficients of the box plot corresponding to the sliding window; The inner limit range of the box plot is calculated by using the upper and lower quartiles in the box plot corresponding to the sliding window; Using the upper quartile, the inner limit range and the upper and lower limit adjustment coefficients, the upper limit of the abnormality in the box plot is calculated; The lower quartile, the inner limit range and the upper and lower limit adjustment coefficients are used to calculate the abnormal lower limit in the box plot.

[0046] For step S3 and the above specific steps: When using the box plot to detect abnormal points in the data over a period of time, since different production data in the required data list have different importance to the outlet nitrogen oxide concentration, different tolerances can be selected for different production data when obtaining potential abnormal data points (corresponding to the coefficients below). ), improve the richness of data points during data cleaning. The specific steps are as follows: The coefficients of the upper and lower limits of the abnormality of the box plot according to the importance of different production data The higher the importance, the closer the upper and lower limits of the abnormality are, and vice versa, the farther the upper and lower limits of the abnormality are; Get the upper and lower quartiles of the box plot corresponding to the current sliding window , and the median ; Calculate the inner limit range of the box plot, that is, the inner limit range ; Get the upper and lower limits of the abnormal box plot , ,in .

[0047] ; ; It should be noted that the interval between the upper and lower limits of the abnormality should not be less than the inner limit range of the box plot. .

[0048] Thus, the box plot of each production data is obtained.

[0049] Step S4, using the box plot to determine a first set of potential abnormal data points for each production data, and using The rule determines a second set of potentially abnormal data points for each production data; Step S5, determining the intersection of abnormal data points between the first set and the second set, and using the intersection of abnormal data points to perform data cleaning on each production data.

[0050] According to the box plot rules, all potential abnormal data points in the current sliding window are filtered to obtain a set of potential abnormal data points. (First set); At the same time, according to The rule identifies the data points in the sliding window as outliers and obtains a set of potential outlier data points. (Second set); Comparing sets of potentially anomalous data points and , get the intersection e of its abnormal data points; Repeat the above steps to obtain the intersection e of each abnormal data point of the production data; It should be noted that since multiple sliding windows are constructed in the process of adjusting the time length according to a reasonable degree, there are multiple intersections e of abnormal data points, and due to different distributions of data points in the sliding window, the abnormal data points are also different.

[0051] Finally, compare and combine the intersection e of abnormal data points of all sliding windows to obtain the total intersection of abnormal data points of each production data .

[0052] Then, using the intersection of the abnormal data points Perform data cleaning on various production data.

[0053] Please refer to Figure 4 In a specific embodiment, the step S5 includes: Step S50, determining the number of abnormal times that any target data point in the intersection of the abnormal data points is identified as abnormal data and the number of sliding windows that include the target data point; Step S51, using the number of abnormalities, the number of windows and the total number of sliding windows corresponding to each production data, calculate the abnormal performance degree of the target data point; Step S52, if the abnormal performance degree is greater than a preset abnormal threshold, the target data point is marked as an abnormal point; Step S53: clean all abnormal data marked as abnormal points in each production data.

[0054] Through the above embodiments, the abnormal data point set and intersection of each production data are obtained. At this time, the potential abnormal data points are cleaned according to the correlation of different production data. The specific steps are as follows: 1. First, filter the intersection of abnormal data points in the production data and count the number of times the qth data point is identified as an abnormality as the target data point. And the number of sliding windows that contain the data point ; 2. Further, calculate the degree of abnormal performance of the qth data point : ; Where m represents the total number of sliding windows.

[0055] 3. Set the preset abnormal threshold as needed to filter potential abnormal data points. For example, the abnormal performance degree of the qth data point can be It is considered as an abnormal point and marked; 4. Repeat the screening steps to obtain all abnormal data points in the current data dimension; 5. Perform data cleaning on the obtained abnormal data points, such as modifying the outliers through the sliding average algorithm; 6. All production data in the SCR flue gas denitrification process are cleaned through the above steps.

[0056] At this point, the data cleaning of various characteristics in the SCR flue gas denitrification production process has been completed.

[0057] In another embodiment, after the step of calculating the abnormal performance degree of the target data point, the method further includes: Determining the rationality of the sliding window corresponding to the target data point; The abnormal performance degree is corrected using the reasonable degree to obtain a corrected abnormal performance degree.

[0058] In this embodiment, the reasonableness of the sliding window where each target data point (potential abnormal data point) identified as abnormal data is located is Determine, when the reasonableness is higher, according to The better the performance of the potential abnormal data points obtained by the rule, that is, the better the effect of identifying abnormal data, the further the abnormal performance of the qth data point is corrected. : ; in, Indicates the degree of abnormal performance after correction.

[0059] It should be noted that when a potential abnormal data point appears in multiple sliding windows, the average of the reasonable degree is the reasonable degree. .

[0060] The present invention can combine the box plot rule and Rules, detect abnormal points in the production data of SCR flue gas denitrification production process, and combine them when constructing the box plot The rule makes the distribution of data points in the sliding window more reasonable. When identifying abnormal points, the intersection of the abnormal point set and the intersection of abnormal points in different windows are obtained to improve the accuracy of identification and generalization ability. At the same time, according to the performance of abnormal points in different sliding windows and the distribution of data points in the window, the performance of abnormal points is deeply obtained to improve the accuracy of identifying abnormal data points.

[0061] In addition, the application effects of the data-driven optimization model in the above-mentioned embodiments of the present invention are as follows: Using the historical data after cleaning and optimization to establish and optimize the model can better solve the problems of multi-coupling, nonlinearity, large lag, etc. between data, so as to discover the relationship between data and target labels, and then optimize the working conditions to stabilize, and set the variables at the reverse control point to achieve energy saving.

[0062] The SCR modeling optimization system has two models, the outlet NOx concentration prediction model and the ammonia injection flow prediction model; the outlet NOx concentration prediction model is divided into side A, side B and total outlet; the ammonia injection flow prediction model is divided into side A and side B. The above models are trained and tuned on historical data, and the training models are saved for optimization.

[0063] The ammonia injection valve opening is the decision variable in the optimization algorithm, that is, the set point variable that needs to be adjusted through data-driven in actual working conditions. The optimization goal is that the mean value is less than the environmental protection limit value. The set point variable determined under the current working conditions is the final optimization result. However, changing the set point variable will cause the working conditions to change. In order to maintain the stability of the working conditions, constraints need to be added. Through historical data analysis, the set point variable is optimized within a reasonable range, and the output is the optimal ammonia injection valve opening.

[0064] Embodiment 2: The embodiment of the present invention also provides a data cleaning device for a data-driven optimization system. The data cleaning device for the data-driven optimization system can be a data processing device such as a computer or a server.

[0065] like Figure 5 As shown, Figure 5 It is a structural diagram of the hardware operating environment of the data cleaning device of the data-driven optimization system involved in the embodiment of the present invention.

[0066] like Figure 5As shown, the data cleaning device of the data-driven optimization system may include: a processor 1001, such as a CPU, a network interface 1004, a user interface 1003, a memory 1005, and a communication bus 1002. Among them, the communication bus 1002 is used to realize the connection and communication between these components. The user interface 1003 may include a display (Display), an input unit such as a control panel, and the optional user interface 1003 may also include a standard wired interface and a wireless interface. The network interface 1004 may optionally include a standard wired interface and a wireless interface (such as a WIFI interface). The memory 1005 may be a high-speed RAM memory or a stable memory (non-volatile memory), such as a disk memory. The memory 1005 may also be a storage device independent of the aforementioned processor 1001. The memory 1005 as a computer storage medium may include a data cleaning program of the data-driven optimization system.

[0067] Those skilled in the art will understand that Figure 5 The hardware structure shown in the figure does not constitute a limitation of the device, and may include more or less components than shown in the figure, or combine certain components, or arrange the components differently.

[0068] Continue to refer to Figure 5 , Figure 5 The memory 1005 as a computer-readable storage medium may include an operating system, a user interface module, a network communication module, and a data cleaning program of the data-driven optimization system.

[0069] exist Figure 5 In the embodiment, the network communication module is mainly used to connect to the server and can communicate data with the server; and the processor 1001 can call the data cleaning program of the data-driven optimization system stored in the memory 1005 and execute the steps in the above embodiments.

[0070] The hardware structure of the data cleaning device based on the above-mentioned data-driven optimization system is used to implement various embodiments of data cleaning of the data-driven optimization system of the present invention.

[0071] In addition, the present invention also provides a data cleaning device for a data driven optimization system, please refer to Figure 6 , the data cleaning device of the data driven optimization system comprises: Data acquisition module A10 is used to obtain various types of production data in the SCR flue gas denitrification system; The box plot analysis module A20 is used to determine the importance of various production data to the outlet nitrogen oxide concentration during the SCR flue gas denitrification process; establish a sliding window for each production data, and determine the box plot of each production data using the importance; The data identification module A30 is used to determine a first set of potential abnormal data points of each production data using the box plot, and use The rule determines a second set of potentially abnormal data points for each production data; A data screening module A40, configured to determine an intersection of abnormal data points between the first set and the second set, and use the intersection of abnormal data points to perform data cleaning on each production data; Furthermore, the box plot analysis module A20 is also used for: Determine other production data in the SCR flue gas denitrification system except for the concentration of nitrogen oxides in the flue gas at the reactor outlet, create a target matching window on the other production data, and create a search window on the control data; Determine each matched window captured by sliding in the search window in a sequential manner, and determine a target matched window with the highest matching degree with the target matching window among each matched window; Determine a first correlation between the target matching window and the target matched window; Determining a second correlation between the reactor outlet flue gas nitrogen oxide concentration and the outlet nitrogen oxide concentration; The importance of each production data to the outlet nitrogen oxide concentration is determined by using the first correlation and the second correlation.

[0072] Furthermore, the box plot analysis module A20 is also used for: Determine the matching degree and time interval between the target matching window and the target matched window; A first correlation between the target matching window and the target matched window is determined by using the matching degree and the time interval.

[0073] Furthermore, the box plot analysis module A20 is also used for: Determine the initial size of the sliding window of each production data to obtain an initial sliding window; Determining the skewness of the distribution of data points in the initial sliding window; According to the skewness, the size of the initial sliding window is adjusted to make it meet the preset normal distribution conditions, and a sliding window of each production data is obtained.

[0074] Furthermore, the box plot analysis module A20 is also used for: Using the skewness, calculating the rationality of the time length of the initial sliding window; If the rationality is greater than a preset rationality threshold, it is determined that the initial sliding window meets the preset normal distribution condition; If the reasonable degree is less than or equal to a preset reasonable threshold, the size of the initial sliding window is enlarged until the reasonable degree is greater than the preset reasonable threshold.

[0075] Furthermore, the box plot analysis module A20 is also used for: Determine the upper and lower quartiles and median in the box plot corresponding to the sliding window; Using the importance, determine the upper and lower limits of the abnormality in the box plot corresponding to the sliding window; Use the upper and lower quartiles, median, and upper and lower abnormal limits to determine the box plot of the sliding window; Summarize the box plots of the sliding window to obtain the box plots of each production data; Furthermore, the box plot analysis module A20 is also used for: Using the importance, calculating the upper and lower limit adjustment coefficients of the box plot corresponding to the sliding window; The inner limit range of the box plot is calculated by using the upper and lower quartiles in the box plot corresponding to the sliding window; The upper quartile, the inner limit range and the upper and lower limit adjustment coefficients are used to calculate the upper limit of the abnormality in the box plot; The lower quartile, the inner limit range and the upper and lower limit adjustment coefficients are used to calculate the abnormal lower limit in the box plot.

[0076] Furthermore, the box plot analysis module A20 is also used for: Obtain the historical average duration of each denitration treatment of the SCR flue gas denitrification system and the frequency of collecting various production data; The initial size of the sliding window is calculated using the historical average duration and the acquisition frequency.

[0077] Furthermore, the data screening module A40 is also used for: Determine the number of abnormal times that any target data point in the intersection of the abnormal data points is identified as abnormal data and the number of sliding windows that include the target data point; The abnormal performance degree of the target data point is calculated by using the number of abnormalities, the number of windows and the total number of sliding windows corresponding to each production data; If the abnormal performance degree is greater than a preset abnormal threshold, the target data point is marked as an abnormal point; All abnormal data marked as abnormal points in various production data are cleaned.

[0078] Furthermore, the data screening module A40 is also used for: Determining the rationality of the sliding window corresponding to the target data point; The abnormal performance degree is corrected using the reasonable degree to obtain a corrected abnormal performance degree.

[0079] The specific implementation of the data cleaning device of the data-driven optimization system of the present invention is basically the same as the various embodiments of the data cleaning method of the data-driven optimization system described above, and will not be repeated here.

[0080] In addition, the present invention also provides a computer-readable storage medium. The computer-readable storage medium of the present invention stores a data cleaning program of a data-driven optimization system, wherein when the data cleaning program of the data-driven optimization system is executed by a processor, the steps of the data cleaning method of the data-driven optimization system as described above are implemented.

[0081] Among them, the method implemented when the data cleaning program of the data-driven optimization system is executed can refer to the various embodiments of the data cleaning method of the data-driven optimization system of the present invention, and will not be repeated here.

[0082] It should be noted that the sequence of the above embodiments of the present invention is only for description and does not represent the advantages and disadvantages of the embodiments. The processes depicted in the accompanying drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0083] The various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referenced to each other, and each embodiment focuses on the differences from other embodiments.

[0084] It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as methods, devices, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0085] The above description is only a preferred embodiment of the present invention, and does not limit the protection scope of the present invention. All equivalent structural changes made by using the contents of the present invention specification and drawings under the inventive concept of the present invention, or directly / indirectly applied in other related technical fields are included in the protection scope of the present invention.

Claims

1. A data cleaning method for a data-driven optimization system, characterized in that: The method comprises: Obtain various types of production data in the SCR flue gas denitrification system; Determine the importance of various production data to the outlet nitrogen oxide concentration during the SCR flue gas denitrification process; Establishing a sliding window for each production data, and determining a box plot of each production data using the importance; The box plot is used to determine a first set of potential abnormal data points for each production data, and The rule determines a second set of potentially abnormal data points for each production data; Determine an intersection of abnormal data points between the first set and the second set, and use the intersection of abnormal data points to perform data cleaning on each production data; The window represents a data set, and the sliding window represents a variable data set of movable time series.

2. The data cleaning method of the data-driven optimization system according to claim 1, characterized in that: Various production data include the concentration of nitrogen oxides in the flue gas at the reactor outlet, and the concentration of nitrogen oxides in the flue gas at the reactor outlet is used as the control data; The steps to determine the importance of various production data to the outlet nitrogen oxide concentration in the SCR flue gas denitrification process include: Determine other production data in the SCR flue gas denitrification system except for the concentration of nitrogen oxides in the flue gas at the reactor outlet, create a target matching window on the other production data, and create a search window on the control data; Determine each matched window captured by sliding in the search window in a sequential manner, and determine a target matched window with the highest matching degree with the target matching window among each matched window; Determine a first correlation between the target matching window and the target matched window; Determining a second correlation between the reactor outlet flue gas nitrogen oxide concentration and the outlet nitrogen oxide concentration; The importance of each production data to the outlet nitrogen oxide concentration is determined by using the first correlation and the second correlation.

3. The data cleaning method of the data-driven optimization system according to claim 2, characterized in that: The step of determining a first correlation between the target matching window and the target matched window comprises: Determine the matching degree and time interval between the target matching window and the target matched window; A first correlation between the target matching window and the target matched window is determined by using the matching degree and the time interval.

4. The data cleaning method of the data-driven optimization system according to claim 1, characterized in that: The steps of establishing a sliding window for each production data include: Determine the initial size of the sliding window of each production data to obtain an initial sliding window; Determining the skewness of the distribution of data points in the initial sliding window; According to the skewness, the size of the initial sliding window is adjusted to make it meet the preset normal distribution conditions, and a sliding window of each production data is obtained.

5. The data cleaning method of the data driven optimization system according to claim 4, characterized in that: The step of determining the initial size of the sliding window of each production data to obtain the initial sliding window includes: Obtain the historical average duration of each denitration treatment of the SCR flue gas denitrification system and the frequency of collecting various production data; The initial size of the sliding window is calculated using the historical average duration and the acquisition frequency.

6. The data cleaning method of the data driven optimization system according to claim 4, characterized in that: According to the skewness, the size of the initial sliding window is adjusted to make it meet the preset normal distribution condition, and the step of obtaining the sliding window of each production data includes: Using the skewness, calculating the rationality of the time length of the initial sliding window; If the rationality is greater than a preset rationality threshold, it is determined that the initial sliding window meets the preset normal distribution condition; If the reasonable degree is less than or equal to a preset reasonable threshold, the size of the initial sliding window is enlarged until the reasonable degree is greater than the preset reasonable threshold.

7. The data cleaning method of the data driven optimization system according to claim 4, characterized in that: The step of establishing a sliding window for each production data and determining a box plot of each production data using the importance comprises: Determine the upper and lower quartiles and median in the box plot corresponding to the sliding window; Using the importance, determine the upper and lower limits of the abnormality in the box plot corresponding to the sliding window; Use the upper and lower quartiles, median, and upper and lower abnormal limits to determine the box plot of the sliding window; Summarize the box plots of the sliding window to obtain the box plots of each production data; The higher the importance, the closer the upper and lower limits of the abnormality are, and vice versa.

8. The data cleaning method of the data driven optimization system according to claim 7, characterized in that: The step of determining the upper and lower limits of the abnormality in the box plot corresponding to the sliding window by using the importance includes: Using the importance, calculating the upper and lower limit adjustment coefficients of the box plot corresponding to the sliding window; The inner limit range of the box plot is calculated by using the upper and lower quartiles in the box plot corresponding to the sliding window; The upper quartile, the inner limit range and the upper and lower limit adjustment coefficients are used to calculate the upper limit of the abnormality in the box plot; The lower quartile, the inner limit range and the upper and lower limit adjustment coefficients are used to calculate the abnormal lower limit in the box plot.

9. The data cleaning method of the data driven optimization system according to claim 1, characterized in that: The step of using the intersection of the abnormal data points to clean the various production data includes: Determine the number of abnormal times that any target data point in the intersection of the abnormal data points is identified as abnormal data and the number of sliding windows that include the target data point; The abnormal performance degree of the target data point is calculated by using the number of abnormalities, the number of windows and the total number of sliding windows corresponding to each production data; If the abnormal performance degree is greater than a preset abnormal threshold, the target data point is marked as an abnormal point; All abnormal data marked as abnormal points in various production data are cleaned.

10. The data cleaning method of the data driven optimization system according to claim 9, characterized in that: After the step of calculating the abnormal performance degree of the target data point, the method further includes: Determining the rationality of the sliding window corresponding to the target data point; The abnormal performance degree is corrected using the reasonable degree to obtain a corrected abnormal performance degree.

Images