Intelligent power plant data processing method based on cloud platform

Abstract

The invention discloses an intelligent power plant data processing method based on a cloud platform, and relates to the technical field of intelligent power plants, and the method comprises a DCS data processing method, and the method comprises the steps: 1, constructing an optimal time sequence model in a cloud platform server; 2, processing the acquired real-time DCS data by the optimal time sequence model, and outputting normalized DCS data; and 3, the standardized DCS data is displayed and monitored by the digital large screen. According to the method, standard DCS data are obtained by carrying out preprocessing, format unification processing, useless data removal processing and outlier elimination processing on acquired DCS data, an optimal time sequence model is obtained through the standard DCS data, and real-time DCS data are normalized by utilizing the optimal time sequence model, so that the real-time DCS data can be obtained. And finally, the DCS data can be uniformly, normatively and accurately accessed to the cloud platform to be displayed and monitored. And the data processing accuracy and the real-time monitoring efficiency are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of smart power plants, and in particular to a data processing method for smart power plants based on a cloud platform. Background Art

[0002] Smart power plants are based on the digitalization and informatization of power companies. They make full use of cloud service platforms, big data centers, the Internet of Things, 3D virtual interaction, AI, robots and other technologies to fully integrate the entire process of traditional power production and management with advanced artificial intelligence technologies, thereby achieving centralized overall control of various sub-platforms of power companies, in-depth information discovery and analysis, autonomous optimization of production processes, scientific planning of maintenance plans, smart decision-making and early warning, etc.

[0003] Smart power plant data usually includes DCS data from the DCS control system and monitoring data from video monitoring equipment. For the overall monitoring of smart power plant data, the prior art has proposed a cloud-based smart power plant data monitoring technology. For example, the patent document with announcement number CN114826747A discloses an in-service smart power plant management and control architecture and its construction method, which specifically discloses that all data of all power plant systems are connected to the plant-side cloud platform, and the cloud platform is bidirectionally connected to the power plant DCS control system, and all data is aggregated through the power plant core switch. This technology allows users to enjoy active personnel safety supervision, equipment risk inspection and early warning, and closed-loop intelligent operation functions, but analysis has found that it still has the following technical problems: 1. This technology cannot standardize data, resulting in poor uniformity and accuracy of data, which is not conducive to supervision.

[0004] 2. This technology can only process DCS data, but cannot achieve unified processing of monitoring data, and its applicability is poor.

[0005] Therefore, it is necessary to provide a new technology to solve the above technical problems. Summary of the invention

[0006] The purpose of the present invention is to overcome the above-mentioned problems existing in the prior art, and to provide a smart power plant data processing method based on a cloud platform. The method obtains standard DCS data by preprocessing the acquired DCS data, unifying the format, removing useless data, and eliminating outliers, obtains an optimal time series model through the standard DCS data, and standardizes the real-time DCS data by using the optimal time series model, so that the DCS data can be finally connected to the cloud platform in a unified, standardized and accurate manner for display and monitoring; in addition, the method can also process the monitoring data at the same time, and improves the accuracy of data processing and the efficiency of real-time monitoring by comprehensively processing the DCS data and the monitoring data, and effectively solves the technical problems that the prior art cannot process data in a standardized manner and cannot realize the unified processing of monitoring data.

[0007] To achieve the above purpose, the technical solution adopted by the present invention is as follows: A cloud platform-based smart power plant data processing method includes a DCS data processing method and a smart power plant data processing method, wherein: The DCS data processing method comprises the following steps: Step 1: Build the optimal time series model in the cloud platform server S1. Obtain DCS data of the DCS system through the cloud platform server, pre-process the DCS data, screen and mark missing data and obviously unreasonable data, and fill in the missing data and interpolate the obviously unreasonable data; S2, after preprocessing, the format of DCS data is unified, useless data columns are removed, and a new DataFrame architecture is constructed; S3. In the DataFrame architecture, use the clustering algorithm to set the number of clusters and thresholds, and classify the DCS data according to the set number of clusters and thresholds, find outliers and remove them, and obtain standard DCS data; S4. Select a time series model, set the time interval for periodic prediction, use the time series model to train and predict standard DCS data, and find the time series model corresponding to the best change period of DCS data by comparing the prediction results at different time intervals; S5, make residual judgment on the time series model obtained in S4, eliminate the interference of trend and periodicity on the data, until the optimal time series model of DCS data conforming to the normal fluctuation range is obtained; Step 2: First, the DCS data is obtained from the DCS communication station server of the DCS system through the forward isolation device, and then the DCS data is sent to the cloud platform server through the data gateway and the switch in sequence. In the cloud platform server, the DCS data is processed by the optimal time series model and the standardized DCS data is output; Step 3: The standardized DCS data is displayed and monitored by a large digital screen.

[0008] The processing process of the smart power plant data processing method is: first use the video surveillance equipment to collect the monitoring video of the smart power plant, then use the cloud platform video aggregation switch connected to the switch to aggregate the monitoring video collected by the video surveillance equipment, and then use the cloud platform video algorithm server to convert the aggregated monitoring video into a format, and obtain standard video data after the format conversion processing, and finally display and monitor the standard video data on the digital large screen.

[0009] In S1, the method for screening missing data and obviously unreasonable data is: First, we use automated data cleaning technology to preliminarily screen DCS data using anomaly detection algorithms based on statistical analysis, and mark out possible missing and obviously unreasonable data points; Then, the isolation forest algorithm and support vector data description method in machine learning technology are applied to verify and confirm possible missing and obviously unreasonable data points, and finally the accurate missing data and obviously unreasonable data are screened out.

[0010] In S1, there are three methods to fill in missing data, as follows: The first method: Based on the DCS data context and historical trends, the moving average method or linear interpolation is used for filling; The second method is to first use Bayesian estimation and multiple interpolation techniques to generate multiple possible filling values, and then use the Monte Carlo simulation method to select the optimal filling value for filling; The third method: Combine the logical relationship between DCS data and use the prediction filling method based on regression analysis to fill in the gaps.

[0011] In S1, the method for filling in obviously unreasonable data is: (1) Re-verify the obviously unreasonable data first, and eliminate and restore the obviously unreasonable data caused by sensor failure or instantaneous noise interference by comparing the data trends at adjacent time points; (2) For obviously unreasonable data that cannot be restored, a rule-based automated discarding process is used to handle it. This process automatically determines and discards obviously unreasonable data based on preset thresholds, upper and lower limit rules, and a preset database; (3) Use data interpolation or stretching technology to interpolate the discarded obviously unreasonable data.

[0012] In S2, the method for unifying the format of DCS data is as follows: the cloud platform server automatically identifies and parses the various fields and data types of DCS data, and automatically classifies and standardizes data formats from different sources by using regular expression-based template matching and pattern recognition algorithms; and then uses data mapping tables and data dictionaries to accurately map various fields and data types to unify the data formats among DCS systems.

[0013] In S2, the method to remove useless data columns is: (1) Using the principal component analysis (PCA) algorithm and the recursive feature elimination (RFE) algorithm, the data columns that are irrelevant to modeling or have low contribution are automatically identified; (2) Use a method based on information gain and Gini coefficient to evaluate the impact of data columns that are irrelevant to modeling or have low contribution. If a data column has no impact on the target variable, the corresponding data column is determined to be a useless data column and removed.

[0014] In S4, the selected time series models include ARIMA model and SARIMA model, and the time intervals for periodic forecasting are set to 1, 3, 6, and 12 months.

[0015] In step 2, the DCS data is first transmitted from the forward isolation device to the data gateway via the TCP protocol, and then converted by the data gateway into the MQTT protocol and transmitted to the cloud platform server.

[0016] In step 2, the switch includes a cloud platform switch and a DCS switch. The DCS switch is connected between the cloud platform switch and the data gateway. The cloud platform video aggregation switch is connected to the cloud platform switch. The cloud platform switch and the DCS switch are configured with the same network segment IP address to realize the acquisition of DCS data between the cloud platform server and the DCS communication station server.

[0017] The video surveillance equipment includes a network video recorder (NVR) and a camera. The camera is used to collect surveillance videos of the smart power plant. The network video recorder (NVR) is used to store the surveillance videos collected by the camera and transmit them to the cloud platform video aggregation switch.

[0018] By adopting the above technical solution, the beneficial technical effects of the present invention are: 1. In the data processing method of the smart power plant of the present invention, for the DCS data of the DCS system, firstly, the acquired DCS data is preprocessed, formatted, removed, and outliers are removed to obtain standard DCS data, and then the standard DCS data is trained and predicted using a time series model to obtain an optimal time series model, and then the real-time DCS data is standardized using the optimal time series model, so that the DCS data can be finally connected to the cloud platform in a unified, standardized and accurate manner for display and monitoring, thereby effectively improving the standardization, uniformity and accuracy of the DCS data.

[0019] 2. The present invention can also process monitoring data simultaneously, and improves the accuracy of data processing and the efficiency of real-time monitoring by comprehensively processing DCS data and monitoring data.

[0020] 3. The present invention can display DCS data and video data information in real time, so that operators can obtain a comprehensive on-site view and enhance the control ability and safety management level of the production process.

[0021] 4. The present invention supports online monitoring, remote diagnosis and predictive maintenance, improves customer satisfaction, and is conducive to building new digital capabilities for product services. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 This is a principle block diagram of the present invention. DETAILED DESCRIPTION

[0023] like Figure 1 As shown, the present invention provides a smart power plant data processing method based on a cloud platform, including a DCS data processing method and a smart power plant data processing method, wherein: The DCS data processing method comprises the following steps: Step 1: Build the optimal time series model in the cloud platform server S1. Obtain DCS data of the DCS system through the cloud platform server and preprocess the acquired DCS data. The preprocessing refers to screening and marking missing data and obviously unreasonable data in the DCS data, and filling in the missing data and interpolating the obviously unreasonable data.

[0024] In this step, the screening method for missing data and obviously unreasonable data is as follows: First, through automated data cleaning technology, the DCS data is preliminarily screened using anomaly detection algorithms based on statistical analysis (such as the Z-score method based on mean and standard deviation, or the IQR method based on quantiles), and possible missing and obviously unreasonable data points are marked.

[0025] Secondly, the Isolation Forest algorithm and One-Class SVM method in machine learning technology are applied to further verify and confirm possible missing and obviously unreasonable data points, and finally screen out accurate missing data and obviously unreasonable data to ensure the accuracy and robustness of abnormal data detection.

[0026] In this step, there are three methods for filling missing data. Different methods can be used to fill missing data according to different situations. The details are as follows: The first method is to fill in common missing data using moving average or linear interpolation based on DCS data context and historical trends.

[0027] The second method: For complex missing data, first use Bayesian Estimation and Multiple Imputation techniques to generate multiple possible filling values, and then use the Monte Carlo Simulation method to select the optimal filling value for filling.

[0028] The third method: For key missing data, combined with the logical relationship between DCS data, a prediction filling method based on regression analysis (such as decision tree regression or random forest regression) is used for refined filling.

[0029] In this step, the method for filling in obviously unreasonable data is: (1) First, re-verify the obviously unreasonable data. By comparing the data trends at adjacent time points, eliminate and restore the obviously unreasonable data caused by sensor failure or instantaneous noise interference.

[0030] (2) For obviously unreasonable data that cannot be restored, a rule-based automated discarding process is used to handle it. This process automatically determines and discards obviously unreasonable data based on preset thresholds, upper and lower limit rules, and a preset database.

[0031] (3) Use data interpolation or stretching technology to interpolate the discarded obviously unreasonable data.

[0032] By filling in obviously unreasonable data, analysis deviation caused by data loss can be avoided, ensuring the continuity and integrity of DCS data.

[0033] S2. After preprocessing, the format of DCS data is unified, useless data columns are removed, and a new DataFrame architecture is constructed.

[0034] In this step, the method of unifying the format of DCS data is: the cloud platform server automatically identifies and parses the various fields and data types of DCS data, and automatically classifies and standardizes the data formats of different sources by using regular expression-based template matching and pattern recognition algorithms; then uses the data mapping table (Data Mapping Table) and data dictionary (Data Dictionary) to accurately map each field and data type to unify the data format of each DCS system. The mapping process can also use a specific rule engine to automatically process complex mapping relationships to ensure data consistency.

[0035] In this step, the method to remove useless data columns is: (1) Using the principal component analysis (PCA) algorithm and the recursive feature elimination (RFE) algorithm, data columns that are irrelevant to modeling or have low contribution are automatically identified.

[0036] (2) Use a method based on information gain and the Gini Index to evaluate the impact of data columns that are irrelevant to modeling or have low contribution. If a data column has no impact on the target variable, the corresponding data column is determined to be a useless data column and removed.

[0037] In this step, a new DataFrame architecture can be generated through an automated script. This architecture is based on a standardized data structure and adopts an efficient memory allocation method based on vectorized calculations, which has the advantage of optimizing data storage and access efficiency. In addition, the new DataFrame architecture can automatically adjust the index and partition strategy of the DataFrame to ensure the optimal performance of data in large-scale processing and distributed computing environments. In addition, the new DataFrame architecture is automatically deployed to the data processing module of the cloud platform server, which can be seamlessly connected with subsequent data analysis, modeling, and visualization processes.

[0038] S3. In the DataFrame architecture, use the clustering algorithm to set the number of clusters and thresholds, and classify the DCS data according to the set number of clusters and thresholds, find outliers and remove them, and obtain standard DCS data.

[0039] S4. Select a time series model, set the time interval for periodic prediction of the time series model, use the time series model to train and predict standard DCS data, and find out the time series model corresponding to the best change period of DCS data by comparing the prediction results at different time intervals.

[0040] In this step, the selected time series models include but are not limited to the ARIMA model and the SARIMA model, and the time intervals for periodic prediction are set to 1, 3, 6, and 12 months. The prediction results of each time series model at different time intervals are compared to evaluate its prediction accuracy and model stability. By comparing the prediction performance of different models, the time series model that best reflects the optimal change period of DCS data is found.

[0041] S5. Perform residual judgment on the time series model obtained in S4 to eliminate the interference of trend and periodicity on the data, until the optimal time series model is obtained in which the DCS data conforms to the normal fluctuation range.

[0042] Step 2: First, obtain real-time DCS data from the DCS communication station server of the DCS system through the forward isolation device, and then send the DCS data to the cloud platform server through the data gateway and switch in sequence. In the cloud platform server, the DCS data is processed by the optimal time series model and the standardized DCS data is output.

[0043] like Figure 1 As shown, the DCS data is first transmitted from the forward isolation device to the data gateway through the TCP protocol, and then converted by the data gateway into the MQTT protocol and transmitted to the cloud platform server.

[0044] like Figure 1 As shown, the switch includes a cloud platform switch and a DCS switch. The DCS switch is connected between the cloud platform switch and the data gateway. The cloud platform video aggregation switch is connected to the cloud platform switch. The cloud platform switch and the DCS switch realize the acquisition of DCS data between the cloud platform server and the DCS communication station server by configuring the same network segment IP address.

[0045] Step 3: The standardized DCS data is displayed and monitored by a large digital screen.

[0046] The processing process of the monitoring data processing method is as follows: Figure 1 As shown, the video surveillance equipment is first used to collect the surveillance video of the smart power plant, and then the cloud platform video aggregation switch connected to the switch is used to aggregate the surveillance video collected by the video surveillance equipment. The cloud platform video algorithm server is then used to convert the aggregated surveillance video into a format, and standard video data is obtained after the format conversion. Finally, the digital large screen displays and monitors the standard video data.

[0047] Preferably, the video surveillance equipment includes a network video recorder NVR and a camera. The camera is used to collect surveillance videos of the smart power plant. The network video recorder NVR is used to store the surveillance videos collected by the camera and transmit them to a cloud platform video aggregation switch.

[0048] In summary, the present invention can not only perform standardized processing on DCS data, but also perform standardized processing on monitoring data at the same time. By comprehensively processing DCS data and monitoring data, the accuracy, standardization and reliability of data processing in smart power plants are effectively improved.

[0049] The above description is only a specific implementation mode of the present invention. Any feature disclosed in this specification, unless otherwise stated, may be replaced by other alternative features that are equivalent or have similar purposes; all disclosed features, or all steps in methods or processes, except for mutually exclusive features and / or steps, may be combined in any way.

Claims

1. A cloud platform-based smart power plant data processing method, characterized in that: Including DCS data processing method and smart power plant data processing method, among which, The DCS data processing method comprises the following steps: Step 1: Build the optimal time series model in the cloud platform server S1. Obtain DCS data of the DCS system through the cloud platform server, pre-process the DCS data, screen and mark missing data and obviously unreasonable data, and fill in the missing data and interpolate the obviously unreasonable data; S2, after preprocessing, the format of DCS data is unified, useless data columns are removed, and a new DataFrame architecture is constructed; S3. In the DataFrame architecture, use the clustering algorithm to set the number of clusters and thresholds, and classify the DCS data according to the set number of clusters and thresholds, find outliers and remove them, and obtain standard DCS data; S4. Select a time series model, set the time interval for periodic prediction, use the time series model to train and predict standard DCS data, and find the time series model corresponding to the best change period of DCS data by comparing the prediction results at different time intervals; S5, make residual judgment on the time series model obtained in S4, eliminate the interference of trend and periodicity on the data, until the optimal time series model of DCS data conforming to the normal fluctuation range is obtained; Step 2: First, the DCS data is obtained from the DCS communication station server of the DCS system through the forward isolation device, and then the DCS data is sent to the cloud platform server through the data gateway and the switch in sequence. In the cloud platform server, the DCS data is processed by the optimal time series model and the standardized DCS data is output; Step 3: The standardized DCS data is displayed and monitored by a digital large screen; The processing process of the monitoring data processing method is: first use the video monitoring equipment to collect the monitoring video of the smart power plant, then use the cloud platform video aggregation switch connected to the switch to aggregate the monitoring video collected by the video monitoring equipment, and then use the cloud platform video algorithm server to convert the aggregated monitoring video into a format, and obtain standard video data after the format conversion, and finally display and monitor the standard video data on the digital large screen.

2. The cloud platform-based smart power plant data processing method according to claim 1, characterized in that: In S1, the method for screening missing data and obviously unreasonable data is: First, we use automated data cleaning technology to preliminarily screen DCS data using anomaly detection algorithms based on statistical analysis, and mark out possible missing and obviously unreasonable data points; Then, the isolation forest algorithm and support vector data description method in machine learning technology are applied to verify and confirm possible missing and obviously unreasonable data points, and finally the accurate missing data and obviously unreasonable data are screened out.

3. The cloud platform-based smart power plant data processing method according to claim 1 is characterized by: In S1, there are three methods to fill in missing data, as follows: The first method: Based on the DCS data context and historical trends, the moving average method or linear interpolation is used for filling; The second method is to first use Bayesian estimation and multiple interpolation techniques to generate multiple possible filling values, and then use the Monte Carlo simulation method to select the optimal filling value for filling; The third method: Combine the logical relationship between DCS data and use the prediction filling method based on regression analysis to fill in the gaps.

4. The method for processing data of a smart power plant based on a cloud platform according to claim 1, characterized in that: In S1, the method for filling in obviously unreasonable data is: (1) Re-verify the obviously unreasonable data first, and eliminate and restore the obviously unreasonable data caused by sensor failure or instantaneous noise interference by comparing the data trends at adjacent time points; (2) For obviously unreasonable data that cannot be restored, a rule-based automated discarding process is used to handle it. This process automatically determines and discards obviously unreasonable data based on preset thresholds, upper and lower limit rules, and a preset database; (3) Use data interpolation or stretching technology to interpolate the discarded obviously unreasonable data.

5. The cloud platform-based smart power plant data processing method according to claim 1 is characterized by: In S2, the method for unifying the format of DCS data is as follows: the cloud platform server automatically identifies and parses the various fields and data types of DCS data, and automatically classifies and standardizes data formats from different sources by using regular expression-based template matching and pattern recognition algorithms; and then uses data mapping tables and data dictionaries to accurately map various fields and data types to unify the data formats of various DCS systems.

6. The cloud platform-based smart power plant data processing method according to claim 1 is characterized by: In S2, the method to remove useless data columns is: (1) Using the principal component analysis (PCA) algorithm and the recursive feature elimination (RFE) algorithm, the data columns that are irrelevant to modeling or have low contribution are automatically identified; (2) Use a method based on information gain and Gini coefficient to evaluate the impact of data columns that are irrelevant to modeling or have low contribution. If a data column has no impact on the target variable, the corresponding data column is determined to be a useless data column and removed.

7. The cloud platform-based smart power plant data processing method according to claim 1 is characterized by: In S4, the selected time series models include ARIMA model and SARIMA model, and the time intervals for periodic forecasting are set to 1, 3, 6, and 12 months.

8. The cloud platform-based smart power plant data processing method according to claim 1 is characterized by: In step 2, the DCS data is first transmitted from the forward isolation device to the data gateway via the TCP protocol, and then converted by the data gateway into the MQTT protocol and transmitted to the cloud platform server.

9. The cloud platform-based smart power plant data processing method according to claim 1, characterized in that: In step 2, the switch includes a cloud platform switch and a DCS switch. The DCS switch is connected between the cloud platform switch and the data gateway. The cloud platform video aggregation switch is connected to the cloud platform switch. The cloud platform switch and the DCS switch are configured with the same network segment IP address to realize the acquisition of DCS data between the cloud platform server and the DCS communication station server.

10. The cloud platform-based smart power plant data processing method according to claim 1, characterized in that: The video surveillance equipment includes a network video recorder (NVR) and a camera. The camera is used to collect surveillance videos of the smart power plant. The network video recorder (NVR) is used to store the surveillance videos collected by the camera and transmit them to the cloud platform video aggregation switch.

Images