Power grid data analysis and prediction method and system

Abstract

The invention belongs to the technical field of data analysis, and relates to a power grid data analysis and prediction method and system, and the method comprises the steps: obtaining original data, carrying out the analysis and processing of the original data, carrying out the data feature selection, constructing the data meeting the data features into a data set, and storing the data set; business scene building is carried out, and a data set is selected according to manual building or intelligent building; and constructing an interaction component, wherein the interaction component calls the selected data set to output interaction information. The method has the beneficial effects that a set of software with high flexibility and high operability is provided, the data utilization capability is improved, the operation efficiency and reliability of a power grid are optimized, and the response capability to emergencies is also improved, so that a power grid enterprise is supported to efficiently extract valuable information from mass data for scene planning, and the power grid enterprise is enabled to have a good development prospect. Strategic decision and daily operation management are assisted.

Description

Technical Field

[0001] The present invention belongs to the technical field of data analysis, and relates to a power grid data analysis and prediction method and system. Background Art

[0002] At present, the power grid business intelligent analysis and scenario planning software (hereinafter referred to as intelligent analysis and planning software) mainly used in power companies is a system that can collect, process and analyze existing data, and quickly and accurately provide visualization scenarios to support management decisions and business operations after effective integration. It involves data connection visualization and intelligent data analysis business scenario construction applications, aiming to provide a set of highly flexible and highly operational software to support power grid companies to efficiently extract valuable information from massive data for scenario planning, assist strategic decision-making and daily operation management. There are many tools for power grid business analysis and scenario construction on the market, but the power grid has its own corporate uniqueness, including large data volume and high complexity, real-time monitoring and reporting, multi-dimensional data analysis, and wide application of visualization scenarios. Existing power grid analysis and construction systems mostly rely on traditional processing methods. First, they are based on a single display of numerous historical data on site, with manual assistance, and do not perform data fusion processing and intelligent analysis. Second, they rely on various manufacturers to build visualization scenarios, which can only be used after a large amount of data analysis and development. Therefore, user participation is low and the construction process is slow. Although these methods can guarantee the daily operation of the power grid to a certain extent, they are not highly integrated with industry scenarios. When faced with complex and emergency response needs in a big data environment, they often seem to be unable to cope with the needs, and have limitations such as slow analysis speed, low accuracy, and weak ability to respond to emergencies.

[0003] The original technical solutions for power grid business analysis and scenario building tools mainly rely on traditional data analysis methods, static models, and relatively basic information technology means. Compared with the current advanced technical solutions, they have the following problems: (1) Limited data processing and analysis: Currently, data processing and analysis are mostly based on statistics and experience, and the complexity and nonlinear relationships of power grid operation are not adequately handled. As a result, massive amounts of power grid operation data cannot be quickly and effectively processed and analyzed, resulting in insufficiently precise monitoring of the power grid status and limited predictive capabilities.

[0004] (2) Serious data silos: There is a lack of effective data integration and sharing mechanisms between systems, resulting in scattered and inconsistent data. This makes it difficult to form a global perspective to analyze the status of the power grid, affecting the quality of decision-making.

[0005] (3) Lack of dynamic push and optimization capabilities: The original simulation tools are usually based on static models, which make it difficult to reflect the dynamic changes and complex interactions of the power grid in real time. The models do not have dynamic tuning and dynamic push, and their guiding role in power grid planning, operation adjustment and emergency response is limited.

[0006] (4) Weak visualization and interaction capabilities: The data display and human-computer interaction design are relatively simple and fixed. There is no intelligent analysis push and visualization construction, which makes it difficult to quickly adapt to changes in demand and upgrade and expansion. This limits the innovation and development of power grid management and services, and reduces work efficiency and decision-making effectiveness. Summary of the invention

[0007] The purpose of the present invention is to provide a system that can collect, store, and analyze existing data, and after effective integration, quickly and accurately provide visualization scenarios to support management decisions and business operations. The application is constructed by taking efficient data analysis, accurate data push, unified data integration, and visualization of scenario construction as the main criteria, solving the problems existing in the prior art.

[0008] The technical solution adopted by the present invention is: A power grid data analysis and prediction method, comprising: Obtain the original data, analyze and process it, select data features, and construct and store the data that meets the data features into a data set; Build business scenarios and select data sets based on manual or intelligent building; An interactive component is constructed, wherein the interactive component retrieves the selected data set and outputs interactive information.

[0009] As a preference, Conduct preliminary analysis of the structure and patterns of sample data sets, including data overview, statistical description, and data visualization; Preprocess the data after pre-analysis, including data deduplication, missing value processing, data conversion, and data reduction; Retrieve highly relevant features from the preprocessed data; As a preference, The original data includes basic records of power grid equipment, online monitoring data, live detection data, and offline test data.

[0010] As a preference, The following steps are involved: Data Overview: Check the data type of each feature in the data set, including the type and naming conventions of each field in the basic ledger of power grid equipment, check whether there are missing values ​​or duplicate values, and count the number and proportion of missing values ​​and duplicate values.

[0011] Statistical description: View the data distribution in the data set, including the maximum value, minimum value, mean value, standard deviation, and variance in the online monitoring data, and quickly obtain the statistical category distribution; as well as the frequency of the basic ledger category of the power grid equipment.

[0012] Data visualization: Use visual charts to analyze data anomalies and category frequencies. This includes analyzing the anomalies of power grid test data using histograms, box plots, and scatter plots, and analyzing the category frequencies of data using bar charts.

[0013] As a preference, The following steps are involved: Data deduplication: remove duplicate data records from a data set.

[0014] Missing value processing: interpolation or deletion of missing data.

[0015] Data conversion: standardize the data.

[0016] Data reduction: The covariance matrix is ​​calculated based on the standardized data through principal component analysis (PCA), and the covariance matrix is ​​eigendecomposed to obtain the eigenvalues ​​and corresponding eigenvectors. The most important eigenvectors are selected as new principal components according to the size of the eigenvalues.

[0017] As a preference, In missing value processing, When the proportion of missing values ​​in the data set is small, or the data points where the missing values ​​are located are not important for the overall analysis, delete the records containing missing values; When missing values ​​cannot be simply deleted, fill them with a global constant, or fill them with the mean, median, or mode of a specific attribute, or fill them with the attribute value of the most similar sample.

[0018] As a preference, Build a data set that provides the data foundation for the component. When you choose to build manually, the data set is generated through the database and API service. The generation of the data set includes: Directly connect to the database: configure database information to connect to the database after data processing, directly query and debug the database table online, store the query results as a data set, and provide data for component rendering.

[0019] API service: configures the interface address and request method, performs queries and online debugging, stores query results as data sets, and provides data for component rendering.

[0020] As a preference, When choosing smart building, include: The user asks a query; the big model performs semantic analysis on the user's question, generates a query SQL statement and a data set, and generates recommended chart components based on the user's question; if the recommended data set and components meet the user's question requirements, the data set will be stored and pushed; otherwise, new data sets and recommended components will be re-optimized and generated until the user's question requirements are met.

[0021] As a preference, The interactive components include: Basic components: including text, media, images, videos and other types of components, which are used for scene display of various static resources or dynamic address rendering.

[0022] Table form components: include buttons, radio check boxes, input boxes, and table type components, which are applied to various business operations and list detail display scenarios.

[0023] Chart components: including bar charts, pie charts, line charts, scatter plots, and radar charts, which are used for displaying various data aggregation and statistical scenarios.

[0024] Resource components: including background, border, icon, and decorative components, which are used for decoration of various scenes.

[0025] Map components: including echarts map, offline map, and Siji map components, which are used for scene display of map rendering and map data interaction.

[0026] A power grid data analysis and prediction system, comprising: The first module is configured to obtain raw data, analyze and process the raw data, select data features, construct data that meet the data features into a data set, and store the data; The second module is configured to build a business scenario and select a data set based on manual or intelligent building; The third module is configured to construct an interactive component, and the interactive component retrieves the selected data set and outputs interactive information.

[0027] Comparison between the present invention and the prior art (1) Improve the analysis efficiency of power grid business, reduce manual intervention and lower analysis costs through data exploration and processing.

[0028] (2) Break down data silos by integrating various platforms in a unified manner, improve data integration and sharing capabilities, and integrate once and reuse multiple times.

[0029] (3) Enhance the flexibility and adaptability of the software through visual construction to quickly respond to new scenarios and new challenges.

[0030] The beneficial effects of the present invention are: Providing a set of highly flexible and highly operational software not only improves data utilization capabilities, optimizes the operational efficiency and reliability of the power grid, but also improves the ability to respond to emergencies, so as to support power grid companies to efficiently extract valuable information from massive data for scenario planning, assist strategic decision-making and daily operations management. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 It is a data processing diagram provided by an embodiment of the present invention.

[0032] Figure 2 It is a construction flow chart provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0033] The present invention is described in detail below with reference to the accompanying drawings and specific embodiments.

[0034] Part I: Description of the complete steps of the method of the present invention This section will elaborate on the specific implementation plan of a power grid business intelligent analysis and scenario construction tool. The tool aims to improve the power grid's data analysis response capability and scenario construction capability by integrating big data analysis, artificial intelligence algorithms, and advanced visualization construction technology.

[0035] Data Mining and Analysis ①Data collection: Collect relevant data, with structured data as the main data.

[0036] Structured data: including basic grid data, online monitoring data, live detection, offline testing, etc. Finally, the structured data is stored in the database.

[0037] Basic data: mainly includes equipment ledger, power station ledger, manufacturers, etc.

[0038] Online monitoring: mainly includes online oil chromatography, SF6 gas, trace water, core grounding current, etc.

[0039] Live detection: mainly includes core grounding current, clamp grounding current, partial discharge, etc.

[0040] Offline test: mainly includes offline oil chromatography, winding DC resistance, dielectric loss, etc.

[0041] Inspection data: mainly includes maintenance records, fault records, hidden danger records, defect records, etc.

[0042] ②Data exploration: The structure and regularity of the sample data set are analyzed by testing the data quality of the data set, drawing charts, calculating certain characteristic quantities, etc. Data exploration helps to select appropriate data preprocessing and modeling methods. This project will explore the collected power grid structured data from two perspectives: data quality analysis and data feature analysis. The following steps are included: (1) Data quality analysis: 1) Missing value analysis: Count the proportion of missing data in each feature of the basic ledger of power grid equipment, online monitoring data, live monitoring data, and offline test data; 2) Outlier analysis: Using 3 The outliers in the power grid data are usually determined by the principle. The power grid data usually follows the normal distribution. In principle, an outlier is defined as a value in a group of measurements that deviates from the mean by more than 3 standard deviations.

[0043] (2) Data feature analysis 1) Distribution analysis: For quantitative power grid data, the following are performed: 1. Find the range: range = maximum value - minimum value; 2 Determine the class interval and number of groups: number of groups = range / class interval; 3. Decide on the points; 4. List the frequency distribution table; 5 Plot a frequency distribution histogram.

[0044] For qualitative data, they are grouped according to the classification type of the variables, and pie charts and bar charts are used to describe the distribution of qualitative variables.

[0045] 2) Statistical analysis: It is used to describe quantitative data with statistical indicators, and analyze from two aspects: central tendency and dispersion tendency.

[0046] The central tendency is analyzed by the mean. The formulas are as follows: , where x is the value of a feature column of the power grid data, and n is the number of feature values.

[0047] The deviation trend is analyzed using standard deviation, and the formula is as follows: ,in is the characteristic value of a column of power grid data, is the average value of the column, and n is the number of values ​​in the column.

[0048] 3) Periodicity analysis: Periodicity analysis is to explore whether a variable in power grid data shows a certain periodic trend over time. The periodic trends with relatively long time scales include annual periodicity trends and seasonal periodicity trends, and the relatively short ones include monthly periodicity trends and weekly periodicity trends. The shorter ones include daily and hourly periodicity trends. By drawing a time series line graph to observe the trend of change, the periodicity of power grid data can be analyzed.

[0049] 4) Correlation analysis: Analyze the strength of the linear correlation between continuous variables in power grid data. The specific steps are as follows: 1 Draw a scatter plot of the variables associated with the power grid data; 2. Draw a scatter matrix diagram of the associated variables of the power grid data; 3 Calculate the correlation coefficient. This project uses the Person correlation coefficient. The specific formula is as follows:

[0050] in, and are the values ​​of two related variables, and is the mean of the two associated variables.

[0051] The value range of the correlation coefficient r is: -1≤r≤1

[0052] 0<|r|<1 indicates the existence of different degrees of linear correlation:

[0053] The data exploration of power grid data will adopt the above steps and principles, explore the collected power grid data from the two perspectives of data quality analysis and data feature analysis, and provide support for determining a reasonable method for the next step of data processing.

[0054] Data overview: Check the data type of each feature in the data set, such as checking the type (numeric, category, text) and naming conventions of each field in the basic ledger of power grid equipment, checking whether there are missing values ​​or duplicate values, and counting the number and proportion of missing values ​​and duplicate values. Statistical description: Check the data distribution in the data set, such as the maximum value, minimum value, mean, standard deviation, variance, etc. in online monitoring data, and quickly obtain the statistical category distribution; for example, the frequency of the basic ledger category of power grid equipment. Data visualization: Use visual charts to analyze data anomalies and category frequencies. For example, the power grid test data analyzes the anomalies of the data set based on histograms, box plots, and scatter plots, and analyzes the category frequency of the data set based on bar charts.

[0055] ③Data preprocessing: Data deduplication: remove duplicate data records from a data set.

[0056] Missing value processing: interpolation or deletion of missing data. When the proportion of missing values ​​in the data set is small, or the data points where the missing values ​​are located are not important for the overall analysis, you can consider directly deleting the records with missing values; when the missing values ​​cannot be simply deleted, you can use a global constant (such as the mean, median, or mode) to fill them, or use the mean, median, or mode of a specific attribute to fill them, or use the attribute value of the most similar sample to fill them.

[0057] There are three types of methods for processing missing values ​​in power grid data. The specific principles and steps are as follows: (1) When the missing values ​​in the data set account for a small proportion and the missing data is not important for the overall analysis, the missing data is directly deleted; (2) If the missing values ​​in the data set account for a small proportion and the existence of missing values ​​has little impact on the overall analysis, they will not be processed; (3) When the existence of missing values ​​has a significant impact on the overall analysis, the following methods are used for interpolation: 1) For missing value features with small changes and values ​​of specific fixed values, the average value of the feature is used for interpolation; Find a known n-1 degree polynomial passing through n points: The coordinates of n points Substituting into the polynomial function, we get

[0058] Solving the Lagrange difference polynomial is:

[0059] Substitute the point x corresponding to the missing function value into the difference polynomial to obtain the approximate value L(x) of the missing value.

[0060] Data transformation: Convert the data into a form suitable for analysis, such as standardization, which standardizes the data so that each feature has zero mean and unit variance, which is suitable for situations where the data distribution is close to normal. Normalization, which scales the data to a fixed range, usually [0, 1] or [-1, 1], is suitable for non-normally distributed data.

[0061] Since different indicators often have different dimensions, the difference between values ​​may be large. In order to eliminate the impact of the differences in dimensions and value ranges between indicators, standardization is performed to scale the data so that it falls into a specific area to facilitate subsequent analysis. The specific principles and steps are as follows: (1) For power grid data that obeys normal distribution, zero-mean normalization is used, and the mean of the processed data is 0 and the standard deviation is 1. The transformation formula is: ,in is the mean of the original data, is the standard deviation of the original data.

[0062] (2) For power grid data that does not conform to the normal distribution, the minimum-maximum normalization is used to perform a linear transformation on the original data and map the values ​​to [0,1]. The conversion formula is as follows: ,in is the maximum value of the sample data, is the minimum value of the sample data. Very poor.

[0063] Data reduction: Reduce the dimension and complexity of data through dimensionality reduction and other methods. Through principal component analysis (PCA), the covariance matrix is ​​calculated based on the standardized data, and the covariance matrix is ​​eigen-decomposed to obtain the eigenvalues ​​and corresponding eigenvectors. The most important eigenvectors are selected as new principal components according to the size of the eigenvalues. Apply the selected dimensionality reduction method to map the original data to a low-dimensional space, and then use the reduced-dimensional data for specific analysis and processing to reduce the number of features in the data set, reduce computing costs and storage requirements, and reduce the risk of overfitting.

[0064] Data reduction creates new attribute dimensions by merging attributes, or directly reduces data dimensions by deleting irrelevant attributes (dimensions), thereby improving efficiency and reducing computing costs. This project uses the PCA (Principal Component Analysis) method for data reduction. The specific calculation steps are as follows: (1) Let the original variable The n-times observation data matrix is:

[0065] (2) Center the data matrix columns. For convenience, the standardized data matrix is ​​still labeled as X.

[0066] (3) Find the correlation matrix R, , is defined as: ,in = , =1,.

[0067] (4) Find the characteristic equation of R .

[0068] (5) Determine the number of principal components m: , Determined based on actual problems, this project takes 80%.

[0069] (6) Calculate m corresponding unit eigenvectors:

[0070] (7) Calculate the principal components:

[0071] ④Feature selection: Select features that are more relevant to the problem from the original data to reduce the data dimension and complexity.

[0072] For example, in fault diagnosis, the Pearson correlation coefficient between each monitoring parameter (such as current, voltage, power, etc.) and the fault type can be calculated, and the features with higher correlation coefficients can be selected as key features; support vector machines (SVMs) can be used for feature selection for power grid load forecasting; in power grid status assessment, Lasso regression can be used for feature selection, etc.

[0073] (1) In the power grid state evaluation data, Lasso regression is used for feature selection. The specific steps are as follows: 1) Define the objective function: Add the L1 regularization term to the original least squares objective function to obtain the latest objective function, as shown in the following formula: ; 2) Solve the optimization problem and find the model parameters that minimize the objective function through iteration; 3) Feature selection: for features with non-zero coefficients, keep their corresponding features; for features with zero coefficients, remove them.

[0074] (2) For power grid load forecast data, SVM is used for feature selection. The specific steps are as follows: 1) Initialize the original feature set S = {1, 2, ..., D}, and the feature sorting set R = [] 2) Loop the process until S=[] Obtain training samples with candidate feature sets; use Train the SVM classifier and get w; use Find the one with the smallest difference sorting score ; Update feature set R=[p,R]; Remove features from S: S=S / p Unified data integration and application According to data needs, build a data center with data reception, storage and pre-processing functions: ①Configuration release of data set: Some data subsets in the database are configured and published regularly, and two forms of direct connection to the database and API service are provided for data differentiation configuration, providing online debugging function.

[0075] ②Visual display of data: Using reports or graphics to display data and present the final results of the entire data processing can help you understand the data situation intuitively, quickly and directly, and understand the data more intuitively.

[0076] Intelligent analysis and push ① Question analysis: By allowing the large model to understand the questions asked by users through natural semantics, and then searching and matching in the data generated by data mining and analysis.

[0077] ②Result push: Perform data query based on the big model, encapsulate it into a data set that matches the built scenario, and automatically push the data set to the target system.

[0078] Visual interactive construction ① Design visualization: Design visualization logic according to project requirements, including dragging and dropping of various components and configuring parameters, events, interactions, etc. to complete page display, and apply data sets to components to present the final effect.

[0079] ②Select data source: The data in the above steps are unified and integrated, and the result data set in the processing step is selected through direct connection to the database and API service.

[0080] ③Preview and share: Preview, share, or publish the completed visualization scene, and update the visualization content and design in a timely manner based on business changes or user feedback.

[0081] Part II: Improve grid operation efficiency: Through intelligent analysis and planning software, the value of data can be fully utilized to quickly respond to and solve problems, thereby improving the overall operating efficiency of the power grid.

[0082] Enhanced decision-making quality: Intelligent analysis and prediction functions can provide scientific decision-making support, help managers make more reasonable strategic choices, and improve overall management level.

[0083] Increase customer satisfaction: Software can enhance customer satisfaction and trust by optimizing grid services, such as reducing outages and improving power quality.

[0084] It is easy for those skilled in the art to understand that the above-described working conditions are only preferred embodiments of the present invention and are not intended to limit the present invention. Any example analysis, equivalent substitution, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A power grid data analysis and prediction method, characterized in that: include Obtain the original data, analyze and process it, select data features, and construct and store the data that meets the data features into a data set; Build business scenarios and select data sets based on manual or intelligent building; An interactive component is constructed, wherein the interactive component retrieves the selected data set and outputs interactive information.

2. A power grid data analysis and prediction method according to claim 1, characterized in that: include Conduct preliminary analysis of the structure and patterns of sample data sets, including data overview, statistical description, and data visualization; Preprocess the data after pre-analysis, including data deduplication, missing value processing, data conversion, and data reduction; Retrieve highly relevant features from the preprocessed data.

3. A power grid data analysis and prediction method according to claim 1, characterized in that: The original data includes basic records of power grid equipment, online monitoring data, live detection data, and offline test data.

4. A power grid data analysis and prediction method according to claim 1, characterized in that: The following steps are involved: Data Overview: Check the data type of each feature in the data set, including the type and naming convention of each field in the basic ledger of power grid equipment, check whether there are missing values ​​or duplicate values, and count the number and proportion of missing values ​​and duplicate values; Statistical description: Check the data distribution in the data set, including the maximum value, minimum value, mean value, standard deviation, and variance in the online monitoring data, and quickly obtain the statistical category distribution; as well as the frequency of the basic account category of power grid equipment; Data visualization: Use visual charts to analyze data anomalies and category frequencies. This includes analyzing the anomalies of power grid test data using histograms, box plots, and scatter plots, and analyzing the category frequencies of data using bar charts.

5. A power grid data analysis and prediction method according to claim 1, characterized in that: The following steps are involved: Data deduplication: remove duplicate data records in the data set; Missing value processing: interpolation or deletion of missing data; Data conversion: standardize the data; Data reduction: The covariance matrix is ​​calculated based on the standardized data through principal component analysis, and the covariance matrix is ​​eigendecomposed to obtain the eigenvalues ​​and corresponding eigenvectors. The most important eigenvectors are selected as new principal components according to the size of the eigenvalues.

6. A power grid data analysis and prediction method according to claim 1, characterized in that: In missing value processing, When the proportion of missing values ​​in the data set is small, or the data points where the missing values ​​are located are not important for the overall analysis, delete the records containing missing values; When missing values ​​cannot be simply deleted, fill them with a global constant, or fill them with the mean, median, or mode of a specific attribute, or fill them with the attribute value of the most similar sample.

7. A power grid data analysis and prediction method according to claim 1, characterized in that: Build a data set that provides the data foundation for the component. When you choose to build manually, the data set is generated through the database and API service. The generation of the data set includes: Directly connect to the database: configure database information to connect to the database after data processing, directly query and debug the database table online, store the query results as a data set, and provide data for component rendering; API service: configures the interface address and request method, performs queries and online debugging, stores query results as data sets, and provides data for component rendering.

8. A power grid data analysis and prediction method according to claim 1, characterized in that: When choosing smart building, include: The user asks a query; the big model performs semantic analysis on the user's question, generates a query SQL statement and a data set, and generates a recommended chart component based on the user's question; If the recommended dataset and components meet the user's question requirements, the dataset will be stored and pushed; otherwise, new datasets and recommended components will be re-optimized and generated until the user's question requirements are met.

9. A power grid data analysis and prediction method according to claim 1, characterized in that: The interactive components include: Basic components: including text, media, image, and video components, which are used for scene display of various static resources or dynamic address rendering; Table form components: including buttons, radio check boxes, input boxes, and table type components, which are applied to various business operations and list detail display scenarios; Chart components: including bar charts, pie charts, line charts, scatter charts, and radar charts, which are used for displaying various data aggregation and statistical scenarios; Resource components: including background, border, icon, and decorative components, which are used for decoration of various scenes; Map components: including echarts map, offline map, and Siji map components, which are used for scene display of map rendering and map data interaction.

10. A power grid data analysis and prediction system, characterized in that: include: The first module is configured to obtain raw data, analyze and process the raw data, select data features, construct data that meet the data features into a data set, and store the data; The second module is configured to build a business scenario and select a data set based on manual or intelligent building; The third module is configured to construct an interactive component, and the interactive component retrieves the selected data set and outputs interactive information.

Images