Power grid business data feature extraction method

Abstract

The invention discloses a power grid business data feature extraction method, which relates to the technical field of power grids and comprises the following steps: S1, data acquisition and preprocessing; s2, performing feature screening; s3, feature extraction; and S4, feature verification and optimization. According to the power grid business data feature extraction method, the related actual problem features are analyzed and selected before feature extraction, so that the problem that the extracted features are possibly not highly related to the actual problems or are too high in redundancy is avoided, and the extracted features can effectively support subsequent data analysis and decision making; meanwhile, the reduction of meaningless extraction operation is helpful for reducing the workload of power grid service data feature extraction and improving the feature extraction efficiency, in addition, when high-dimensional data is processed, the data dimension and the calculation complexity are reduced through dimension reduction processing, the required calculation resources are evaluated and prepared before task execution, and the calculation efficiency is improved. And smooth feature extraction is ensured.

Description

Technical Field

[0001] The present invention relates to the field of power grid technology, and in particular to a method for extracting power grid service data features. Background Art

[0002] The power grid is a unified whole that connects power generation and power consumption, consisting of power transmission, substation, distribution equipment and corresponding auxiliary systems. The function of the power grid is to transmit the electric power generated by the power plant to the substation bus through the transmission line, collect and distribute the electric power of the same voltage level through the bus, and distribute it to the distribution network of different voltage levels through the transformer. The power grid can generally be divided into a transmission network and a distribution network, and its operation control is implemented in a unified command by the power dispatching control center.

[0003] Grid business data refers to all kinds of data related to grid operation, management, and maintenance, including grid production data, grid operation data, and grid enterprise management data. Grid business data is large in scale, diverse in types, and has high timeliness requirements. It is an important basis for grid management and operation decision-making. In order to extract effective information from massive and complex grid business data for more efficient data analysis, processing, and decision-making, it is necessary to extract features from grid business data.

[0004] For example, the patent document with application number 202311505453.5 discloses a method for extracting features of power grid business data. This method is based on the establishment of a multi-scale deep feature fusion model, which can quickly extract power grid data sets with good scalability and adapt to different scales and complexities and combine the data features in the data sets, thereby reducing the computational overhead of the model and the complexity of business data analysis, and realizing fast and efficient extraction of power grid data features. By closely combining feature extraction with actual business needs, it can more effectively reflect the actual status of power grid equipment, which is of great significance to improving the accuracy and timeliness of power grid equipment fault prediction.

[0005] However, the existing power grid business data feature extraction similar to the above application still has the following shortcomings: First, when processing massive, high-dimensional power grid business data, it is easy to encounter the problem of dimensionality disaster, resulting in excessive computational complexity and resource consumption. In addition, due to the lack of analysis and selection of the actual problem features involved before feature extraction, some of the extracted features may not be highly relevant to the actual problem or are too redundant, and cannot effectively support subsequent data analysis and decision-making. Such meaningless extraction operations will increase the workload of power grid business data feature extraction and affect the efficiency of feature extraction.

[0006] Therefore, there is an urgent need to improve this shortcoming. The present invention studies and improves the existing structure and deficiencies, and provides a method for extracting power grid business data features. Summary of the invention

[0007] The purpose of the present invention is to provide a method for extracting features of power grid service data to solve the problems raised in the above background technology.

[0008] To achieve the above object, the present invention provides the following technical solution: a method for extracting features of power grid business data, comprising the following steps: S1. Data collection and preprocessing: Collect and organize business data of all aspects of the power grid, including operation, management, and maintenance, in accordance with unified formats and standards, and preprocess the data; S2. Feature screening: Based on business experience and domain knowledge, preliminarily screen out features related to actual problems, then use statistical methods or machine learning algorithms to evaluate the correlation between features and target variables, weight the correlations calculated by different methods based on data analysis, obtain comprehensive correlations, and select the final features based on the comprehensive correlations; S3. Feature extraction: For high-dimensional data, select appropriate methods to reduce the dimensionality, and use deep learning models to learn the features of power grid business data to extract deep features; S4. Feature verification and optimization: Verify the effectiveness of the extracted features through experiments or actual applications, and optimize the feature extraction method based on the verification results, such as adjusting model parameters, improving algorithms, etc.

[0009] Furthermore, in step S1, the power grid business data includes the following collection methods: On-site monitoring: Install sensors on power plants, transmission lines, substations and other equipment to monitor power production, transmission and consumption in real time; Automation system: Establish an automatic collection and transmission system for power market data, automatically collect data such as power prices and transaction volumes through instruments, sensors, and communication equipment, and transmit the data to the data center through the network to realize automatic data collection and processing; Manual survey: Collect qualitative data through questionnaires, interviews, data declaration and annual reports; Database collection: directly integrate with the enterprise business backend server to write the business records generated by the enterprise business backend into the database to collect historical data, quality and safety data of the power market; System log collection: collects log data for offline and online real-time analysis, and is used to extract information such as power operation, faults, and maintenance from system logs; Web data collection: Obtain data from websites through web crawlers and public APIs provided by website platforms to extract, clean, and convert unstructured and semi-structured data into structured data; Industry platforms and agency websites: Visit data platforms, government agency websites or corporate websites related to the power industry, such as China Electric Power Knowledge Base, National Energy Administration, China Electricity Council, etc.

[0010] Furthermore, in step S1, the preprocessing specifically includes the following operations: Data deduplication: remove duplicate items from the data to avoid the impact of duplicate data on subsequent analysis; Missing value processing: For missing values, linear interpolation or polynomial interpolation is used to fill them, or they are supplemented according to relevant rules; Outlier detection and processing: Detect outliers in data through statistical analysis or data mining, and perform corresponding processing, such as deleting or replacing them with mean or median; Data denoising: remove noise from data through filtering and other methods; Data standardization: converting data to the same dimension to facilitate feature comparison and model training; Data reduction: reduce the dimension and quantity of data through data compression and dimensionality reduction; Data validation: Validate the processed data to ensure that the data complies with predetermined rules and requirements.

[0011] Furthermore, in step S2, the preliminary screening specifically includes the following operations: Clarify business needs and problems: Clarify the actual needs of power grid business and the specific problems that need to be solved; Collect domain knowledge: Collect professional knowledge in the field of power grid, including the topology, operation principle, equipment characteristics, etc. of the power grid, and understand the industry standards, specifications and best practices of power grid business; Understand data sources: Study the source, structure and attributes of power grid business data, including real-time monitoring data, historical record data, equipment status data, etc., and identify key fields and potential features in the data; Preliminary screening based on experience: Using the long-term experience of analysts or experts, preliminary screening of potential features is carried out, focusing on features that are closely related to practical problems and have a significant impact on grid operation; Feature importance assessment: Preliminary assessment of the importance of the selected features for solving practical problems, and use evaluation by business experts, simple statistical analysis or visualization tools to help understand the relationship between features and target variables; Eliminate irrelevant or redundant features: Based on business experience and domain knowledge, eliminate features that are irrelevant or redundant to the actual problem.

[0012] Furthermore, in step S2, the statistical method is based on probability theory and mathematical statistics, and is used to evaluate the statistical relationship between two or more variables, and detect the nonlinear relationship between features and target variables. It is applicable to the correlation analysis between discrete features and discrete target variables, and does not require a large amount of computing resources. It is not applicable to the relationship between continuous features and continuous target variables. The statistical method specifically includes: Chi-square test: It determines whether there is a significant association between two categorical variables by comparing the difference between the observed frequency and the expected frequency, and does not rely on the distribution assumption of the data; Pearson correlation coefficient: The Pearson correlation coefficient is calculated by the covariance and standard deviation of two variables to measure the degree of linear correlation between the two variables. The value range is [-1, +1], where -1 indicates a complete negative correlation, +1 indicates a complete positive correlation, and 0 indicates no linear correlation. Spearman correlation coefficient: a non-parametric correlation analysis method used to measure the degree of monotonic correlation between two variables. It does not require the variables to be linearly related and has no strict requirements on the distribution of data. The value range is [-1, +1]; Coefficient of determination: a statistical indicator that measures the degree to which an independent variable explains a dependent variable. It is applicable to regression models and indicates the proportion of the variation of the dependent variable that can be explained by the independent variable. Its value range is [0, +1], and the closer the value is to 1, the higher the degree to which the independent variable explains the dependent variable. Hypothesis test: used to test whether there is a statistically significant correlation between two variables. By setting up the null hypothesis and alternative hypothesis, calculating statistics (such as t value, F value, etc.), and calculating the p value, and comparing it with the significance level to draw a conclusion; Regression analysis: A statistical method used to establish a mathematical model between variables. It can measure the degree of linear correlation between two variables and predict the value of the dependent variable. It includes linear regression, polynomial regression, logistic regression and other types. It determines the relationship between two variables by minimizing the residual sum of squares between the dependent variable and the independent variable.

[0013] Furthermore, the weighting based on data analysis in step S2 is specifically as follows:

[0014] in, a i represents the weight obtained based on data analysis, i Indicates i A correlation calculation method, b i Indicates i The correlation calculation results are n is the total number of methods used to calculate correlation; a i The calculation process is as follows: ; in a 0 is the preset initial weight, f j is the confidence level, obtained from the result of feature verification, k is the number of iterations, and each round of verification is performed. k Will add 1, Δ a is the preset step size.

[0015] Furthermore, in step S3, the dimensionality reduction process specifically includes the following method: Principal component analysis: It converts data into a low-dimensional space through orthogonal transformation while retaining key information in the data as much as possible. It is suitable for processing numerical data. t-distributed random neighbor embedding: suitable for embedding high-dimensional data into two-dimensional or three-dimensional space for visualization; Linear discriminant analysis: find the feature subspace that maximizes the difference between classes and minimizes the difference within classes; Local linear embedding: It is used to process nonlinear high-dimensional data by finding the local linear structure of high-dimensional data. That is, each point representing high-dimensional data in low-dimensional space can be reconstructed through linear combinations on the neighborhood. It is suitable for data sets with nonlinear structures. Multidimensional scaling analysis: It converts the distance between each point in a high-dimensional data set into the Euclidean distance in a low-dimensional space for dimensionality reduction, aiming to preserve the distance relationship between different data points; Singular value decomposition: It decomposes the data to find the key features or principal components in the data. It is an extension of principal component analysis.

[0016] Furthermore, in step S3, according to the characteristics and requirements of different power grid business data, a suitable deep learning model is selected, and the power grid business data is processed using data processing methods including convolutional neural networks, recurrent neural networks, long short-term memory networks, and gated recurrent units. Before performing deep learning tasks, computing resources are fully evaluated and prepared to ensure smooth training and testing of the model.

[0017] Furthermore, in step S4, the specific operation of feature verification is as follows: prepare an original data set of power grid business data, and divide it into a training set, a validation set and a test set in a ratio of 60%:20%:20% or 70%:15%:15%, and then extract features from the original data through steps S1 to S3, and build a prediction or classification model based on the extracted features, then train the model on the training set, and monitor the performance of the model on the validation set in real time, and then use feature verification methods including performance evaluation indicators, cross-validation, feature importance evaluation, and baseline model comparison for verification, and at the same time deploy the model to an actual power grid business scenario, use real-time data for verification, monitor the performance of the model in actual applications, including prediction accuracy, response time, etc., and verify the validity of the extracted features through multiple evaluation methods and actual application scenarios.

[0018] Furthermore, the feature verification method specifically includes: 1) Performance evaluation indicators Accuracy: the ratio of the number of correctly classified or predicted samples to the total number of samples; Recall rate: the ratio of the number of instances correctly predicted as positive samples to the number of all positive sample instances, applicable to classification tasks; F1 score: The harmonic mean of precision and recall, used to balance the two; Mean square error or mean absolute error: measures the error between the predicted value and the actual value, suitable for regression tasks; 2) Cross-validation Use K-fold cross validation, divide the data set into K parts each time, use K-1 parts for training, and the remaining part for testing; Repeat K times, making sure that each piece of data is used as a test set, and record the performance indicators each time; Calculate the mean and standard deviation of K performance indicators to evaluate the stability and generalization ability of the model; 3) Feature Importance Evaluation For tree-based models, feature importance scores are used to measure the contribution of each feature to the model performance; Analyze the correlation between features to ensure there are no redundant or highly correlated features; 4) Baseline model comparison Set up a baseline model that does not use the extracted features; Compare the performance of the baseline model with the model using the extracted features under the same dataset and evaluation criteria; If the model using feature extraction significantly outperforms the baseline model in terms of performance metrics, the extracted features are considered effective.

[0019] The present invention provides a method for extracting power grid service data features, which has the following beneficial effects: The present invention analyzes and selects the features of the actual problems involved before feature extraction, thereby avoiding the problem that the extracted features may not be highly relevant to the actual problems or have excessive redundancy. In addition, through weighted processing based on data analysis and weight calculation before processing, the extracted features can effectively support subsequent data analysis and decision-making, while reducing meaningless extraction operations, which helps to reduce the workload of power grid business data feature extraction and improve feature extraction efficiency. In addition, when processing high-dimensional data, not only the data dimension and calculation complexity are reduced through dimensionality reduction processing, and the required computing resources are evaluated and prepared before the task is executed to ensure the smooth progress of feature extraction, but also the effectiveness of the extracted features is verified through experiments and practical applications to ensure that the extracted feature results are accurate and reliable. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 A schematic diagram of the operation flow of a method for extracting features of power grid business data according to the present invention; Figure 2 A schematic diagram of a preliminary screening process of a method for extracting features of power grid business data according to the present invention; Figure 3 A schematic diagram of a statistical method for extracting power grid service data features according to the present invention; Figure 4 A schematic diagram of a feature verification method of a power grid business data feature extraction method of the present invention. DETAILED DESCRIPTION

[0021] The following embodiments of the present invention are described in further detail in conjunction with the accompanying drawings and examples. The following examples are used to illustrate the present invention, but are not intended to limit the scope of the present invention.

[0022] like Figure 1-Figure 4 As shown, a method for extracting features of power grid business data includes the following steps: S1. Data collection and preprocessing: Collect and organize business data of all aspects of the power grid, including operation, management, and maintenance, in accordance with unified formats and standards, and preprocess the data; In step S1, the power grid business data includes the following collection methods: On-site monitoring: Install sensors on power plants, transmission lines, substations and other equipment to monitor power production, transmission and consumption in real time. This method can collect quantitative data in real time, such as real-time data on power production and consumption, real-time values ​​of power load and power consumption, etc. Automation system: Establish an automatic collection and transmission system for power market data, automatically collect data such as power prices and transaction volumes through instruments, sensors, and communication equipment, and transmit the data to the data center through the network to realize automatic data collection and processing; Manual survey: Collect qualitative data through questionnaires, interviews, data reporting and annual reports. These data include the qualifications of power market participants, the competition status of the power market, power market policies and regulations, etc. Database collection: directly integrate with the enterprise business backend server to write the business records generated by the enterprise business backend into the database to collect historical data, quality and safety data of the power market; System log collection: collects log data for offline and online real-time analysis, and is used to extract information such as power operation, faults, and maintenance from system logs; Web data collection: Obtain data from websites through web crawlers and public APIs provided by website platforms to extract, clean, and convert unstructured and semi-structured data into structured data; Industry platforms and institutional websites: Visit data platforms, government agency websites or corporate websites related to the power industry, such as China Electric Power Knowledge Base, National Energy Administration, China Electricity Council, etc. These websites provide information, data, reports, etc. on the power industry; In step S1, the preprocessing specifically includes the following operations: Data deduplication: remove duplicate items from the data to avoid the impact of duplicate data on subsequent analysis; Missing value processing: For missing values, linear interpolation or polynomial interpolation is used to fill them, or they are supplemented according to relevant rules; Outlier detection and processing: Detect outliers in data through statistical analysis or data mining, and perform corresponding processing, such as deleting or replacing them with mean or median; Data denoising: remove noise from data through filtering and other methods; Data standardization: converting data to the same dimension to facilitate feature comparison and model training; Data reduction: reduce the dimension and quantity of data through data compression and dimensionality reduction; Data verification: Verify the processed data to ensure that the data complies with the predetermined rules and requirements; S2. Feature screening: Based on business experience and domain knowledge, preliminarily screen out features related to actual problems, then use statistical methods or machine learning algorithms to evaluate the correlation between features and target variables, weight the correlations calculated by different methods based on data analysis, obtain comprehensive correlations, and select the final features based on the comprehensive correlations; It should be noted that in step S2, the preliminary screening specifically includes the following operations: Clarify business needs and problems: Clarify the actual needs of power grid business and the specific problems that need to be solved; Collect domain knowledge: Collect professional knowledge in the field of power grid, including the topology, operation principle, equipment characteristics, etc. of the power grid, and understand the industry standards, specifications and best practices of power grid business; Understand data sources: Study the source, structure and attributes of power grid business data, including real-time monitoring data, historical record data, equipment status data, etc., and identify key fields and potential features in the data; Preliminary screening based on experience: Using the long-term experience of analysts or experts, preliminary screening of potential features is carried out, focusing on features that are closely related to practical problems and have a significant impact on grid operation; Feature importance assessment: Preliminary assessment of the importance of the selected features for solving practical problems, and use evaluation by business experts, simple statistical analysis or visualization tools to help understand the relationship between features and target variables; Eliminate irrelevant or redundant features: Based on business experience and domain knowledge, eliminate features that are irrelevant to the actual problem or redundant. Redundant features refer to those that are highly correlated with other features and carry duplicate information. In step S2, the statistical method is based on probability theory and mathematical statistics, and is used to evaluate the statistical relationship between two or more variables, and to detect the nonlinear relationship between features and target variables. It is applicable to the correlation analysis between discrete features and discrete target variables, and does not require a large amount of computing resources. It is not applicable to the relationship between continuous features and continuous target variables. The statistical method specifically includes: Chi-square test: It determines whether there is a significant association between two categorical variables by comparing the difference between the observed frequency and the expected frequency, and does not rely on the distribution assumption of the data; Pearson correlation coefficient: The Pearson correlation coefficient is calculated by the covariance and standard deviation of two variables to measure the degree of linear correlation between the two variables. The value range is [-1, +1], where -1 indicates a complete negative correlation, +1 indicates a complete positive correlation, and 0 indicates no linear correlation. Spearman correlation coefficient: a non-parametric correlation analysis method used to measure the degree of monotonic correlation between two variables. It does not require the variables to be linearly related and has no strict requirements on the distribution of data. The value range is [-1, +1]; Coefficient of determination: a statistical indicator that measures the degree to which an independent variable explains a dependent variable. It is applicable to regression models and indicates the proportion of the variation of the dependent variable that can be explained by the independent variable. Its value range is [0, +1], and the closer the value is to 1, the higher the degree to which the independent variable explains the dependent variable. Hypothesis test: used to test whether there is a statistically significant correlation between two variables. By setting up the null hypothesis and alternative hypothesis, calculating statistics (such as t value, F value, etc.), and calculating the p value, and comparing it with the significance level to draw a conclusion; Regression analysis: A statistical method used to establish a mathematical model between variables. It can measure the linear correlation between two variables and predict the value of the dependent variable. It includes linear regression, polynomial regression, logistic regression and other types. It determines the relationship between two variables by minimizing the residual sum of squares between the dependent variable and the independent variable. The weighting based on data analysis in step S2 is specifically as follows:

[0023] in, a i represents the weight obtained based on data analysis, i Indicates i A correlation calculation method, b i Indicates i The correlation calculation results are n is the total number of methods used to calculate correlation; For example, b 1 for the correlation obtained using the chi-square test; b 2 is the calculated Pearson correlation coefficient, b 3 is the calculated Spearman correlation coefficient; b 4 is the calculated determination coefficient, b 5 is the correlation obtained through hypothesis testing, b 6 is the correlation obtained by regression analysis; and a i The calculation process is as follows: ; in a 0 is the preset initial weight, f j is the confidence level, obtained from the result of feature verification, k is the number of iterations, and each round of verification is performed. k Will add 1, Δ a is the preset step size.

[0024] It should be noted that the present invention obtains a 0The value of f j The value of is obtained from the confidence of the verification result of the feature. For example, if a data feature passes the final verification result or indicates validity, it can be assigned a value of 1 or 0.9, etc. If it fails or is invalid, it can be assigned a value of 0 or 0.1, etc. k Indicates the number of evaluations of a certain data. For example, in the first evaluation, the evaluation is invalid, but in the second evaluation, the evaluation is valid. At this time, there should be a fault tolerance measure for the data. The k in the present invention is also the corresponding fault tolerance measure. Each time an evaluation is performed, the validity of the feature data can be comprehensively and objectively evaluated. The preset step size Δ a , you can adjust the positive and negative values ​​and their own values ​​in real time.

[0025] In general, the present invention obtains a correlation coefficient value by weighting multiple correlation coefficients based on data analysis. Through this value, a preset value or range can be judged. If it is within the preset range, the corresponding feature can be extracted, otherwise it will not be extracted.

[0026] In step S2, the machine learning algorithm automatically establishes a relationship model between features and target variables by learning the inherent laws and patterns of the data, such as building a machine learning model, such as a random forest, a neural network, etc., and using the relationship between the output and input features of the model to evaluate the importance of the features; S3. Feature extraction: For high-dimensional data, select appropriate methods to reduce the dimensionality, and use deep learning models to learn the features of power grid business data to extract deep features; It should be noted that, in step S3, before feature extraction, weighting is performed, and after weighting, dimensionality reduction is performed.

[0027] For example, if the selected feature has a strong correlation with the target feature and the correlation weight is c, as calculated by the above formula, the value of the feature is multiplied by the weight and used as the processed value, and then dimensionality reduction is performed to reduce the amount of calculation.

[0028] In step S3, the dimensionality reduction process specifically includes the following methods: Principal component analysis: It converts data into a low-dimensional space through orthogonal transformation while retaining key information in the data as much as possible. It is suitable for processing numerical data and has good processing capabilities for linear relationships of data. t-distributed random neighbor embedding: suitable for embedding high-dimensional data into two-dimensional or three-dimensional space for visualization. t-distributed random neighbor embedding can maintain the local structure in the data, making the reduced-dimensional data easier to understand visually. Linear discriminant analysis: finds the feature subspace that can maximize the difference between classes and minimize the difference within classes. Linear discriminant analysis pays special attention to the category label of the data, so that after the data is projected, the data points of the same class are as close as possible, and the data points of different classes are as far away as possible; Local linear embedding: It is used to process nonlinear high-dimensional data by finding the local linear structure of high-dimensional data. That is, each point representing high-dimensional data in low-dimensional space can be reconstructed through linear combinations on the neighborhood. It is suitable for data sets with nonlinear structures. Multidimensional scaling analysis: It converts the distance between each point in a high-dimensional data set into the Euclidean distance in a low-dimensional space for dimensionality reduction, aiming to preserve the distance relationship between different data points; Singular value decomposition: By decomposing the data, the key features or principal components in the data are found, which is an extension of principal component analysis; In step S3, according to the characteristics and requirements of different power grid business data, a suitable deep learning model is selected, and the power grid business data is processed using data processing methods including convolutional neural networks, recurrent neural networks, long short-term memory networks, and gated recurrent units. Before performing deep learning tasks, computing resources are fully evaluated and prepared to ensure smooth training and testing of the model; S4. Feature verification and optimization: Verify the effectiveness of the extracted features through experiments or actual applications, and optimize the feature extraction method based on the verification results, such as adjusting model parameters, improving algorithms, etc. In step S4, the specific operation of feature verification is as follows: prepare the original data set of power grid business data, and divide it into a training set, a validation set and a test set in a ratio of 60%:20%:20% or 70%:15%:15%, then extract features from the original data through steps S1 to S3, and build a prediction or classification model based on the extracted features, then train the model on the training set, and monitor the performance of the model on the validation set in real time, and then use feature verification methods including performance evaluation indicators, cross-validation, feature importance evaluation, and baseline model comparison for verification, and at the same time deploy the model to the actual power grid business scenario, use real-time data for verification, monitor the performance of the model in actual applications, including prediction accuracy, response time, etc., and verify the validity of the extracted features through multiple evaluation methods and actual application scenarios; wherein, the feature verification method specifically includes: 1) Performance evaluation indicators Accuracy: the ratio of the number of correctly classified or predicted samples to the total number of samples; Recall rate: the ratio of the number of instances correctly predicted as positive samples to the number of all positive sample instances, applicable to classification tasks; F1 score: The harmonic mean of precision and recall, used to balance the two; Mean square error or mean absolute error: measures the error between the predicted value and the actual value, suitable for regression tasks; 2) Cross-validation Use K-fold cross validation (such as K=5 or 10), divide the data set into K parts each time, use K-1 parts for training, and the remaining part for testing; Repeat K times, making sure that each piece of data is used as a test set, and record the performance indicators each time; Calculate the mean and standard deviation of K performance indicators to evaluate the stability and generalization ability of the model; 3) Feature Importance Evaluation For tree-based models (such as random forests), use feature importance scores to measure the contribution of each feature to the model performance; Analyze the correlation between features to ensure there are no redundant or highly correlated features; 4) Baseline model comparison Establish a baseline model that does not use the extracted features (e.g., using only raw data or traditional statistical methods); Compare the performance of the baseline model with the model using the extracted features under the same dataset and evaluation criteria; If the model using feature extraction significantly outperforms the baseline model in terms of performance metrics, the extracted features are considered effective.

[0029] The embodiments of the present invention are given for the purpose of illustration and description, and are not intended to be exhaustive or to limit the invention to the disclosed forms. Many modifications and variations will be apparent to those of ordinary skill in the art. The embodiments are selected and described in order to better illustrate the principles and practical applications of the present invention and to enable those of ordinary skill in the art to understand the present invention and thereby design various embodiments with various modifications suitable for specific uses.

Claims

1. A method for extracting features from power grid service data, characterized in that: The following steps are involved: S1. Data collection and preprocessing: Collect and organize business data of all aspects of the power grid, including operation, management, and maintenance, in accordance with unified formats and standards, and preprocess the data; S2. Feature screening: Based on business experience and domain knowledge, preliminarily screen out features related to actual problems, then use statistical methods or machine learning algorithms to evaluate the correlation between features and target variables, weight the correlations calculated by different methods based on data analysis, obtain comprehensive correlations, and select the final features based on the comprehensive correlations; The specific weighting formula based on data analysis in step S2 is as follows: in, a i represents the weight obtained based on data analysis, i Indicates i A correlation calculation method, b i Indicates i The correlation calculation results are n is the total number of methods used to calculate correlation; a i The calculation process is as follows: ； in a 0 is the preset initial weight, f j is the confidence level, obtained from the result of feature verification, k is the number of iterations, and each round of verification is performed. k Will add 1, Δ a is the preset step length; S3. Feature extraction: For high-dimensional data, select appropriate methods to reduce the dimensionality, and use deep learning models to learn the features of power grid business data to extract deep features; S4. Feature verification and optimization: Verify the effectiveness of the extracted features through experiments or practical applications, and optimize the feature extraction method based on the verification results.

2. A method for extracting features of power grid service data according to claim 1, characterized in that: In step S1, the power grid business data includes the following collection methods: On-site monitoring: Install sensors on equipment to monitor power production, transmission and consumption in real time; Automation system: Establish an automatic collection and transmission system for power market data. By automatically collecting data and transmitting the data to the data center through the network, automatic data collection and processing can be achieved; Manual survey: Collect qualitative data through questionnaires, interviews, data declaration and annual reports; Database collection: directly integrate with the enterprise business backend server to write the business records generated by the enterprise business backend into the database; System log collection: collect log data for offline and online real-time analysis; Web data collection: Obtain data from websites through web crawlers and public APIs provided by website platforms to extract, clean, and convert unstructured and semi-structured data into structured data; Industry platforms and agency websites: Visit data platforms, government agency websites or corporate websites related to the power industry.

3. A method for extracting features of power grid service data according to claim 1, characterized in that: In step S1, the preprocessing specifically includes the following operations: Data deduplication: remove duplicate items from the data to avoid the impact of duplicate data on subsequent analysis; Missing value processing: For missing values, linear interpolation or polynomial interpolation is used to fill them, or they are supplemented according to relevant rules; Outlier detection and processing: Detect outliers in data through statistical analysis or data mining, and perform corresponding processing; Data denoising: remove noise from data through methods; Data standardization: converting data to the same dimension to facilitate feature comparison and model training; Data reduction: reduce the dimension and quantity of data through data compression and dimensionality reduction; Data validation: Validate the processed data to ensure that the data complies with predetermined rules and requirements.

4. A method for extracting features of power grid service data according to claim 1, characterized in that: In step S2, the preliminary screening specifically includes the following operations: Clarify business needs and problems: Clarify the actual needs of power grid business and the specific problems that need to be solved; Collect domain knowledge: Collect expertise in the power grid field and understand the industry standards, specifications and best practices of power grid business; Understand data sources: Study the source, structure and attributes of power grid business data, and identify key fields and potential features in the data; Preliminary screening based on experience: Using the long-term experience of analysts or experts, preliminary screening of potential features is carried out, focusing on features that are closely related to practical problems and have a significant impact on grid operation; Feature importance assessment: Preliminary assessment of the importance of the selected features for solving practical problems, and use evaluation by business experts, simple statistical analysis or visualization tools to help understand the relationship between features and target variables; Eliminate irrelevant or redundant features: Based on business experience and domain knowledge, eliminate features that are irrelevant or redundant to the actual problem.

5. A method for extracting features of power grid service data according to claim 1, characterized in that: In step S2, the statistical method specifically includes: Chi-square test: It determines whether there is a significant association between two categorical variables by comparing the difference between the observed frequency and the expected frequency, and does not rely on the distribution assumption of the data; Pearson correlation coefficient: The Pearson correlation coefficient is calculated by the covariance and standard deviation of two variables to measure the degree of linear correlation between two variables; Spearman correlation coefficient: a non-parametric correlation analysis method used to measure the degree of monotonic correlation between two variables. It does not require the variables to be linearly related and has no strict requirements on the distribution of data. Coefficient of determination: A statistical indicator that measures the degree to which an independent variable explains a dependent variable. It is applicable to regression models and indicates the proportion of the variation in the dependent variable that can be explained by the independent variable. Hypothesis testing: used to test whether there is a statistically significant correlation between two variables; Regression analysis: A statistical method used to establish a mathematical model between variables, measure the degree of linear correlation between two variables, and predict the value of the dependent variable.

6. A method for extracting features of power grid service data according to claim 1, characterized in that: In step S3, before feature extraction, weighting is performed, and after weighting, dimension reduction is performed, wherein the dimension reduction method includes: Principal component analysis: It converts data into a low-dimensional space through orthogonal transformation while retaining key information in the data as much as possible. It is suitable for processing numerical data. t-distributed random neighbor embedding: suitable for embedding high-dimensional data into two-dimensional or three-dimensional space for visualization; Linear discriminant analysis: find the feature subspace that maximizes the difference between classes and minimizes the difference within classes; Local linear embedding: It is used to process nonlinear high-dimensional data by finding the local linear structure of high-dimensional data. That is, each point representing high-dimensional data in low-dimensional space can be reconstructed through linear combinations on the neighborhood. It is suitable for data sets with nonlinear structures. Multidimensional scaling analysis: It converts the distance between each point in a high-dimensional data set into the Euclidean distance in a low-dimensional space for dimensionality reduction, aiming to preserve the distance relationship between different data points; Singular value decomposition: It decomposes the data to find the key features or principal components in the data. It is an extension of principal component analysis.

7. A method for extracting features of power grid service data according to claim 1, characterized in that: In step S3, according to the characteristics and requirements of different power grid business data, a suitable deep learning model is selected, and the power grid business data is processed using data processing methods including convolutional neural networks, recurrent neural networks, long short-term memory networks, and gated recurrent units. Before performing deep learning tasks, computing resources are fully evaluated and prepared to ensure smooth training and testing of the model.

8. A method for extracting features of power grid service data according to claim 1, characterized in that: In the step S4, the specific operation of feature verification is as follows: prepare an original data set of power grid business data, and divide it into a training set, a validation set and a test set in a ratio of 60%:20%:20% or 70%:15%:15%, then extract features from the original data through steps S1 to S3, and build a prediction or classification model based on the extracted features, then train the model on the training set, and monitor the performance of the model on the validation set in real time, and then use feature verification methods including performance evaluation indicators, cross-validation, feature importance evaluation, and baseline model comparison for verification, and at the same time deploy the model to an actual power grid business scenario, use real-time data for verification, and monitor the performance of the model in actual applications to verify the validity of the extracted features.

9. A method for extracting features of power grid service data according to claim 8, characterized in that: The feature verification method specifically includes: 1) Performance evaluation indicators Accuracy: the ratio of the number of correctly classified or predicted samples to the total number of samples; Recall rate: the ratio of the number of instances correctly predicted as positive samples to the number of all positive sample instances, applicable to classification tasks; F1 score: The harmonic mean of precision and recall, used to balance the two; Mean square error or mean absolute error: measures the error between the predicted value and the actual value, suitable for regression tasks; 2) Cross-validation Use K-fold cross validation, divide the data set into K parts each time, use K-1 parts for training, and the remaining part for testing; Repeat K times, making sure that each piece of data is used as a test set, and record the performance indicators each time; Calculate the mean and standard deviation of K performance indicators to evaluate the stability and generalization ability of the model; 3) Feature Importance Evaluation For tree-based models, use feature importance scores to measure the contribution of each feature to model performance; analyze the correlation between features to ensure there are no redundant or highly correlated features; 4) Baseline model comparison Establish a baseline model that does not use the extracted features; compare the performance of the baseline model with the model using the extracted features under the same dataset and evaluation criteria; if the model using feature extraction significantly outperforms the baseline model in terms of performance indicators, the extracted features are considered to be effective.

Images