Optimization method and system for power data value extraction efficiency

Abstract

The invention discloses an optimization method and system for electric power data value extraction efficiency, and relates to the technical field of electric power data process.The method comprises the steps that a multi-source heterogeneous data set is constructed from multiple links in an electric power system, and cleaning, standardization and binning processing are conducted on the multi-source heterogeneous data set; according to the method, the data set is more standardized and consistent before feature extraction, noise, abnormal values and data redundancy in the data are reduced through the preprocessing operation, the accuracy of subsequent data analysis is improved, and it is ensured that the system can process and analyze huge power data more efficiently. By introducing a similarity coefficient Xsxs and a data distribution coefficient Xbxs, the system can carry out similarity and distribution analysis according to the actual condition of each parameter, so that a priority estimation coefficient Ygxs is dynamically generated, and the system can estimate the priority of each parameter according to the dependency relationship between real-time data and historical data and the dispersion state of each parameter in time. And the priority of data extraction is reasonably determined.

Description

Technical Field

[0001] The present invention relates to the technical field of power data processing, and in particular to a method and system for optimizing the efficiency of power data value extraction. Background Art

[0002] With the digital development of modern power systems, multi-source heterogeneous data from power generation, transmission, distribution and user ends have increased dramatically, and the value mining and extraction efficiency of power data have become one of the core links in power system optimization and management. In the specific field of power data value extraction efficiency, by analyzing the data collected from different time periods in the power system, the key features of the system can be identified and the operating efficiency and reliability of the system can be improved. Especially in the analysis of data distribution, the distribution of power data in different time periods is often uneven. The amount of data in some time periods is too much, resulting in excessive computing load, while the data in other time periods is sparse, causing local bottlenecks in the extraction process. Therefore, how to efficiently extract the value of data in this context and ensure balanced processing of data in each time period is the key to optimizing the efficiency of power data value extraction.

[0003] In the process of analyzing the efficiency of value extraction from power data, uneven data distribution is a common phenomenon. If the amount of data is too large in certain time periods and less in other time periods, local bottlenecks may occur in the data extraction process, resulting in unreasonable resource allocation and inefficiency in the system when processing data. Existing systems often lack effective countermeasures to this uneven distribution, resulting in fluctuations in extraction efficiency. In addition, existing methods usually rely on static or fixed rules when estimating the priority of each parameter, and fail to dynamically consider the similarity and distribution between data. Therefore, unreasonable arrangement of the extraction order may result in key parameters not being processed in a timely manner, affecting the efficiency and accuracy of the overall system. Summary of the invention

[0004] In view of the deficiencies in the prior art, the present invention provides a method and system for optimizing the efficiency of power data value extraction, which solves the problems in the above-mentioned background technology.

[0005] To achieve the above objectives, the present invention is implemented through the following technical solutions: A method for optimizing the efficiency of power data value extraction, comprising the following steps:

[0006] S1. The power status data monitored by several groups of monitoring devices at different time periods in various links of the power system are collected in advance to construct a multi-source heterogeneous data set;

[0007] S2, preprocessing the multi-source heterogeneous data set obtained in S1 before feature extraction, performing data cleaning, standardization and binning on the multi-source heterogeneous data set to generate a power data set;

[0008] S3. Based on the power data set and in combination with the power status data monitored in real time by several groups of monitoring devices, the dependency between the real-time parameters and the corresponding parameters in the power data set is analyzed to construct a similarity coefficient Xsxs, and in the time series, the dispersion state of the corresponding parameters in different monitoring periods is analyzed to calculate the data distribution coefficient Xbxs of the corresponding parameters, and the priority estimation coefficient Ygxs is obtained by associating the similarity coefficient Xsxs with the data distribution coefficient Xbxs of the corresponding parameters, and the parameter extraction order is determined based on the priority estimation coefficient Ygxs;

[0009] S4. Based on the extraction order of each parameter in S3, analyze the timeliness of the data extraction process to obtain the data freshness Sqxd, and combine the trained efficiency prediction model and linear normalization processing to fit the output extraction efficiency index Txzs;

[0010] S5. Preset the safety threshold Q, and compare and analyze the extraction efficiency index Txzs with the safety threshold Q to comprehensively judge the extraction efficiency of the parameters in the corresponding monitoring period during feature extraction, and take corresponding optimization measures based on the judgment results.

[0011] Preferably, the specific steps of S1 include:

[0012] S11, and using several groups of monitoring devices to monitor the power status data in different time periods to construct a multi-source heterogeneous data set, wherein the multi-source heterogeneous data set includes the power status data monitored by different monitoring devices in different time periods.

[0013] Preferably, the specific steps of S2 include:

[0014] S21, preprocessing the multi-source heterogeneous data set obtained in S11, wherein the preprocessing includes data cleaning, standardization and binning;

[0015] S211, wherein the parameters of the multi-source heterogeneous data set are divided into equal-frequency bins to obtain several groups of power data in monitoring periods;

[0016] S22, summarizing the power data of several groups of monitoring periods in time series to re-obtain the power data set.

[0017] Preferably, the specific steps of S3 include:

[0018] S31. According to the power data set and in combination with the power status data output in real time in the power system, the dependency between the real-time parameters and the corresponding parameters in the power data set is analyzed, and each parameter in the power status data is mapped to the three-dimensional space through nonlinear mapping to obtain the similarity coefficient Xsxs of the corresponding parameter, which is obtained specifically in the following manner:

[0019]

[0020] Where n represents the monitoring period in the power data set; i = 1, 2, 3, ..., n; a i It is expressed as the contribution of the parameter of the ith monitoring period in the power data set to the parameter output in real time in the current power system; hi Represented as the parameter of the ith monitoring period in the power data set; x n It is represented by the real-time output parameter of the current power system; K(x hi , x n ) is represented as kernel function;

[0021] S311, the kernel function K(x hi , x n ) is obtained by the following formula:

[0022]

[0023] In the formula, σ represents the parameter in the Gaussian kernel function; exp(x) represents the exponential growth function, specifically e x , where e is Euler's constant; ‖x hi -x n ‖ is represented by the Euclidean distance between the parameter of the i-th monitoring period in the power data set and the parameter output in real time in the current power system.

[0024] Preferably, the specific step S3 also includes:

[0025] S32. In the time series, the dispersion state of the corresponding parameters in different monitoring periods is analyzed to calculate the data distribution coefficient Xbxs of the parameters in the corresponding monitoring period, which is obtained specifically in the following manner:

[0026]

[0027] Where m represents the monitoring period of power data, j = 1, 2, 3, ..., m, x j Expressed as the parameter of the jth monitoring period, x avg Expressed as the average parameter during the monitoring period.

[0028] Preferably, the specific step S3 also includes:

[0029] S33, based on the similarity coefficient Xsxs obtained in S31 and S32 and the data distribution coefficient Xbxs of the corresponding parameter, after linear normalization processing, the corresponding data value is mapped to the interval [0,1] to calculate the priority estimation coefficient Ygxs, which is obtained specifically in the following manner:

[0030] Ygxs=α*Xsxs+β*Xbxs+C;

[0031] In the formula, α and β are weight values, C represents the first correction constant, and the specific values ​​of α and β are set by the user according to the situation;

[0032] S34. Based on the method of obtaining the priority estimation coefficient Ygxs in S33, respectively obtain the priority estimation coefficients Ygxs of several groups of parameters in the power status data output in real time in the power system, and sort the priority estimation coefficients Ygxs of the several groups of parameters by numerical values ​​to construct a priority data column, and according to the priority data column, perform feature extraction on several groups of parameters in the power status data output in real time in the power system according to priority.

[0033] Preferably, the specific steps of S4 include:

[0034] S41, in the process of extracting features according to the priority in S34, real-time monitoring of the freshness of the data extraction process to obtain relevant extraction status data information, the relevant extraction status data information including the preprocessing time T e , data generation time T g and data extraction time T c ;

[0035] S42. Calculate and obtain data freshness Sqxd according to relevant extraction status data information. The data freshness Sqxd is obtained by the following formula:

[0036]

[0037] Where, T e is the preprocessing time, which means the time for preprocessing multi-source heterogeneous data; T g T is the data generation time, which means the time when the data is generated by the sensor or other equipment; c is the data extraction time, which indicates the moment when the system extracts data from the power dataset.

[0038] Preferably, the specific step S4 also includes:

[0039] S42. Construct an efficiency prediction model using convolutional neural network technology, input the data freshness Sqxd and the priority estimation coefficient Ygxs into the efficiency prediction model, and after linear normalization, fit the output extraction efficiency index Txzs from the efficiency prediction model. The extraction efficiency index Txzs is obtained by the following formula:

[0040]

[0041] Where K represents the number of parameters, k = 1, 2, 3, ..., K, Sqxd k Represented as the data freshness of the kth parameter, Sqxd avg Expressed as average data freshness, Ygxs k Expressed as the priority estimation coefficient of the kth parameter, Ygxs avg It is expressed as the average priority estimation coefficient, F1 and F2 are both weight values, P is expressed as the second correction constant, and the specific values ​​of F1 and F2 are set by the user according to the situation.

[0042] Preferably, the specific steps of S5 include:

[0043] S51, by comparing and analyzing the preset safety threshold Q with the extraction efficiency index Txzs, a comprehensive judgment is made on the extraction efficiency state of the parameters in the corresponding monitoring period when the features are extracted. The specific judgment contents are as follows:

[0044] If the extraction efficiency index Txzs falls within the safety threshold Q, it is judged that the extraction efficiency of the parameters in the corresponding monitoring period is in a normal state when the features are extracted. At this time, the feature extraction operation of the current power status data output in real time in the power system will continue;

[0045] S52. If the extraction efficiency index Txzs does not fall within the safety threshold Q, it is determined that the extraction efficiency of the parameters in the corresponding monitoring period is not in a normal state during feature extraction. At this time, the cache technology will be used to extract and process the data in advance, and the priority of each parameter will be re-evaluated according to the actual application scenario.

[0046] A system for optimizing the efficiency of power data value extraction, comprising a data acquisition module, a preprocessing module, an extraction sequence analysis module, a comprehensive evaluation module and a feedback module;

[0047] The data acquisition module is used to collect power status data monitored by several groups of monitoring devices at different time periods from various links in the power system in advance to construct a multi-source heterogeneous data set;

[0048] The preprocessing module is used to perform preprocessing operations on the multi-source heterogeneous data sets before feature extraction, and perform data cleaning, standardization and binning on the multi-source heterogeneous data sets to generate a power data set;

[0049] The extraction sequence analysis module is used to analyze the dependency between the real-time parameters and the corresponding parameters in the power data set based on the power data set and in combination with the power status data monitored in real time by several groups of monitoring devices, so as to construct a similarity coefficient Xsxs, and analyze the dispersion state of the corresponding parameters in different monitoring periods in the time series to calculate the data distribution coefficient Xbxs of the corresponding parameters, and obtain the priority estimation coefficient Ygxs by associating the similarity coefficient Xsxs with the data distribution coefficient Xbxs of the corresponding parameters, and determine the parameter extraction order based on the priority estimation coefficient Ygxs;

[0050] The comprehensive evaluation module is used to analyze the timeliness of the data extraction process based on the extraction order of each parameter in S3 to obtain the data freshness Sqxd, and combine the trained efficiency prediction model and linear normalization processing to fit the output extraction efficiency index Txzs;

[0051] The feedback module is used to pre-set the safety threshold Q, and compare and analyze the extraction efficiency index Txzs with the safety threshold Q to comprehensively judge the extraction efficiency of the parameters in the corresponding monitoring period during feature extraction, and take corresponding optimization measures based on the judgment results.

[0052] The present invention provides a method and system for optimizing the efficiency of power data value extraction, which has the following beneficial effects:

[0053] (1) This method constructs a multi-source heterogeneous data set from multiple links in the power system, and cleans, standardizes and bins it, making the data set more standardized and consistent before feature extraction. This preprocessing operation reduces noise, outliers and data redundancy in the data, improves the accuracy of subsequent data analysis, and ensures that the system can process and analyze huge power data more efficiently. By introducing the similarity coefficient Xsxs and the data distribution coefficient Xbxs, the system can perform similarity and distribution analysis based on the actual situation of each parameter, thereby dynamically generating the priority estimation coefficient Ygxs, which enables the system to reasonably determine the priority of data extraction based on the dependency relationship between real-time data and historical data, as well as the time dispersion state of each parameter, further avoiding the bottleneck problem caused by uneven data distribution, ensuring that key data can be processed first, and thus improving the overall extraction efficiency. In the feature extraction process, this method introduces the data freshness Sqxd evaluation mechanism to ensure that the system can judge the timeliness of extraction based on the real-time nature of the data. Through real-time calculation of freshness, the system can identify data delay problems and adjust the data extraction strategy according to the delay situation. This mechanism enhances the system's sensitivity when processing real-time data, which helps the real-time monitoring and rapid response of the power system. The fitting output of the extraction efficiency index Txzs, combined with the setting of the safety threshold Q, can help the system quickly determine whether the extraction efficiency of each parameter in the current monitoring period is in a normal state, so as to ensure that the overall extraction efficiency of the system remains at a high level.

[0054] (2) This method analyzes the dependency between the parameters in the power data set and the real-time output parameters, and maps these parameters to three-dimensional space using kernel functions. This nonlinear mapping can capture complex feature patterns in the original space that cannot be extracted by linear methods. In particular, in power data, there are often nonlinear relationships between parameters such as voltage and load fluctuations. By mapping to a higher-dimensional space, these implicit feature patterns can be better extracted, which helps to discover potential correlations between different parameters. The similarity coefficient Xsxs is obtained based on the nonlinear mapping of these complex relationships, which enables the system to accurately measure the similarity between the parameters in the power data set and the current real-time parameters, thereby more accurately monitoring and analyzing the power system.

[0055] (3) By analyzing the dispersion state of the corresponding parameters in different monitoring periods, the system can identify the degree of data fluctuation in each period. By calculating the data distribution coefficient Xbxs, the dispersion degree of different parameters in the monitoring period can be quantified, which helps to distinguish which parameters have large fluctuations and which parameters are relatively stable in different time periods. This in-depth analysis of the data dispersion state can help the system prioritize the data segments with large fluctuations, ensuring that the focus is on those parameters that are critical to power system monitoring and dispatching during the feature extraction process, thereby improving the accuracy and effectiveness of the overall feature extraction. Based on the similarity coefficient Xsxs and the data distribution coefficient Xbxs, the corresponding data values ​​are mapped to a unified interval through linear normalization processing, ensuring that the parameters of different monitoring periods can be compared under the same standard. Through this comprehensive processing, the system can more accurately calculate the priority estimation coefficient Ygxs, effectively improving the rationality of the priority sorting of each parameter. This process enables the system to dynamically adjust the priority of different parameters to ensure that high-priority key parameters (such as load fluctuations, voltage anomalies, etc.) can be processed first, thereby avoiding inefficient system monitoring or missing important information due to improper processing order.

[0056] (4) During the priority feature extraction process, the system can monitor the status information of data extraction in real time, including preprocessing time, data generation time, and data extraction time, so as to ensure that the system has comprehensive control over the data extraction and processing process at each stage of feature extraction. This function enables the system to effectively avoid delays, data obsolescence, and other problems, thereby improving the timeliness of data processing and ensuring that the latest status data can be obtained in the power system in a timely manner to help make fast and effective responses and decisions. By comprehensively considering the preprocessing time, data generation time, and data extraction time, the system can accurately calculate the data freshness Sqxd to measure the overall delay from data generation to preprocessing and then to extraction. The dynamic evaluation of data freshness helps to ensure that data can be quickly extracted and processed after generation, further avoiding data aging or obsolescence. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] Figure 1 A schematic flow chart of a method for optimizing the efficiency of value extraction of power data according to the present invention;

[0058] Figure 2 The present invention is a system block diagram for optimizing the efficiency of value extraction of power data. DETAILED DESCRIPTION

[0059] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0060] Example 1

[0061] See also Figure 1 The present invention provides a method for optimizing the efficiency of power data value extraction, comprising the following steps:

[0062] S1. The power status data monitored by several groups of monitoring devices at different time periods in various links of the power system are collected in advance to construct a multi-source heterogeneous data set;

[0063] S2, preprocessing the multi-source heterogeneous data set obtained in S1 before feature extraction, performing data cleaning, standardization and binning on the multi-source heterogeneous data set, that is, dividing the continuous variables into discrete intervals to generate a power data set;

[0064] S3. Based on the power data set and in combination with the power status data monitored in real time by several groups of monitoring devices, the dependency between the real-time parameters and the corresponding parameters in the power data set is analyzed to construct a similarity coefficient Xsxs, and in the time series, the dispersion state of the corresponding parameters in different monitoring periods is analyzed to calculate the data distribution coefficient Xbxs of the corresponding parameters, and the priority estimation coefficient Ygxs is obtained by associating the similarity coefficient Xsxs with the data distribution coefficient Xbxs of the corresponding parameters, and the parameter extraction order is determined based on the priority estimation coefficient Ygxs;

[0065] S4. Based on the extraction order of each parameter in S3, analyze the timeliness of the data extraction process to obtain the data freshness Sqxd, and combine the trained efficiency prediction model and linear normalization processing to fit the output extraction efficiency index Txzs;

[0066] S5. Preset the safety threshold Q, and compare and analyze the extraction efficiency index Txzs with the safety threshold Q to comprehensively judge the extraction efficiency of the parameters in the corresponding monitoring period during feature extraction, and take corresponding optimization measures based on the judgment results.

[0067] In this embodiment, the method constructs a multi-source heterogeneous data set by acquiring power status data of different periods from different links and multiple monitoring devices in the power system. Through pre-processing operations such as data cleaning, standardization processing and binning processing, the consistency and availability of the data are further ensured, the noise and redundant information in the data are reduced, and the subsequent feature extraction is more efficient and accurate. This processing method solves the problem of complex sources and different structures of power data, effectively improves the processing efficiency and quality of data, and provides a reliable data basis for feature extraction. The method constructs a similarity coefficient Xsxs by analyzing the dependency between real-time monitoring data and historical power data, analyzes the dispersion state of parameters in the time series, calculates the data distribution coefficient Xbxs, associates the similarity coefficient with the data distribution coefficient, generates a priority estimation coefficient Ygxs, and dynamically determines the extraction order of each parameter according to the priority. This method not only considers the similarity between data, but also analyzes the data distribution characteristics of different periods, ensuring that key parameters can be extracted first in the period with large data volume, thereby avoiding the local bottleneck problem caused by uneven data distribution and improving the efficiency and accuracy of data extraction. By analyzing the timeliness of the data extraction process, the data freshness Sqxd is obtained and input into the trained efficiency prediction model. Through linear normalization processing, the extraction efficiency index Txzs is fitted and output. This step enables the system to evaluate the efficiency of the data extraction process in real time, ensuring that key data can be extracted and processed in time, and further optimizing the response speed and processing capacity of the entire system. By setting the safety threshold Q, the present invention can monitor the extraction efficiency in real time during the data extraction process. When the extraction efficiency index Txzs falls into the safety threshold, the system can automatically identify inefficient data extraction and take corresponding optimization measures (such as adjusting the extraction order or optimizing parameters) to ensure that the system always maintains an efficient state. This dynamic tuning mechanism not only improves the stability of the system, but also significantly enhances the processing efficiency of power data. In summary, this method achieves a comprehensive improvement in the efficiency of power data value extraction through the construction and preprocessing of multi-source heterogeneous data sets, the combination of similarity coefficients and data distribution coefficients, real-time analysis of data freshness, and comparison and optimization of extraction efficiency indexes. This method can effectively avoid the bottleneck problem caused by uneven data distribution, ensure that the system can still operate efficiently in a complex power data environment, improve the data processing capability and response speed of the power system, and has important practical value.

[0068] Example 2

[0069] Please refer to Figure 1 , specifically: S1 specific steps include:

[0070] S11, and using several groups of monitoring devices to monitor the power status data in different time periods to construct a multi-source heterogeneous data set, wherein the multi-source heterogeneous data set includes the power status data monitored by different monitoring devices in different time periods.

[0071] The specific steps of S2 include:

[0072] S21. Preprocess the multi-source heterogeneous data set obtained in S11, and the preprocessing operation includes data cleaning, standardization and binning. Data cleaning is the first step of preprocessing, and its purpose is to remove or repair anomalies, errors and redundant information in the data, and fill in missing values ​​through linear interpolation, Lagrange interpolation or spline interpolation, and identify and eliminate outliers (such as data points outside 3 times the standard deviation) through the mean and standard deviation.

[0073] S211, wherein, by dividing the internal parameters of the multi-source heterogeneous data set into equal-frequency bins, the number of data in each interval is made roughly the same, so as to obtain power data of several groups of monitoring periods;

[0074] S22, summarizing the power data of several groups of monitoring periods in time series to re-obtain the power data set.

[0075] In this embodiment, by deploying multiple monitoring devices, the system can obtain multi-source heterogeneous data from different links of the power system in real time. These data contain power status information collected by different time periods and different devices, ensuring the diversity and comprehensiveness of data sources. Combined with the data cleaning step, the system can effectively remove or repair abnormal, erroneous and redundant data, and fill in missing values ​​through linear interpolation, Lagrange interpolation or spline interpolation to ensure the integrity of the data. The data cleaning process also uses the 3 times standard deviation method to remove abnormal values, further improving the accuracy of the data, thereby laying a solid foundation for subsequent feature extraction. When preprocessing multi-source heterogeneous data sets, the system uses equal frequency binning technology to make the number of data in each bin roughly the same. This processing method effectively solves the problem of uneven data distribution, ensures that the data of each monitoring period can be balanced in subsequent analysis, and reduces the analysis deviation caused by differences in data volume. Through equal frequency binning, the system can better capture the data change trend of different time periods and improve the representativeness and availability of data. The last step of preprocessing is to summarize the data from multiple monitoring periods in time series and regenerate the power data set. This process reduces the time series inconsistency of the data by summarizing the data in different time periods and lays the foundation for subsequent time series analysis. Time series aggregation ensures that the system can maintain the continuity and consistency of data when processing complex time series data, further avoids the impact of data jumps or missing data on the analysis results, and improves the system's ability to capture and analyze data in the time dimension. This method further improves the integrity, accuracy, balance and consistency of power data through various preprocessing operations, including data cleaning, outlier removal, equal frequency binning and time series aggregation. Through these optimization steps, the system can process multi-source heterogeneous data more accurately, ensure the quality of feature extraction and subsequent analysis, and ultimately improve the overall efficiency and accuracy of power data value extraction.

[0076] Example 3

[0077] Please refer to Figure 1 , specifically: S3 specific steps include:

[0078] S31. According to the power data set and in combination with the power status data output in real time in the power system, the dependency between the real-time parameters and the corresponding parameters in the power data set is analyzed, and each parameter in the power status data is mapped to the three-dimensional space through nonlinear mapping, and the inseparable characteristic patterns in the original space are extracted therefrom to obtain the similarity coefficient Xsxs of the corresponding parameter, which is obtained specifically in the following manner:

[0079]

[0080] Where n represents the monitoring period in the power data set; i = 1, 2, 3, ..., n; ai It is expressed as the contribution of the parameter of the ith monitoring period in the power data set to the parameter output in real time in the current power system; hi Represented as the parameter of the ith monitoring period in the power data set; x n It is represented by the real-time output parameter of the current power system; K(x hi , x n ) is expressed as a kernel function, which maps the parameters of the ith monitoring period in the power data set and the parameters of the real-time output in the current power system to the three-dimensional space to capture the nonlinear relationship between the parameters of the ith monitoring period in the power data set and the parameters of the real-time output in the current power system;

[0081] The similarity coefficient Xsxs of the corresponding parameters reflects the characteristic representation of the current data point x in the high-dimensional space.

[0082] S311, the kernel function K(x hi , x n ) is obtained by the following formula:

[0083]

[0084] Where σ is a parameter in the Gaussian kernel function, which controls the "smoothness" or "width" of the Gaussian kernel function, and determines the decay rate of the similarity between data points; exp(x) is an exponential growth function, specifically e x , where e is the Euler constant, approximately equal to 2.718; ‖x hi -x n ‖ is represented by the Euclidean distance between the parameter of the i-th monitoring period in the power data set and the parameter output in real time in the current power system.

[0085] In this embodiment, each parameter in the power data is mapped to a three-dimensional high-dimensional space by using a kernel function. This method effectively captures the complex nonlinear feature patterns in the original space that cannot be separated by linear methods. This nonlinear mapping can reflect the hidden deep relationship between different parameters in the power system, especially the dependence of complex parameters such as power load and voltage fluctuation. Through these feature extraction methods, the system can obtain the similarity coefficient Xsxs between the power state data and the real-time parameters in each monitoring cycle, thereby achieving more accurate state monitoring and prediction, and improving the effectiveness and accuracy of data extraction. By analyzing the contribution of each parameter in the power data set to the real-time output parameter, and combining the calculation of the similarity coefficient Xsxs, the system can accurately determine the relationship between each parameter in different monitoring periods based on the dependence of each parameter on the current real-time power state. This dependency analysis can play a key role in data extraction in different time periods, help determine the priority extraction order, and improve the overall response speed of the system. The use of kernel functions further improves the system's ability to capture nonlinear relationships in power data sets, allowing complex feature relationships to be well characterized in high-dimensional space. In particular, the precise nonlinear mapping achieved through the Gaussian kernel function can effectively evaluate the similarity between data points, ensuring that the system can better cope with nonlinear relationships when processing multi-source data. The parameter σ in the kernel function controls the "smoothness" or "width" of the Gaussian kernel function, allowing the system to flexibly adjust the decay rate of similarity. This parameterized design provides the system with greater flexibility in dealing with different power state data, allowing similarity analysis to be adjusted according to specific application scenarios, thereby enhancing the adaptability of data analysis. Whether facing power load data with large fluctuations or stable voltage data, more sophisticated similarity analysis can be performed by adjusting the smoothness of the kernel function.

[0086] Example 4

[0087] Please refer to Figure 1 , specifically: S3 specific steps also include:

[0088] S32. In the time series, the dispersion state of the corresponding parameters in different monitoring periods is analyzed to calculate the data distribution coefficient Xbxs of the parameters in the corresponding monitoring period, which is obtained specifically in the following manner:

[0089]

[0090] Where m represents the monitoring period of power data, j = 1, 2, 3, ..., m, x j Expressed as the parameter of the jth monitoring period, x avg Expressed as the average parameter during the monitoring period.

[0091] The specific steps of S3 also include:

[0092] S33, based on the similarity coefficient Xsxs obtained in S31 and S32 and the data distribution coefficient Xbxs of the corresponding parameter, after linear normalization processing, the corresponding data value is mapped to the interval [0,1] to calculate the priority estimation coefficient Ygxs, which is obtained specifically in the following manner:

[0093] Ygxs=α*Xsxs+β*Xbxs+C;

[0094] In the formula, α and β are weight values, C represents the first correction constant, and the specific values ​​of α and β are set by the user according to the situation;

[0095] S34. Based on the method of obtaining the priority estimation coefficient Ygxs in S33, respectively obtain the priority estimation coefficients Ygxs of several groups of parameters in the power status data output in real time in the power system, and sort the priority estimation coefficients Ygxs of the several groups of parameters by numerical values ​​to construct a priority data column, and according to the priority data column, perform feature extraction on several groups of parameters in the power status data output in real time in the power system according to priority.

[0096] In this embodiment, by analyzing the dispersion state of power data parameters in different monitoring periods in the time series and calculating the corresponding data distribution coefficient Xbxs, the volatility and distribution uniformity of data in each time period can be effectively measured. This step enables the system to identify which time periods have large data fluctuations and which time periods have more concentrated or sparse data, so as to better adjust the subsequent feature extraction strategy. This analysis process helps to avoid the extraction bottleneck caused by uneven data distribution and improves the stability and accuracy of the system when facing a large amount of data. This method combines the similarity coefficient Xsxs obtained in S31 and the data distribution coefficient Xbxs obtained in S32, and after linear normalization, maps the two to a unified interval, and calculates the priority estimation coefficient Ygxs of each parameter. This processing method not only considers the similarity of each parameter with historical data, but also combines the data distribution in different time periods to ensure that the priority estimation is more accurate. This method can effectively optimize the feature extraction order, ensure that key data is given priority, reduce data lag and resource waste, and improve extraction efficiency. Based on the calculation of the priority estimation coefficient Ygxs, the system prioritizes the various parameters output in real time in the power system and constructs a priority data column so that the key or higher priority parameters can be first extracted. This dynamic sorting mechanism can ensure that the system can reasonably allocate computing resources and processing time according to the importance and current status of the data when processing data in real time, thereby improving the response speed and real-time processing capability of the entire system. Through the construction of priority data columns, the system can avoid processing bottlenecks caused by excessive data volume or uneven distribution, ensuring that the system can accurately and quickly obtain key features at high load or critical moments, thereby optimizing the operation and monitoring of the power system. Through the dynamic adjustment mechanism of the priority estimation coefficient, the system can automatically optimize the order of feature extraction according to the real-time changes of power status data. The system no longer relies on static or preset rules, but can adaptively adjust the feature extraction strategy in real time according to actual conditions. This flexibility enables the system to cope with different operating conditions and improves the accuracy of power system monitoring and analysis. In summary, this method can accurately evaluate the distribution and similarity of power data, and on this basis, prioritize different time periods and parameters. Combined with the similarity coefficient and data distribution coefficient, after linear normalization, the system can dynamically adjust the order of data extraction to ensure that important parameters are given priority. This mechanism not only improves the efficiency and accuracy of data extraction, but also enhances the system's processing capabilities and flexibility during high-load periods, effectively improving the power system's operational optimization capabilities and ensuring that the efficiency of data value extraction always remains at a high level.

[0097] Example 5

[0098] Please refer to Figure 1 , specifically: S4 specific steps include:

[0099] S41, in the process of extracting features according to the priority in S34, real-time monitoring of the freshness of the data extraction process to obtain relevant extraction status data information, the relevant extraction status data information including the preprocessing time T e , data generation time T g and data extraction time T c ;

[0100] S42. Calculate and obtain data freshness Sqxd according to relevant extraction status data information. The data freshness Sqxd is obtained by the following formula:

[0101]

[0102] Where, T e is the preprocessing time, which means the time for preprocessing multi-source heterogeneous data; T g T is the data generation time, which means the time when the data is generated by the sensor or other equipment; c is the data extraction time, which indicates the moment when the system extracts data from the power data set; e -T g It is used to measure the total time difference from data generation to preprocessing time, indicating the delay from data generation to preprocessing time; T c -T g It is used to measure the time difference from data generation to data extraction. The smaller the time difference, the more timely the data extraction is and the closer the data is to the latest status.

[0103] Data freshness measures the delay from data generation to preprocessing and then to extraction. Through this calculation, we can evaluate whether the data is preprocessed quickly after generation, and whether the data is extracted quickly after preprocessing, further avoiding excessive lags.

[0104] The above preprocessing time T e , data generation time T g , data extraction time T c It can be monitored through smart meters, data acquisition systems or edge computing devices.

[0105] In this embodiment, the method can dynamically evaluate the timeliness of data extraction by real-time monitoring of relevant status information of data extraction during the feature extraction process, including preprocessing time, data generation time and data extraction time. In this way, the system can accurately calculate the data freshness Sqxd to ensure that the data can be quickly extracted and processed after generation. For scenarios such as power systems that have high real-time requirements, monitoring of data freshness can effectively prevent the system from processing outdated data, thereby improving the accuracy of data analysis and the timeliness of decision-making, and timely discovering possible bottlenecks or delay links. If the time of a certain link is too long, the system can take optimization measures in time, such as adjusting the data extraction frequency or optimizing the preprocessing process, so as to reduce data extraction delay and improve the real-time performance of data processing. By evaluating the data freshness, the system can identify which data is delayed during the extraction process, and then dynamically adjust the data extraction strategy. Especially in the process of real-time monitoring of the power status, the system can prioritize the latest data according to the level of data freshness, reducing decision deviations or system misjudgments caused by data lags. By accurately assessing the freshness of data, the system can ensure faster processing and response of power data, further avoiding system scheduling delays caused by data lags. Especially when dealing with critical scenarios such as power load fluctuations and equipment failure warnings, the system can respond quickly based on the latest data information, significantly improving the stability and safety of the power system.

[0106] Example 6

[0107] Please refer to Figure 1 Specifically: S4 includes the following steps:

[0108] S42. Construct an efficiency prediction model using convolutional neural network technology, input the data freshness Sqxd and the priority estimation coefficient Ygxs into the efficiency prediction model, and after linear normalization, fit the output extraction efficiency index Txzs from the efficiency prediction model. The extraction efficiency index Txzs is obtained by the following formula:

[0109]

[0110] Where K represents the number of parameters, k = 1, 2, 3, ..., K, Sqxd k Represented as the data freshness of the kth parameter, Sqxd avg Expressed as average data freshness, Ygxs k Expressed as the priority estimation coefficient of the kth parameter, Ygxs avg It is expressed as the average priority estimation coefficient, F1 and F2 are both weight values, P is expressed as the second correction constant, and the specific values ​​of F1 and F2 are set by the user according to the situation.

[0111] The specific steps of S5 include:

[0112] S51, by comparing and analyzing the preset safety threshold Q with the extraction efficiency index Txzs, a comprehensive judgment is made on the extraction efficiency state of the parameters in the corresponding monitoring period when the features are extracted. The specific judgment contents are as follows:

[0113] If the extraction efficiency index Txzs falls within the safety threshold Q, it is judged that the extraction efficiency of the parameters in the corresponding monitoring period is in a normal state when the features are extracted. At this time, the feature extraction operation of the current power status data output in real time in the power system will continue;

[0114] S52. If the extraction efficiency index Txzs does not fall within the safety threshold Q, it is judged that the extraction efficiency of the parameters in the corresponding monitoring period is not in a normal state during feature extraction. At this time, the cache technology will be used to extract and process the data in advance to ensure that the system can still maintain a high degree of freshness during high-load periods, and re-evaluate the priority of each parameter according to the actual application scenario. Some key parameters (such as load volatility, voltage stability, etc.) may be more important than other parameters in certain periods. Therefore, the priority of each parameter can be dynamically adjusted to improve the data processing priority of important parameters and reduce irrelevant or redundant features. In the convolutional neural network, multi-scale convolution kernels are applied to obtain data features at different resolutions, which is especially suitable for complex nonlinear features in power data.

[0115] In this embodiment, the present invention uses convolutional neural network technology to construct an efficiency prediction model, takes data freshness Sqxd and priority estimation coefficient Ygxs as input variables, and predicts the extraction efficiency of different parameters after linear normalization. The convolutional neural network can perform in-depth analysis of the complex nonlinear relationship in the power data through multi-level feature extraction, so as to more accurately fit the extraction efficiency index Txzs. Through this method, the system can more effectively evaluate the extraction efficiency of each parameter and ensure the quality and speed of feature extraction. By comparing and analyzing the extraction efficiency index Txzs with the preset safety threshold, the extraction efficiency state of the parameters in each monitoring period can be judged in real time. When the extraction efficiency index falls within the safety threshold, the system can continue to perform data extraction operations normally; when the efficiency does not meet the standard, the system will automatically identify the abnormality and take corresponding optimization measures. This dynamic adjustment mechanism ensures that the system can timely identify inefficient states during the processing process, reduces potential bottleneck problems, and ensures the stability and efficiency of the power data extraction process. When the extraction efficiency is not up to standard, the system will use cache technology to extract and process data in advance, so as to ensure that the system can still maintain a high level of data freshness during high-load periods. This mechanism can effectively cope with data flow fluctuations and reduce data lag problems caused by excessive load. In addition, the system dynamically adjusts the priority of each parameter and prioritizes certain key parameters (such as load fluctuations, voltage stability, etc.), thereby improving the response speed to key data, ensuring that important data can be processed quickly and improving the real-time performance of the overall system.

[0116] Example 7

[0117] Please refer to Figure 2 ,Specifically: A system for optimizing the efficiency of value extraction of power data includes a data acquisition module, a preprocessing module, an extraction sequence analysis module, a comprehensive evaluation module and a feedback module;

[0118] The data acquisition module is used to collect power status data monitored by several groups of monitoring devices at different time periods from various links in the power system in advance to construct a multi-source heterogeneous data set;

[0119] The preprocessing module is used to perform preprocessing operations on the multi-source heterogeneous data sets before feature extraction, and perform data cleaning, standardization and binning on the multi-source heterogeneous data sets to generate a power data set;

[0120] The extraction sequence analysis module is used to analyze the dependency between the real-time parameters and the corresponding parameters in the power data set based on the power data set and in combination with the power status data monitored in real time by several groups of monitoring devices, so as to construct a similarity coefficient Xsxs, and analyze the dispersion state of the corresponding parameters in different monitoring periods in the time series to calculate the data distribution coefficient Xbxs of the corresponding parameters, and obtain the priority estimation coefficient Ygxs by associating the similarity coefficient Xsxs with the data distribution coefficient Xbxs of the corresponding parameters, and determine the parameter extraction order based on the priority estimation coefficient Ygxs;

[0121] The comprehensive evaluation module is used to analyze the timeliness of the data extraction process based on the extraction order of each parameter in S3 to obtain the data freshness Sqxd, and combine the trained efficiency prediction model and linear normalization processing to fit the output extraction efficiency index Txzs;

[0122] The feedback module is used to pre-set the safety threshold Q, and compare and analyze the extraction efficiency index Txzs with the safety threshold Q to comprehensively judge the extraction efficiency of the parameters in the corresponding monitoring period during feature extraction, and take corresponding optimization measures based on the judgment results.

[0123] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A method for optimizing the efficiency of power data value extraction, characterized in that: The following steps are included: S1. The power status data monitored by several groups of monitoring devices at different time periods in various links of the power system are collected in advance to construct a multi-source heterogeneous data set; S2, preprocessing the multi-source heterogeneous data set obtained in S1 before feature extraction, performing data cleaning, standardization and binning on the multi-source heterogeneous data set to generate a power data set; S3. Based on the power data set and in combination with the power status data monitored in real time by several groups of monitoring devices, the dependency between the real-time parameters and the corresponding parameters in the power data set is analyzed to construct a similarity coefficient Xsxs, and in the time series, the dispersion state of the corresponding parameters in different monitoring periods is analyzed to calculate the data distribution coefficient Xbxs of the corresponding parameters, and the priority estimation coefficient Ygxs is obtained by associating the similarity coefficient Xsxs with the data distribution coefficient Xbxs of the corresponding parameters, and the parameter extraction order is determined based on the priority estimation coefficient Ygxs; S4. Based on the extraction order of each parameter in S3, analyze the timeliness of the data extraction process to obtain the data freshness Sqxd, and combine the trained efficiency prediction model and linear normalization processing to fit the output extraction efficiency index Txzs; S5. Preset the safety threshold Q, and compare and analyze the extraction efficiency index Txzs with the safety threshold Q to comprehensively judge the extraction efficiency of the parameters in the corresponding monitoring period during feature extraction, and take corresponding optimization measures based on the judgment results.

2. The method for optimizing the efficiency of power data value extraction according to claim 1, characterized in that: The specific steps of S1 include: S11, and using several groups of monitoring devices to monitor the power status data in different time periods to construct a multi-source heterogeneous data set, wherein the multi-source heterogeneous data set includes the power status data monitored by different monitoring devices in different time periods.

3. The method for optimizing the efficiency of power data value extraction according to claim 2, characterized in that: The specific steps of S2 include: S21, preprocessing the multi-source heterogeneous data set obtained in S11, wherein the preprocessing includes data cleaning, standardization and binning; S211, wherein the parameters of the multi-source heterogeneous data set are divided into equal-frequency bins to obtain several groups of power data in monitoring periods; S22, summarizing the power data of several groups of monitoring periods in time series to re-obtain the power data set.

4. The method for optimizing the efficiency of power data value extraction according to claim 3, characterized in that: The specific steps of S3 include: S31. According to the power data set and in combination with the power status data output in real time in the power system, the dependency between the real-time parameters and the corresponding parameters in the power data set is analyzed, and each parameter in the power status data is mapped to the three-dimensional space through nonlinear mapping to obtain the similarity coefficient Xsxs of the corresponding parameter, which is obtained specifically in the following manner: Where n represents the monitoring period in the power data set; i = 1, 2, 3, ..., n; a i It is expressed as the contribution of the parameter of the ith monitoring period in the power data set to the parameter output in real time in the current power system; hi Represented as the parameter of the ith monitoring period in the power data set; x n It is represented by the real-time output parameter of the current power system; K(x hi , x n ) is represented as kernel function; S311, the kernel function K(x hi , x n ) is obtained by the following formula: In the formula, σ represents the parameter in the Gaussian kernel function; exp(x) represents the exponential growth function, specifically e x , where e is Euler's constant; ||x hi -x n || is represented by the Euclidean distance between the parameter of the ith monitoring period in the power data set and the parameter output in real time in the current power system.

5. The method for optimizing the efficiency of power data value extraction according to claim 4, characterized in that: The specific steps of S3 also include: S32. In the time series, the dispersion state of the corresponding parameters in different monitoring periods is analyzed to calculate the data distribution coefficient Xbxs of the parameters in the corresponding monitoring period, which is obtained specifically in the following manner: Where m represents the monitoring period of power data, j = 1, 2, 3, ..., m, x j Expressed as the parameter of the jth monitoring period, x avg Expressed as the average parameter during the monitoring period.

6. The method for optimizing the efficiency of power data value extraction according to claim 5, characterized in that: The specific steps of S3 also include: S33, based on the similarity coefficient Xsxs obtained in S31 and S32 and the data distribution coefficient Xbxs of the corresponding parameter, after linear normalization processing, the corresponding data value is mapped to the interval [0, 1] to calculate the priority estimation coefficient Ygxs, which is specifically obtained in the following manner: Ygxs=α*Xsxs+β*Xbxs+C; In the formula, α and β are weight values, C represents the first correction constant, and the specific values ​​of α and β are set by the user according to the situation; S34. Based on the method of obtaining the priority estimation coefficient Ygxs in S33, respectively obtain the priority estimation coefficients Ygxs of several groups of parameters in the power status data output in real time in the power system, and sort the priority estimation coefficients Ygxs of the several groups of parameters by numerical values ​​to construct a priority data column, and according to the priority data column, perform feature extraction on several groups of parameters in the power status data output in real time in the power system according to priority.

7. The method for optimizing the efficiency of power data value extraction according to claim 6, characterized in that: The specific steps of S4 include: S41, in the process of feature extraction according to the priority in S34, real-time monitoring of the freshness of the data extraction process to obtain relevant extraction status data information, the relevant extraction status data information includes preprocessing time Te, data generation time T g and data extraction time T c ; S42. Calculate and obtain data freshness Sqxd according to relevant extraction status data information. The data freshness Sqxd is obtained by the following formula: Where, T e is the preprocessing time, which means the time for preprocessing multi-source heterogeneous data; T g T is the data generation time, which means the time when the data is generated by the sensor or other equipment; c is the data extraction time, which indicates the moment when the system extracts data from the power dataset.

8. The method for optimizing the efficiency of power data value extraction according to claim 7, characterized in that: The specific steps of S4 also include: S42. Construct an efficiency prediction model using convolutional neural network technology, input the data freshness Sqxd and the priority estimation coefficient Ygxs into the efficiency prediction model, and after linear normalization, fit the output extraction efficiency index Txzs from the efficiency prediction model. The extraction efficiency index Txzs is obtained by the following formula: Where K represents the number of parameters, k = 1, 2, 3, ..., K, Sqxd k Represented as the data freshness of the kth parameter, Sqxd avg Expressed as average data freshness, Ygxs k Expressed as the priority estimation coefficient of the kth parameter, Ygxs avg It is expressed as the average priority estimation coefficient, F1 and F2 are both weight values, P is expressed as the second correction constant, and the specific values ​​of F1 and F2 are set by the user according to the situation.

9. The method for optimizing the efficiency of power data value extraction according to claim 1, characterized in that: The specific steps of S5 include: S51, by comparing and analyzing the preset safety threshold Q with the extraction efficiency index Txzs, a comprehensive judgment is made on the extraction efficiency state of the parameters in the corresponding monitoring period when the features are extracted. The specific judgment contents are as follows: If the extraction efficiency index Txzs falls within the safety threshold Q, it is judged that the extraction efficiency of the parameters in the corresponding monitoring period is in a normal state when the features are extracted. At this time, the feature extraction operation of the current power status data output in real time in the power system will continue; S52. If the extraction efficiency index Txzs does not fall within the safety threshold Q, it is determined that the extraction efficiency of the parameters in the corresponding monitoring period is not in a normal state during feature extraction. At this time, the cache technology will be used to extract and process the data in advance, and the priority of each parameter will be re-evaluated according to the actual application scenario.

10. A system for optimizing the efficiency of value extraction of electric power data, used to implement the method for optimizing the efficiency of value extraction of electric power data as described in any one of claims 1 to 9, characterized in that: It includes data acquisition module, preprocessing module, extraction sequence analysis module, comprehensive evaluation module and feedback module; The data acquisition module is used to collect power status data monitored by several groups of monitoring devices at different time periods from various links in the power system in advance to construct a multi-source heterogeneous data set; The preprocessing module is used to perform preprocessing operations on the multi-source heterogeneous data sets before feature extraction, and perform data cleaning, standardization and binning on the multi-source heterogeneous data sets to generate a power data set; The extraction sequence analysis module is used to analyze the dependency between the real-time parameters and the corresponding parameters in the power data set based on the power data set and in combination with the power status data monitored in real time by several groups of monitoring devices, so as to construct a similarity coefficient Xsxs, and analyze the dispersion state of the corresponding parameters in different monitoring periods in the time series to calculate the data distribution coefficient Xbxs of the corresponding parameters, and obtain the priority estimation coefficient Ygxs by associating the similarity coefficient Xsxs with the data distribution coefficient Xbxs of the corresponding parameters, and determine the parameter extraction order based on the priority estimation coefficient Ygxs; The comprehensive evaluation module is used to analyze the timeliness of the data extraction process based on the extraction order of each parameter in S3 to obtain the data freshness Sqxd, and combine the trained efficiency prediction model and linear normalization processing to fit the output extraction efficiency index Txzs; The feedback module is used to pre-set the safety threshold Q, and compare and analyze the extraction efficiency index Txzs with the safety threshold Q to comprehensively judge the extraction efficiency of the parameters in the corresponding monitoring period during feature extraction, and take corresponding optimization measures based on the judgment results.

Images