A method and system for classifying and grading power data based on ensemble learning
By preprocessing, feature screening and grading rules for power data based on integrated learning, a data division model is constructed, which solves the problem of insufficient accuracy of power data classification and grading, and improves data analysis and management efficiency.
Patent Information
- Application Number
- CN202410985098.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-23
- Publication Date
- 2024-11-08
- Estimated Expiration
- 2044-07-23
Smart Images

Figure CN118520378B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of power data processing, and in particular to a power data classification and grading method and system based on ensemble learning. Background Art
[0002] As the intelligence and automation of power systems continue to improve, a large amount of power data is generated and accumulated. These data cover multiple aspects such as voltage, current, and power. How to scientifically and reasonably classify and grade the collected power data has become a major challenge in the power industry.
[0003] At present, due to the diversity and complexity of power data, existing technologies are unable to accurately divide it, resulting in insufficient accuracy in the classification and grading of power data, causing technical problems such as low efficiency and poor processing quality in subsequent power data analysis and processing. Summary of the invention
[0004] The purpose of the present invention is to provide a method and system for classifying and grading power data based on ensemble learning, so as to solve the technical problem that due to the diversity and complexity of power data, the existing technology cannot accurately divide it, resulting in insufficient accuracy of power data classification and grading, resulting in low efficiency and poor processing quality of subsequent power data analysis and processing.
[0005] In view of the above problems, the present invention provides a method and system for classifying and grading power data based on ensemble learning.
[0006] In the first aspect, the present invention provides a method for classifying and grading electric power data based on ensemble learning, and the method is implemented by a system for classifying and grading electric power data based on ensemble learning, wherein the method includes: coordinating source-end electric power data, preprocessing and regularizing the source-end electric power data in combination with a preprocessing model, and determining an electric power data set; traversing the electric power data set, extracting attribute features and screening key features, and performing feature conversion on the key features based on feature stability and universality to determine effective data features, wherein the conversion principle is to not change feature attributes; setting data classification rules, wherein the data classification rules are based on data sensitivity type and core degree, and are set by combining qualitative and quantitative methods. Determine; train a data classifier based on the valid data features and the power data set, train a data classifier based on the data classification rules, and jointly construct a data partitioning model; monitor and transmit back the time series power data, perform data preprocessing at the edge node in combination with the preprocessing model, determine the pre-divided power data and transmit it back to the data center; perform horizontal classification and vertical classification processing on the pre-divided power data in combination with the data partitioning model to determine the data partitioning result, wherein the data partitioning result includes a data partitioning system; identify the data partitioning result, perform back-end classification and storage on the pre-divided power data, perform front-end visualization on the data partitioning system, and establish data mapping between the front-end and the back-end.
[0007] In the second aspect, the present invention also provides an electric power data classification and grading system based on ensemble learning, which is used to execute an electric power data classification and grading method based on ensemble learning as described in the first aspect, wherein the system includes: a data preprocessing module, which is used to coordinate the source power data, and preprocess and regularize the source power data in combination with the preprocessing model to determine the power data set; an effective data feature determination module, which is used to traverse the power data set, extract attribute features and screen key features, and perform feature conversion on the key features based on feature stability and universality to determine the effective data features, wherein the conversion principle is not to change the feature attributes; a data classification rule setting module, which is used to set data classification rules, and the data classification rules are set based on the data sensitivity type and core degree, combining qualitative and quantitative methods; data partitioning model composition Module, the data partitioning model construction module is used to train a data classifier based on the effective data features and the power data set, train a data classifier based on the data classification rules, and jointly construct a data partitioning model; a pre-partitioned power data feedback module, the pre-partitioned power data feedback module is used to monitor and feedback time series power data, perform data preprocessing in combination with the preprocessing model at the edge node, determine the pre-partitioned power data and return it to the data center; a classification and grading processing module, the classification and grading processing module is used to combine the data partitioning model to perform horizontal classification and vertical grading processing on the pre-partitioned power data to determine the data partitioning result, wherein the data partitioning result includes a data partitioning system; a data mapping establishment module, the data mapping establishment module is used to identify the data partitioning result, perform back-end classification storage on the pre-partitioned power data, perform front-end visualization on the data partitioning system, and establish data mapping between the front-end and the back-end.
[0008] One or more technical solutions provided in the present invention have at least the following technical effects or advantages:
[0009] By coordinating the source power data, the source power data is preprocessed and regularized in combination with the preprocessing model to determine the power data set; traversing the power data set, extracting attribute features and screening key features, based on feature stability and universality, the key features are converted to determine effective data features, wherein the conversion principle is not to change the feature attributes; setting data classification rules, the data classification rules are based on data sensitivity type and core degree, and are set in a qualitative and quantitative manner; training a data classifier based on the effective data features and the power data set, and training a data classifier based on the data classification rules, and jointly forming a data partitioning model; monitoring and returning time series power data, performing data preprocessing in combination with the preprocessing model at the edge node, determining pre-divided power data and returning it to the data center; combining the data partitioning model, performing horizontal classification and vertical classification processing on the pre-divided power data, and determining the data partitioning result, wherein the data partitioning result includes a data partitioning system; identifying the data partitioning result, performing back-end classification storage on the pre-divided power data, performing front-end visualization on the data partitioning system, and establishing data mapping between the front-end and the back-end. That is to say, by acquiring the power data at the source end, and performing data cleaning and data normalization on the power data at the source end, a power data set is obtained; then, based on the power data set, key features are screened, and the key features that do not meet the predetermined feature stability and universality are converted based on the principle of not changing the feature attributes, and effective data features are determined; on the other hand, based on the data sensitivity type and core degree, data classification rules are configured in a qualitative and quantitative manner; multiple data classifiers and data classifiers are constructed based on a feedforward neural network, and then multiple data classifiers are supervised and trained through effective data features and power data sets, and multiple convergent data classifiers are integrated and fused based on a predetermined classifier integration strategy, multiple data classifiers are trained through data classification rules and the power data set, and multiple data classifiers are integrated and fused, and finally a data partitioning model is constructed by combining the integrated data classifier and the integrated data classifier; finally, the pre-divided power data is subjected to horizontal classification and vertical classification processing through the data partitioning model to obtain the data partitioning result. The precision and accuracy of power data classification and grading can be improved, which is convenient for subsequent targeted power data analysis and processing, thereby achieving the technical effect of improving power data management efficiency and processing quality.
[0010] The above description is only an overview of the technical solution of the present invention. In order to more clearly understand the technical means of the present invention, it can be implemented according to the contents of the specification, and in order to make the above and other purposes, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are specifically cited below. It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present invention, nor is it intended to limit the scope of the present invention. Other features of the present invention will become easy to understand through the following description. BRIEF DESCRIPTION OF THE DRAWINGS
[0011] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings in the following description are only exemplary, and for ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.
[0012] Figure 1 A schematic diagram of a flow chart of a method for classifying and grading power data based on ensemble learning according to the present invention;
[0013] Figure 2 A schematic diagram of a process of extracting attribute features and screening key features in a power data classification and grading method based on ensemble learning of the present invention;
[0014] Figure 3 The present invention is a structural schematic diagram of a power data classification and grading system based on ensemble learning.
[0015] Description of reference numerals:
[0016] Data preprocessing module 11, effective data feature determination module 12, data classification rule setting module 13, data partitioning model construction module 14, pre-divided power data feedback module 15, classification and grading processing module 16, data mapping establishment module 17. DETAILED DESCRIPTION
[0017] The present invention provides a method and system for classifying and grading power data based on ensemble learning, which solves the problem that the existing technology cannot accurately classify power data due to its diversity and complexity, resulting in insufficient accuracy of power data classification and grading, and causing low efficiency and poor processing quality of subsequent power data analysis and processing. The precision and accuracy of power data classification and grading can be improved, which facilitates the subsequent targeted analysis and processing of power data, thereby achieving the technical effect of improving power data management efficiency and processing quality.
[0018] Below, the technical solutions in the present invention will be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all of the embodiments of the present invention. It should be understood that the present invention is not limited to the example embodiments described herein. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention. It should also be noted that, for the convenience of description, only the parts related to the present invention are shown in the accompanying drawings, rather than all of them.
[0019] Embodiment 1
[0020] Please see attached Figure 1 The present invention provides a method for classifying and grading power data based on ensemble learning, wherein the method is applied to a power data classification and grading system based on ensemble learning, and the method specifically comprises the following steps:
[0021] Step 1: Coordinate the source power data, preprocess and regularize the source power data in combination with the preprocessing model, and determine the power data set;
[0022] Specifically, the power data management system is connected to collect source power data, wherein the source power data refers to power data with complete coverage, that is, power data including the entire power circulation cycle, such as power generation data, transmission data, substation data, power consumption data, etc. For example, power generation data includes specific power generation parameters such as power generation, power generation capacity, and power generation efficiency. Then, the source power data is input into the preprocessing model for data processing and data regularization to obtain a power data set. The preprocessing model is embedded with a predetermined data processing strategy, and the predetermined data processing strategy includes at least a data cleaning strategy and a normalization processing strategy, wherein the data cleaning strategy is used to clean and correct outliers, missing values, duplicate records, etc. in the data, and the normalization processing strategy is used to perform dimensionless processing on the power data to facilitate subsequent data analysis.
[0023] By cleaning and normalizing the source power data, we can eliminate data noise, outliers and other problems, improve the accuracy and reliability of the power data set, and provide strong support for subsequent data analysis.
[0024] Step 2: traverse the power data set, extract attribute features and screen key features, perform feature conversion on the key features based on feature stability and universality, and determine valid data features, wherein the conversion principle is to not change the feature attributes;
[0025] Specifically, the power data set is traversed and analyzed, and attribute features are extracted based on the power data set, where attribute features refer to key indicators that can reflect the state, performance and operation mode of the power system, such as: voltage, current, power, load, equipment status, etc.; then the attribute features are screened to determine key features, which include time domain features, frequency domain features, statistical features and other data.
[0026] Determine the feature stability and feature universality, where feature stability and universality are two important indicators for evaluating the quality of key features in power data sets. Feature stability reflects the degree of change of feature values in time or space, while universality refers to the versatility and applicability of features in different scenarios or conditions. Based on these two benchmarks, we can perform feature transformation on key features to improve their stability and universality. Then, based on feature stability and feature universality as the benchmark and without changing feature attributes as the transformation principle, perform feature transformation on key features that do not meet the feature stability and feature universality. Feature transformation is used to improve the intuitiveness and adaptability of key features. For example, convert a certain unpopular feature information into a feature with higher adaptability, while the feature attributes remain unchanged. The feature dimensions are transformed to obtain valid data features.
[0027] By performing key feature screening and feature conversion, the sophistication and accuracy of effective data feature settings can be improved, while the stability and universality of features can be improved.
[0028] Step 3: Set data classification rules, which are based on data sensitivity type and core degree, and are set in a combination of qualitative and quantitative methods;
[0029] Specifically, based on the data sensitivity type and core degree, data classification rules are set based on a combination of qualitative and quantitative methods, where the data sensitivity type is determined by the privacy and confidentiality of the data, where the higher the data privacy and confidentiality, the higher the data sensitivity; the data core degree refers to the importance of the data, which can be set based on actual conditions, where the higher the data importance, the higher the core degree. First, based on the data sensitivity type and core degree, a subjective division of data classification is performed to determine the initial data classification rules, and then based on data quantitative indicators, such as access frequency, modification frequency, data volume, etc., the initial data classification rules are optimized and adjusted to obtain data classification rules, where data levels include important data, general data and other types, and each data type includes multiple levels, such as first-level important data, second-level important data, etc. By configuring data classification rules based on a combination of qualitative and quantitative methods based on data sensitivity type and core degree, the scientificity, precision and rationality of data classification rule setting can be improved, thereby improving the sophistication and accuracy of data classification.
[0030] Step 4: training a data classifier based on the valid data features and the power data set, and training a data classifier based on the data classification rules, and jointly forming a data partitioning model;
[0031] Specifically, a network architecture of a data classifier and a data classifier is constructed based on a feedforward neural network to obtain multiple initial data classifiers and multiple initial data classifiers, wherein the initial data classifier and the initial data classifier are neural network models that can be iteratively optimized in machine learning. Then, multiple initial data classifiers are supervised and trained using effective data features and power data sets to obtain multiple converged data classifiers, and then multiple converged data classifiers are integrated and fused based on a predetermined classifier integration strategy to obtain a data classifier.
[0032] Based on the data classification rules, the power data in the power data set is classified and processed to determine the data classification set; then multiple initial data classifiers are trained through the data classification set and the power data set, and the data classifier is integrated and fused, and finally the data classifier and the data classifier are connected in series to construct a data partitioning model, that is, in the data partitioning model, the output layer of the data classifier is connected to the input layer of the data classifier, that is, the data is first classified and processed, and then the data classification processing results are input into the data classifier for classification processing. By constructing a data partitioning model based on a feedforward neural network, the accuracy of power data classification and grading can be improved, while saving data processing time and improving data classification and grading efficiency.
[0033] Step 5: monitor and transmit back the time series power data, perform data preprocessing in combination with the preprocessing model at the edge node, determine the pre-divided power data and transmit it back to the data center;
[0034] Specifically, monitor and transmit back time-series power data, which refers to power data with time node identifiers, that is, power data columns recorded in chronological order. These data are mainly continuously generated by various measurement points or event sources in the power system, and are crucial for the monitoring, analysis and management of the power system, and can reflect the real-time operating status and changing trends of the power system. And at the edge node, the time-series power data is cleaned and normalized through the preprocessing model to obtain pre-divided power data that has been preprocessed, and the pre-divided power data is transmitted back to the data center. By preprocessing the time-series power data at the edge node, redundant and irrelevant data can be removed, and valuable data can be retained, thereby improving data transmission efficiency.
[0035] Step 6: In combination with the data partitioning model, perform horizontal classification and vertical classification processing on the pre-divided power data to determine a data partitioning result, wherein the data partitioning result includes a data partitioning system;
[0036] Specifically, the pre-divided power data is then input into the data partitioning model, and horizontal classification processing and vertical grading processing are performed respectively, and the data partitioning results are output, wherein the data partitioning results include a data partitioning system, and the data partitioning system refers to the classification and grading results of the power data and category labels, and the category labels can be set based on actual needs, for example: according to the application scenario, the categories of power generation, transmission, distribution, and power consumption are set.
[0037] Step 7: Identify the data partitioning result, perform back-end classification storage on the pre-partitioned power data, perform front-end visualization on the data partitioning system, and establish data mapping between the front-end and the back-end.
[0038] Specifically, the data division results are further identified, and the pre-divided power data is stored in the back-end server according to preset rules and structures to ensure the integrity, consistency and efficiency of the data; the data division system is visualized on the front end, that is, the data classification and grading results and category labels are displayed in a graphical and intuitive manner to facilitate data viewing and analysis; and a data mapping is established between the data division system on the front end and the pre-divided power data on the back end to provide an efficient and intuitive platform for the management and analysis of power data, which is convenient for subsequent data analysis and management.
[0039] The method for classifying and grading electric power data based on ensemble learning is applied to an electric power data classification and grading system based on ensemble learning, which can solve the technical problems that due to the diversity and complexity of electric power data, the existing technology cannot accurately classify it, resulting in insufficient accuracy of electric power data classification and grading, and causing low efficiency and poor processing quality of subsequent electric power data analysis and processing. First, coordinate the source power data, and combine the preprocessing model to preprocess and regularize the source power data to determine the power data set; then, traverse the power data set, extract attribute features and screen key features, and perform feature conversion on the key features based on feature stability and universality to determine valid data features, wherein the conversion principle is not to change feature attributes; then, set data classification rules, and the data classification rules are based on data sensitivity type and core degree, and are set in a combination of qualitative and quantitative methods; next, train a data classifier based on the valid data features and the power data set, and train a data classifier based on the data According to the classification rules, the data classifier is trained to jointly form a data partitioning model; in addition, the time series power data is monitored and transmitted back, and the data preprocessing is performed at the edge node in combination with the preprocessing model to determine the pre-divided power data and transmit it back to the data center; further, in combination with the data partitioning model, the pre-divided power data is subjected to horizontal classification and vertical classification processing to determine the data partitioning result, wherein the data partitioning result includes a data partitioning system; finally, the data partitioning result is identified, the pre-divided power data is classified and stored at the back end, the data partitioning system is visualized at the front end, and data mapping between the front end and the back end is established. The precision and accuracy of the classification and grading of power data can be improved, which is convenient for subsequent targeted analysis and processing of power data, thereby achieving the technical effect of improving the efficiency of power data management and the processing quality.
[0040] Further, the extraction of attribute features and screening of key features, such as the attached Figure 2 As shown, step 2 of the present invention comprises:
[0041] Determine the feature screening principles, among which feature importance is the first screening principle, feature personalization is the second screening principle, and boundary clarity is the third screening principle;
[0042] Determine a principle threshold, wherein the principle threshold is a critical value of each screening principle;
[0043] Attribute features based on the power data set are extracted, and the attribute features are screened based on the feature screening principle and the principle threshold to determine the key features, wherein the key features are identified with weight coefficients, and the weight coefficients are positively correlated with the principle magnitude.
[0044] Specifically, first, determine the feature screening principle, where feature importance is the first screening principle, feature personalization is the second screening principle, and boundary clarity is the third screening principle, where feature importance refers to the core degree of the feature; feature personalization is used to ensure that the feature is unique to the class, which facilitates accurate feature division; boundary clarity refers to boundary clarity recognition, and data with high boundary clarity recognition can usually be more clearly and accurately identified and distinguished. Then determine the principle threshold of each screening principle, which is the critical value of each screening principle, and can be set by technicians in this field based on actual conditions.
[0045] Then, attribute features based on the power data set are extracted, wherein attribute features refer to key indicators that can reflect the state, performance and operation mode of the power system, such as voltage, current, power, load, equipment status, etc. Then, based on the feature screening principle and the principle threshold, the attribute features are screened, that is, the attribute features whose attribute features are greater than the feature importance threshold, greater than the feature personalization threshold, and greater than the boundary clarity threshold are extracted and set as the key features, wherein the key features are identified with weight coefficients, and the weight coefficients are positively correlated with the principle magnitude, that is, the higher the feature importance corresponding to the key features, the higher the feature personalization degree, and the more obvious the boundary clarity, the greater the weight coefficient of the key features.
[0046] Further, the key features are converted into features, and step 2 of the present invention includes:
[0047] Setting critical stability and critical universality, screening the key features, and determining pre-conversion features;
[0048] Based on the feature dimension and feature depth, the pre-conversion features are converted and replaced with features under the same attributes to determine effective key features;
[0049] Multimodal mapping association is performed on the effective key features to determine the effective data features.
[0050] Specifically, first, set the critical stability and critical universality, where the characteristic stability reflects the degree of change of the characteristic value in time or space, and the characteristic universality refers to the versatility and applicability of the characteristic in different scenarios or conditions. The critical stability and critical universality can be set by technicians in this field based on actual conditions. Then, based on the critical stability and the critical universality, the key features are screened, that is, the key features whose stability is less than the critical stability and / or whose universality is less than the critical universality are extracted and set as pre-conversion features.
[0051] Then, based on the feature dimension and feature depth, that is, based on the principle of feature conversion without changing the feature dimension and feature depth, the pre-conversion features are converted and replaced with the same attributes to determine the effective key features. Commonly used feature conversion methods include standardization, normalization, logarithmic transformation, box plot processing, combined features and other methods. For example, by combining multiple key features to create a new composite feature, the stability and universality of the feature can be further improved. For example, voltage and current can be combined to create new features such as power or power factor. These composite features can more comprehensively reflect the state and behavior of the power system.
[0052] The effective key features are further subjected to multimodal mapping association to determine the effective data features, wherein multimodal mapping association refers to associating data features of different modes (such as power data, text descriptions, images, etc.) in some way to provide more comprehensive and accurate information. By performing multimodal mapping association on the effective key features, corresponding identification and extraction can be quickly performed, and extraction bias caused by modal differences can be avoided.
[0053] Further, based on the effective data features and the power data set training data classifier, step 4 of the present invention includes:
[0054] Based on the power business perspective, hierarchical clustering and inter-class association are performed on the effective data features to determine a feature classification tree;
[0055] Evenly dividing the power data set to determine N power data groups;
[0056] Traversing the N power data groups to extract a group of power data, combining the feature classification tree, and supervising the training of a first classifier;
[0057] The training of the Nth classifier is completed, and the first classifier to the Nth classifier are fused to determine the data classifier.
[0058] Specifically, first, based on the power business perspective, the valid data features are hierarchically clustered and associated between classes to determine a feature classification tree, wherein the power business perspective can be set based on actual conditions, for example: residential electricity, commercial electricity, industrial electricity, etc., that is, the valid data features are clustered based on hierarchical relationships and inter-class relationships to generate a feature classification tree.
[0059] Then, the power data set is divided into N equal parts to generate N power data groups, and N data classifiers are constructed based on a feedforward neural network, wherein the data classifier includes an input layer, multiple hidden layers, and an output layer, wherein the input data of the input layer is power data, and the output data is a feature classification result; then, a group of power data is extracted from the N power data groups, wherein the first group of power data is any one of the N power data groups, and then the first classifier is supervised and trained based on the first group of power data and the feature classification tree to obtain a first classifier that tends to converge; the N data classifiers are supervised and trained using the N power data groups to obtain N classifiers. Then, the first classifier to the Nth classifier are fused to generate the data classifier.
[0060] Furthermore, the present invention further comprises the following steps:
[0061] Based on the N power data groups, cross-validate the N classifiers to determine the convergence degree of the classifiers;
[0062] Traversing the classifier convergence, extracting overfitting classifiers that do not meet the convergence threshold, dividing random samples based on the power data set, and performing retraining and verification to obtain N classifiers that meet the convergence conditions;
[0063] Determine a classifier integration strategy, perform fusion processing on the N classifiers, and obtain the data classifier.
[0064] Specifically, the first classifier to the Nth classifier are fused to determine the method of the data classifier as follows: first, based on the N power data groups, the N classifiers are cross-validated, wherein cross-validation refers to using the non-training data of the classifier as verification data, performing verification test on the accuracy of the output results of the N classifiers, and determining the classifier convergence based on the verification test results, wherein the classifier convergence is the classifier output accuracy.
[0065] A convergence threshold is obtained, which is an expected output accuracy index and can be set based on actual needs; the convergence of the classifier is judged based on the convergence threshold, and the classifier corresponding to the classifier convergence less than the convergence threshold is extracted and set as an overfitting classifier, where the overfitting classifier refers to a classifier that does not meet the standard. Then, random samples are divided based on the power data set, and the overfitting classifier is retrained and verified based on the random samples until N classifiers that meet the convergence threshold are obtained.
[0066] Determine the classifier integration strategy, which is a classifier integration method. The classifier integration strategy can be set as a voting strategy, for example: select the mode value of the output results of N classifiers and set it as the final output result. The weighted voting method can also be used, that is, different voting weights are set according to the convergence degree of the classifier; based on the classifier integration strategy, the N classifiers are fused to obtain the data classifier. By constructing multiple classifiers based on a feedforward neural network and fusing multiple classifiers based on a classifier integration strategy to obtain a data classifier, the accuracy and stability of the data classifier construction can be improved, thereby improving the accuracy and stability of power data classification.
[0067] In addition, the construction principle and training method of the data classifier are consistent with those of the data classifier. In order to reflect the conciseness of the specification, they are not elaborated here.
[0068] Further, after establishing the data mapping between the front end and the back end, step seven of the present invention also includes:
[0069] Continuously transmit dynamic power data and perform classification and grading processing to locate pre-change nodes, where the pre-change nodes are marked with adjustment features that are synchronized with the power business angle;
[0070] Incrementally synchronizing the data partition system based on the dynamic power data, identifying adjustment features based on the pre-changed nodes, adjusting the data partition system, and determining a changed partition system;
[0071] The mapping between the change division system and the back-end storage power data is updated.
[0072] Specifically, after establishing the data mapping between the front end and the back end, the dynamic power data is continuously transmitted back and classified and graded, and the pre-change node is located, wherein the pre-change node refers to a key node in the power system that needs to adjust data features or change business logic due to changes in business needs, equipment status or operating environment. These adjustments need to be synchronized with the power business perspective to ensure the validity of the data and the smooth progress of the business. The pre-change node is marked with adjustment features that are synchronized with the power business perspective. Then, the data partitioning system is incrementally synchronized based on the dynamic power data, and the adjustment features of the pre-change node are identified. Then, the data partitioning system is adjusted according to the adjustment features, that is, the data partitioning system is optimized in real time to obtain a changed partitioning system, and the mapping between the changed partitioning system and the back-end storage power data is updated. By adjusting the data partitioning system in real time based on the power business perspective, the adaptability of the data partitioning system to the actual power business can be improved, and the accuracy of subsequent data analysis can be improved.
[0073] Further, after establishing the data mapping between the front-end and the back-end, configuring the data management measures and access control strategies, step seven of the present invention includes:
[0074] Determine common control standards and individual control standards based on the data classification rules and the power business perspective;
[0075] The data partitioning system is traversed, and the data management measures and the access control policies are configured based on the mapped common management and control standards and individual management and control standards.
[0076] Specifically, first, the common control standards and individual control standards based on the data classification rules and the power business perspective are determined. The common control standards refer to the control standards that are unified with the data classification rules and the power business perspective. The individual control standards are independent control standards with special characteristics. Common control standards and individual control standards can be set based on actual conditions. By determining common control standards and individual control standards, it can be ensured that power data is effectively managed and utilized at different levels and in different business scenarios, providing strong support for the smooth development of power business.
[0077] Then traverse the data division system, and configure data management measures and access control strategies according to the mapped common control standards and individual control standards, wherein basic data management measures and basic access control strategies are configured based on the common control standards, such as: unified identity authentication method, unified data access rights, etc.; independent data management measures and independent access control strategies are configured based on the individual control standards, that is, some data only has access rights for data classification rules or power business perspectives. By configuring data management measures and access control strategies based on common control standards and individual control standards, the precision of data management measures and access control strategy settings can be improved, thereby improving the security of data access.
[0078] In summary, the power data classification and grading method based on ensemble learning provided by the present invention has the following technical effects:
[0079] 1. Obtain the power data from the source end, and perform data cleaning and data normalization on the power data from the source end to obtain the power data set; then select key features based on the power data set, and perform feature conversion on key features that do not meet the predetermined feature stability and universality based on the principle of not changing the feature attributes, and determine the effective data features; on the other hand, configure the data classification rules based on the data sensitivity type and core degree in a qualitative and quantitative manner; construct multiple data classifiers and data classifiers based on the feedforward neural network, and then supervise the multiple data classifiers through the effective data features and the power data set, and integrate and fuse multiple convergent data classifiers based on the predetermined classifier integration strategy, train multiple data classifiers through the data classification rules and the power data set, and integrate and fuse multiple data classifiers, and finally jointly construct a data partitioning model with the integrated data classifier and the integrated data classifier; finally, perform horizontal classification and vertical classification processing on the pre-divided power data through the data partitioning model to obtain the data partitioning result. The precision and accuracy of power data classification and grading can be improved, which is convenient for subsequent targeted power data analysis and processing, thereby achieving the technical effect of improving power data management efficiency and processing quality.
[0080] 2. By constructing multiple classifiers based on a feedforward neural network and fusing multiple classifiers based on a classifier integration strategy to obtain a data classifier, the accuracy and stability of data classifier construction can be improved, thereby improving the accuracy and stability of power data classification.
[0081] Embodiment 2
[0082] Based on the same inventive concept as the power data classification and grading method based on ensemble learning in the aforementioned embodiment, the present invention also provides a power data classification and grading system based on ensemble learning, see the attached Figure 3 , the system comprising:
[0083] A data preprocessing module 11, the data preprocessing module 11 is used to coordinate the source power data, preprocess and regularize the source power data in combination with the preprocessing model, and determine the power data set;
[0084] An effective data feature determination module 12, the effective data feature determination module 12 is used to traverse the power data set, extract attribute features and screen key features, perform feature conversion on the key features based on feature stability and universality, and determine effective data features, wherein the conversion principle is to not change the feature attributes;
[0085] A data classification rule setting module 13, which is used to set data classification rules. The data classification rules are set based on the data sensitivity type and core degree, and are set in a qualitative and quantitative manner;
[0086] A data partition model forming module 14, wherein the data partition model forming module 14 is used to train a data classifier based on the effective data features and the power data set, and to train a data classifier based on the data classification rules, and jointly form a data partition model;
[0087] A pre-divided power data feedback module 15, which is used to monitor and feedback time-series power data, perform data preprocessing in combination with the preprocessing model at the edge node, determine the pre-divided power data and transmit it back to the data center;
[0088] A classification and grading processing module 16, wherein the classification and grading processing module 16 is used to perform horizontal classification and vertical grading processing on the pre-divided power data in combination with the data partitioning model to determine a data partitioning result, wherein the data partitioning result includes a data partitioning system;
[0089] The data mapping establishment module 17 is used to identify the data division result, perform back-end classification storage on the pre-divided power data, perform front-end visualization on the data division system, and establish data mapping between the front-end and the back-end.
[0090] Furthermore, the effective data feature determination module 12 in the system is also used for:
[0091] Determine the feature screening principles, among which feature importance is the first screening principle, feature personalization is the second screening principle, and boundary clarity is the third screening principle;
[0092] Determine a principle threshold, wherein the principle threshold is a critical value of each screening principle;
[0093] Attribute features based on the power data set are extracted, and the attribute features are screened based on the feature screening principle and the principle threshold to determine the key features, wherein the key features are identified with weight coefficients, and the weight coefficients are positively correlated with the principle magnitude.
[0094] Furthermore, the effective data feature determination module 12 in the system is also used for:
[0095] Setting critical stability and critical universality, screening the key features, and determining pre-conversion features;
[0096] Based on the feature dimension and feature depth, the pre-conversion features are converted and replaced with features under the same attributes to determine effective key features;
[0097] Multimodal mapping association is performed on the effective key features to determine the effective data features.
[0098] Furthermore, the data partition model forming module 14 in the system is also used for:
[0099] Based on the power business perspective, hierarchical clustering and inter-class association are performed on the effective data features to determine a feature classification tree;
[0100] Evenly dividing the power data set to determine N power data groups;
[0101] Traversing the N power data groups to extract a group of power data, combining the feature classification tree, and supervising the training of a first classifier;
[0102] The training of the Nth classifier is completed, and the first classifier to the Nth classifier are fused to determine the data classifier.
[0103] Furthermore, the data partition model forming module 14 in the system is also used for:
[0104] Based on the N power data groups, cross-validate the N classifiers to determine the convergence degree of the classifiers;
[0105] Traversing the classifier convergence, extracting overfitting classifiers that do not meet the convergence threshold, dividing random samples based on the power data set, and performing retraining and verification to obtain N classifiers that meet the convergence conditions;
[0106] Determine a classifier integration strategy, perform fusion processing on the N classifiers, and obtain the data classifier.
[0107] Furthermore, the data mapping establishment module 17 in the system is also used for:
[0108] Continuously transmit dynamic power data and perform classification and grading processing to locate pre-change nodes, where the pre-change nodes are marked with adjustment features that are synchronized with the power business angle;
[0109] Incrementally synchronizing the data partition system based on the dynamic power data, identifying adjustment features based on the pre-changed nodes, adjusting the data partition system, and determining a changed partition system;
[0110] The mapping between the change division system and the back-end storage power data is updated.
[0111] Furthermore, the data mapping establishment module 17 in the system is also used for:
[0112] Determine common control standards and individual control standards based on the data classification rules and the power business perspective;
[0113] The data partitioning system is traversed, and the data management measures and the access control policies are configured based on the mapped common management and control standards and individual management and control standards.
[0114] Each embodiment in this specification is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The method for classifying and grading electric data based on ensemble learning and the specific examples in the aforementioned embodiment 1 are also applicable to the system for classifying and grading electric data based on ensemble learning in this embodiment. Through the aforementioned detailed description of the method for classifying and grading electric data based on ensemble learning, those skilled in the art can clearly know the system for classifying and grading electric data based on ensemble learning in this embodiment, so for the sake of brevity of the specification, it will not be described in detail here. For the system disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part description.
[0115] The above description of the disclosed embodiments enables one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.
[0116] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention belong to the scope of the present invention and its equivalent technologies, the present invention is also intended to include these changes and variations.
Claims
1. A method for classifying and grading power data based on ensemble learning. It is characterized in that The method comprises: Coordinate the source power data, preprocess and regularize the source power data in combination with the preprocessing model, and determine the power data set; Traversing the power data set, extracting attribute features and screening key features, performing feature conversion on the key features based on feature stability and universality, and determining effective data features, wherein the conversion principle is to not change the feature attributes; Setting data classification rules, which are based on the data sensitivity type and core degree, and are set in a combination of qualitative and quantitative methods; A data classifier is trained based on the effective data features and the power data set, and a data classifier is trained based on the data classification rules, and the data classification model is jointly constructed; Monitor and transmit back the time series power data, perform data preprocessing in combination with the preprocessing model at the edge node, determine the pre-divided power data and transmit it back to the data center; In combination with the data partitioning model, horizontal classification and vertical classification processing are performed on the pre-divided power data to determine a data partitioning result, wherein the data partitioning result includes a data partitioning system; The data partitioning result is identified, the pre-partitioned power data is classified and stored at the back end, the data partitioning system is visualized at the front end, and data mapping between the front end and the back end is established.
2. The method according to claim 1, It is characterized in that The method of extracting attribute features and screening key features further comprises: Determine the feature screening principles, among which feature importance is the first screening principle, feature personalization is the second screening principle, and boundary clarity is the third screening principle; Determine a principle threshold, wherein the principle threshold is a critical value of each screening principle; Attribute features based on the power data set are extracted, and the attribute features are screened based on the feature screening principle and the principle threshold to determine the key features, wherein the key features are identified with weight coefficients, and the weight coefficients are positively correlated with the principle magnitude.
3. The method according to claim 1, It is characterized in that Performing feature conversion on the key features, the method further comprising: Setting critical stability and critical universality, screening the key features, and determining pre-conversion features; Based on the feature dimension and feature depth, the pre-conversion features are converted and replaced with features under the same attributes to determine effective key features; Multimodal mapping association is performed on the effective key features to determine the effective data features.
4. The method according to claim 1, It is characterized in that The method further comprises: training a data classifier based on the valid data features and the power data set; Based on the power business perspective, hierarchical clustering and inter-class association are performed on the effective data features to determine a feature classification tree; Evenly dividing the power data set to determine N power data groups; Traversing the N power data groups to extract a group of power data, combining the feature classification tree, and supervising the training of a first classifier; The training of the Nth classifier is completed, and the first classifier to the Nth classifier are fused to determine the data classifier.
5. The method according to claim 4, It is characterized in that The first classifier to the Nth classifier are integrated to determine the data classifier, and the method further includes: Based on the N power data groups, cross-validate the N classifiers to determine the convergence degree of the classifiers; Traversing the classifier convergence, extracting overfitting classifiers that do not meet the convergence threshold, dividing random samples based on the power data set, and performing retraining and verification to obtain N classifiers that meet the convergence conditions; Determine a classifier integration strategy, perform fusion processing on the N classifiers, and obtain the data classifier.
6. The method according to claim 4, It is characterized in that After establishing the data mapping between the front end and the back end, the method further includes: Continuously transmit dynamic power data and perform classification and grading processing to locate pre-change nodes, where the pre-change nodes are marked with adjustment features that are synchronized with the power business angle; Incrementally synchronizing the data partition system based on the dynamic power data, identifying adjustment features based on the pre-changed nodes, adjusting the data partition system, and determining a changed partition system; The mapping between the change division system and the back-end storage power data is updated.
7. The method according to claim 4, It is characterized in that After establishing the data mapping between the front-end and the back-end, configure data management measures and access control policies, including: Determine common control standards and individual control standards based on the data classification rules and the power business perspective; The data partitioning system is traversed, and the data management measures and the access control policies are configured based on the mapped common management and control standards and individual management and control standards.
8. A power data classification and grading system based on ensemble learning, It is characterized in that For implementing the steps of the method according to any one of claims 1 to 7, the system comprises: A data preprocessing module, the data preprocessing module is used to coordinate the source power data, preprocess and regularize the source power data in combination with the preprocessing model, and determine the power data set; An effective data feature determination module, the effective data feature determination module is used to traverse the power data set, extract attribute features and screen key features, perform feature conversion on the key features based on feature stability and universality, and determine effective data features, wherein the conversion principle is to not change feature attributes; A data classification rule setting module, which is used to set data classification rules. The data classification rules are set based on the data sensitivity type and core degree, and are set in a combination of qualitative and quantitative methods; A data partition model construction module, the data partition model construction module is used to train a data classifier based on the effective data features and the power data set, train a data classifier based on the data classification rules, and jointly construct a data partition model; A pre-divided power data feedback module, which is used to monitor and feedback time-series power data, perform data preprocessing in combination with the preprocessing model at the edge node, determine the pre-divided power data and transmit it back to the data center; A classification and grading processing module, wherein the classification and grading processing module is used to perform horizontal classification and vertical grading processing on the pre-divided power data in combination with the data partitioning model to determine a data partitioning result, wherein the data partitioning result includes a data partitioning system; A data mapping establishment module is used to identify the data division result, perform back-end classification storage on the pre-divided power data, perform front-end visualization on the data division system, and establish data mapping between the front-end and the back-end.
Citation Information
Patent Citations
Power data text classification algorithm based on selective ensemble learning
CN114625868A
Automatic classification and grading method and device for power sensitive data and storage medium
CN115879017A
Multi-modal power sample classification method and system based on meta-learning
CN118194171A
Systems and methods for quantitative analysis of histopathology images using multi-classifier ensemble schemes
WO2015054666A1
