Sample preprocessing method and system based on power multi-business knowledge class

Abstract

The embodiment of the invention provides a sample preprocessing method and system based on electric power multi-business knowledge classes, and belongs to the technical field of electric power systems and electric power engineering. The sample preprocessing method comprises the following steps: acquiring a power multi-service data sample, wherein the power multi-service data sample comprises data of power generation, power transmission, power distribution, power sale and power system operation and management; performing data cleaning on the power multi-service data sample; classifying and marking the cleaned data according to the power multi-business knowledge class to generate structured data; performing feature extraction on the structured data, and extracting key features related to the power business; and carrying out standardization processing on the extracted key features to generate preprocessed sample data. According to the sample preprocessing method and system, the quality and applicability of the sample data can be improved.

Description

Technical Field

[0001] The present invention relates to the technical field of power systems and power engineering, and particularly to a sample preprocessing method and system based on power multi-service knowledge classes. Background Art

[0002] With the rapid development of the power industry, the scale and complexity of power systems have been continuously increasing, covering multiple business fields such as power generation, power transmission, power distribution, power sales, and power system operation and management. The power industry has generated a vast amount of data, including power generation, transmission losses, distribution network status, power sales records, and system operation logs. These data are characterized by multi-source, heterogeneous, and high-dimensional, which directly affect the quality and usability of the data.

[0003] Traditional power data processing methods usually target a single business field and lack the ability to comprehensively process multi-service data, making it difficult to meet the requirements of intelligent power systems for high-quality data. Moreover, with the development of smart grids and the energy Internet, the power industry has put forward higher requirements for data-driven intelligent applications (such as fault diagnosis, load forecasting, and market analysis), and existing preprocessing methods are difficult to meet these needs. Summary of the Invention

[0004] The purpose of the embodiments of the present invention is to provide a sample preprocessing method and system based on power multi-service knowledge classes, which can improve the quality and applicability of sample data.

[0005] To achieve the above purpose, the embodiments of the present invention provide a sample preprocessing method based on power multi-service knowledge classes, including:

[0006] Obtain power multi-service data samples, where the power multi-service data samples include data on power generation, power transmission, power distribution, power sales, and power system operation and management;

[0007] Perform data cleaning on the power multi-service data samples;

[0008] Classify and label the cleaned data according to power multi-service knowledge classes to generate structured data;

[0009] Extract features from the structured data to extract key features related to power services;

[0010] Perform standardization processing on the extracted key features to generate preprocessed sample data.

[0011] Optionally, performing data cleaning on the power multi-service data samples includes:

[0012] Use the interpolation method to fill in missing values;

[0013] Detect and correct outliers through statistical methods;

[0014] Delete duplicate data.

[0015] Optionally, classify and label the cleaned data according to power multi-business knowledge categories to generate structured data, including:

[0016] Divide data samples into categories such as power generation, power transmission, power distribution, power sales, and power system operation and management according to power multi-business knowledge categories;

[0017] Label the data of each category, and generate the structured data after processing.

[0018] Optionally, extract features from the structured data, and extract key features related to power business, including:

[0019] Extract features such as power generation, power generation efficiency, and equipment status from power generation data;

[0020] Extract features such as power transmission loss, line load, and failure rate from power transmission data;

[0021] Extract features such as distribution network topology, load distribution, and equipment health status from distribution data;

[0022] Extract features such as power sales volume, electricity price, and user type from power sales data;

[0023] Extract features such as system stability, dispatching instructions, and operation logs from power system operation and management data.

[0024] Optionally, perform standardization processing on the extracted key features to generate preprocessed sample data, including:

[0025] Perform normalization processing on the extracted key features to make them fall within a unified range;

[0026] Encode the normalized features to generate data suitable for the input format of machine learning models.

[0027] Optionally, the sample preprocessing method further includes:

[0028] Perform quality assessment on the preprocessed sample data;

[0029] Save the sample data that meets the quality standards.

[0030] Optionally, the sample preprocessing method further includes:

[0031] Dynamically update the sample data according to power multi-business knowledge categories;

[0032] Use a machine learning model to train and validate the preprocessed sample data, and optimize the preprocessing process.

[0033] On the other hand, the present invention also provides a sample preprocessing system based on power multi-service knowledge classes. The sample preprocessing system includes a controller, and the controller is used to execute the sample preprocessing method as described in any one of the above.

[0034] Through the above technical solutions, the sample preprocessing method and system based on power multi-service knowledge classes provided by the present invention select knowledge in multiple fields such as power generation, power transmission, power distribution, power sales, and power system operation and management as initial data samples, realizing comprehensive preprocessing of power industry data. By introducing power multi-service knowledge classes, classifying, labeling, and feature extracting the sample data, the structuring and usability of the sample data are significantly improved. Combining the characteristics of the power industry, a targeted data cleaning, feature extraction, and standardization processing process is designed, solving the deficiencies existing in the application of existing methods in the power industry. Through a dynamic update and quality assessment mechanism, the timeliness and accuracy of the sample data are ensured, providing reliable data support for the intelligent application of the power industry, thereby improving the quality and applicability of the sample data.

[0035] Other features and advantages of the embodiments of the present invention will be described in detail in the subsequent specific implementation part. Brief Description of the Drawings

[0036] The drawings are used to provide a further understanding of the embodiments of the present invention, and constitute a part of the specification. Together with the following specific implementation, they are used to explain the embodiments of the present invention, but do not constitute a limitation to the embodiments of the present invention. In the drawings:

[0037] Figure 1 is a flowchart of a sample preprocessing method based on power multi-service knowledge classes according to an embodiment of the present invention;

[0038] Figure 2 is a flowchart of a method for data cleaning of power multi-service data samples in a sample preprocessing method based on power multi-service knowledge classes according to an embodiment of the present invention;

[0039] Figure 3 is a flowchart of a method for classifying and labeling the cleaned data according to power multi-service knowledge classes to generate structured data in a sample preprocessing method based on power multi-service knowledge classes according to an embodiment of the present invention;

[0040] Figure 4 is a flowchart of a method for feature extracting structured data to extract key features related to power services in a sample preprocessing method based on power multi-service knowledge classes according to an embodiment of the present invention;

[0041] Figure 5 It is a flowchart of a method for generating preprocessed sample data by standardizing the extracted key features in the sample preprocessing method based on power multi-service knowledge categories according to an embodiment of the present invention;

[0042] Figure 6 and Figure 7 It is a supplementary flowchart in the sample preprocessing method based on power multi-service knowledge categories according to an embodiment of the present invention. Specific Embodiments

[0043] The following will describe in detail the specific embodiments of the embodiments of the present invention with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are only for explaining and illustrating the embodiments of the present invention, and are not used to limit the embodiments of the present invention.

[0044] It should be noted that the acquisition, transmission, storage, use, processing, etc. of data in the technical solution of this application all comply with the relevant regulations of national laws and regulations. In the embodiments of this application, some industry-existing solutions such as certain software, components, models, etc. may be mentioned, and they should be regarded as exemplary. Their purpose is only to illustrate the feasibility in the implementation of the technical solution of this application, but it does not mean that the applicant has already or necessarily used this solution.

[0045] As Figure 1 shown is a flowchart of a sample preprocessing method based on power multi-service knowledge categories according to an embodiment of the present invention. In this Figure 1 sample preprocessing method may include the following steps:

[0046] In step S10, power multi-service data samples are acquired, and the power multi-service data samples include data on power generation, power transmission, power distribution, power sales, and power system operation and management;

[0047] In step S11, the power multi-service data samples are subjected to data cleaning;

[0048] In step S12, the cleaned data is classified and labeled according to power multi-service knowledge categories to generate structured data;

[0049] In step S13, feature extraction is performed on the structured data to extract key features related to power services;

[0050] In step S14, the extracted key features are standardized to generate preprocessed sample data.

[0051] In this Figure 1In the method shown, step S11 is used to clean the power multi-service data samples. In this embodiment, the method for cleaning the power multi-service data samples can be in various forms known to those skilled in the art. In a preferred example of the present invention, the method for cleaning the power multi-service data samples may include as Figure 2 shown in the steps. In this Figure 2 , the method for cleaning the power multi-service data samples may include the following steps:

[0052] In step S110, the interpolation method is used to fill in the missing values;

[0053] In step S111, the outlier values are detected and corrected through statistical methods;

[0054] In step S112, the duplicate data is deleted.

[0055] In this Figure 1 shown in the method, step S12 is used to classify and label the cleaned data according to the power multi-service knowledge categories to generate structured data. In this embodiment, the method for classifying and labeling the cleaned data according to the power multi-service knowledge categories to generate structured data can be in various forms known to those skilled in the art. In a preferred example of the present invention, the method for classifying and labeling the cleaned data according to the power multi-service knowledge categories to generate structured data may include as Figure 3 shown in the steps. In this Figure 3 , the method for classifying and labeling the cleaned data according to the power multi-service knowledge categories to generate structured data may include the following steps:

[0056] In step S120, the data samples are divided into categories such as power generation, power transmission, power distribution, power sales, and power system operation and management according to the power multi-service knowledge categories;

[0057] In step S121, the data of each category is labeled, and the structured data is generated after processing.

[0058] In this Figure 1 shown in the method, step S13 is used to extract features from the structured data and extract the key features related to the power business. In this embodiment, the method for extracting features from the structured data and extracting the key features related to the power business can be in various forms known to those skilled in the art. In a preferred example of the present invention, the method for extracting features from the structured data and extracting the key features related to the power business may include as Figure 4 shown in the steps. In this Figure 4Among them, the method for extracting features from structured data and extracting key features related to power services may include the following steps:

[0059] In step S130, features such as power generation amount, power generation efficiency, and equipment status are extracted from the power generation data;

[0060] In step S131, features such as transmission loss, line load, and failure rate are extracted from the transmission data;

[0061] In step S132, features such as distribution network topology, load distribution, and equipment health status are extracted from the distribution data;

[0062] In step S133, features such as power sales volume, electricity price, and user type are extracted from the power sales data;

[0063] In step S134, features such as system stability, dispatching instructions, and operation logs are extracted from the power system operation and management data.

[0064] In the Figure 1 method shown, step S14 is used to perform standardization processing on the extracted key features to generate preprocessed sample data. In this embodiment, for the method of performing standardization processing on the extracted key features to generate preprocessed sample data, it can be various forms known to those skilled in the art. In a preferred example of the present invention, the method of performing standardization processing on the extracted key features to generate preprocessed sample data may include as Figure 5 shown in the steps. In the Figure 5 method, the method of performing standardization processing on the extracted key features to generate preprocessed sample data may include the following steps:

[0065] In step S140, the extracted key features are normalized so that they fall within a unified range;

[0066] In step S141, the normalized features are encoded to generate data suitable for the input format of the machine learning model.

[0067] In order to persist the preprocessed sample data, in an embodiment of the present invention, this sample preprocessing method may further include as Figure 6 shown in the steps. In the Figure 6 method, this sample preprocessing method may further include the following steps:

[0068] In step S15, the quality of the preprocessed sample data is evaluated;

[0069] In step S16, the sample data meeting the quality standards is saved.

[0070] To ensure the timeliness and accuracy of sample data, in one embodiment of the present invention, the sample preprocessing method may further include as Figure 7 shown in the steps. In this Figure 7 , the sample preprocessing method may further include the following steps:

[0071] In step S17, the sample data is dynamically updated according to the power multi-service knowledge class;

[0072] In step S18, the preprocessed sample data is trained and verified using a machine learning model to optimize the preprocessing process.

[0073] On the other hand, the present invention also provides a sample preprocessing system based on the power multi-service knowledge class. The sample preprocessing system includes a controller, and the controller is used to execute the sample preprocessing method described in any of the above.

[0074] Through the above technical solutions, the sample preprocessing method and system based on the power multi-service knowledge class provided by the present invention realize the comprehensive preprocessing of power industry data by selecting knowledge in multiple fields such as power generation, power transmission, power distribution, power sales, and power system operation and management as initial data samples. By introducing the power multi-service knowledge class, the sample data is classified, labeled, and feature extracted, significantly improving the structuring and usability of the sample data. Combining the characteristics of the power industry, a targeted data cleaning, feature extraction, and standardization processing process is designed, solving the deficiencies existing in the application of existing methods in the power industry. Through the dynamic update and quality assessment mechanism, the timeliness and accuracy of the sample data are ensured, providing reliable data support for the intelligent application of the power industry, thereby improving the quality and applicability of the sample data.

[0075] Those skilled in the art should understand that the embodiments of the present application can be provided as a method, a system, or a computer program product. Therefore, the present application can adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0076] This application is described with reference to the flowcharts and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the present application. It should be understood that each flow and / or block in the flowchart and / or block diagram can be implemented by computer program instructions, as well as the combination of flows and / or blocks in the flowchart and / or block diagram. These computer program instructions can be provided to the processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing device to generate a machine, such that the instructions executed by the processor of the computer or other programmable data processing device produce means for implementing the specified functions in one or more flows Figure 1 one or more flows and / or blocks Figure 1 or means for implementing the specified functions in one or more blocks.

[0077] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, such that the instructions stored in the computer-readable memory produce a manufacture including instruction means that implement the specified functions in one or more flows Figure 1 one or more flows and / or blocks Figure 1 or means for implementing the specified functions in one or more blocks.

[0078] These computer program instructions can also be loaded onto a computer or other programmable data processing device, such that a series of operation steps are executed on the computer or other programmable device to generate a computer-implemented process, and thus the instructions executed on the computer or other programmable device provide steps for implementing the specified functions in one or more flows Figure 1 one or more flows and / or blocks Figure 1 or means for implementing the specified functions in one or more blocks.

[0079] In a typical configuration, a computing device includes one or more processors (CPUs), an input / output interface, a network interface, and memory.

[0080] The memory may include non-permanent memory in the form of computer-readable media, random access memory (RAM), and / or non-volatile memory, such as read-only memory (ROM) or flash memory (flash RAM). The memory is an example of computer-readable media.

[0081] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0082] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0083] The above are only embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included within the scope of the claims of the present application.

Claims

1. A sample preprocessing method based on electric power multi-business knowledge class, characterized in that: The sample preprocessing method comprises: Acquire a multi-service data sample of electric power, wherein the multi-service data sample of electric power includes data of power generation, transmission, distribution, sale of electric power, and operation and management of electric power system; Performing data cleaning on the electric power multi-service data sample; Classify and annotate the cleaned data according to the multi-business knowledge categories of electricity to generate structured data; Performing feature extraction on the structured data to extract key features related to the power business; The extracted key features are standardized to generate preprocessed sample data.

2. The sample pretreatment method according to claim 1, characterized in that: The power multi-service data sample is cleaned, including: Fill missing values ​​using interpolation; Detect and correct outliers through statistical methods; Remove duplicate data.

3. The sample preprocessing method according to claim 1, characterized in that: The cleaned data is classified and labeled according to the multi-business knowledge of electricity to generate structured data, including: According to the multi-business knowledge of electricity, the data samples are divided into categories such as power generation, transmission, distribution, sales, and power system operation and management; The data of each category is labeled and processed to generate the structured data.

4. The sample preprocessing method according to claim 1, characterized in that: Feature extraction is performed on the structured data to extract key features related to the power business, including: Extract features such as power generation, power generation efficiency, and equipment status from power generation data; Extracting features such as transmission loss, line load, and failure rate from transmission data; Extract distribution network topology, load distribution, equipment health status and other features from distribution data; Extracting features such as electricity sales volume, electricity price, and user type from electricity sales data; Extract features such as system stability, dispatch instructions, and operation logs from power system operation and management data.

5. The sample preprocessing method according to claim 1, characterized in that: The extracted key features are standardized to generate preprocessed sample data, including: Normalize the extracted key features to make them fall into a unified range; Encode the normalized features to generate data in a format suitable for machine learning model input.

6. The sample preprocessing method according to claim 1, characterized in that: The sample preprocessing method further comprises: Conduct quality assessment on preprocessed sample data; Save sample data that meets quality standards.

7. The sample preprocessing method according to claim 1, characterized in that: The sample preprocessing method further comprises: Dynamically update sample data according to the multi-business knowledge category of electricity; Use machine learning models to train and validate preprocessed sample data and optimize the preprocessing process.

8. A sample preprocessing system based on electric power multi-business knowledge class, characterized in that: The sample preprocessing system comprises a controller, and the controller is used to execute the sample preprocessing method according to any one of claims 1 to 7.

Images