Distribution network information storage method based on big data and double database
By using a storage method based on big data and dual databases, the problem of slow response and low flexibility of traditional fault diagnosis technology in large-scale distribution network systems has been solved, enabling rapid and accurate diagnosis of distribution network faults and improving the system's processing capacity and reliability.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- STATE GRID SHANGHAI MUNICIPAL ELECTRIC POWER CO
- Filing Date
- 2022-12-30
- Publication Date
- 2026-05-29
AI Technical Summary
Traditional fault diagnosis techniques suffer from problems such as slow system response and low flexibility due to large data volumes when dealing with large-scale distribution network systems. Furthermore, the trend judgment of traditional methods is inaccurate and has large errors, making it difficult to meet the needs of rapid fault diagnosis in power grids.
By adopting a storage method based on big data and dual databases, and through data preprocessing, feature extraction and model training, normal data and distribution network fault information are stored in a normal database and a distributed database respectively. Combined with data fusion, analysis and visualization, accurate judgment of distribution network fault trends can be achieved.
It improves the parallelism of CPU and I/O devices, reduces interrupt frequency, enhances the data processing capability and flexibility of power distribution network faults, and improves the reliability and efficiency of fault diagnosis.
Smart Images
Figure CN116303424B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of artificial intelligence-based algorithms for analyzing distribution network faults, particularly a distribution network information storage method based on big data and dual databases. Background Technology
[0002] As the scale of social electricity consumption expands, the construction scale of distribution network systems has also continued to expand. However, due to the influence of geographical conditions, climate change, environment and other factors, distribution network systems are now facing multiple threats of failure. In particular, the ever-increasing environmental pollution has brought great challenges to the safe operation of distribution network systems.
[0003] In recent years, fault diagnosis technology has developed rapidly, especially in response to the increasing diagnostic needs of complex systems. Ensuring stable power supply and identifying and resolving power grid faults have become urgent issues. Big data is now widely used in various fields, and its strategic significance lies not only in mastering massive data information technology but, more importantly, in how to professionally analyze and process this vast amount of information. Traditional integrated computer software tools are unable to capture, coordinate, and analyze big data within a specified timeframe. Therefore, new approaches are needed to provide more powerful decision-making, presentation, analysis, and problem-solving capabilities.
[0004] However, existing fault diagnosis rules suffer from slow system response and low flexibility due to large amounts of data and repetitive data. Summary of the Invention
[0005] The increased interference from faults in power systems leads to inaccurate trend assessments and large errors when using traditional methods for determining distribution network fault trends. To address this, a big data analytics-based method for determining distribution network fault trends is proposed. Building upon existing fault analysis and judgment techniques, this method incorporates big data analytics, employing data fusion, analysis, visualization, and state identification to determine distribution network fault trends.
[0006] To solve the above technical problems, the technical solution adopted by the present invention is as follows:
[0007] A distribution network information storage method based on big data and dual databases, the storage method includes:
[0008] S1, Obtain raw data and preprocess the raw data;
[0009] S2, extract features from the preprocessed data and build a data feature table;
[0010] S3, establish a database, which includes a normal database and a distributed database. In the database, data is divided into normal data and distribution network fault information based on a data feature table. The normal data is stored in the distributed database, and the distribution network fault information is stored in the normal database. Specifically, in the distribution network waveform data, the event ID is used as the primary key based on the data feature table.
[0011] S31 determines the fault time T1, queries the time when the first violent fluctuation occurs, and obtains it by the time when the voltage suddenly drops from a stable value to a certain threshold.
[0012] S32 determines the fault recovery time T2, which is the time when the voltage stabilizes and returns to the normal value, and the end time when the voltage returns to the stable range and Θt no longer exceeds the threshold.
[0013] S33 determines the stable time T0 before the fault, where T0 is the time Δt before time T1;
[0014] S34 determines the total station telemetry, teleindication, and meteorological data within σt before and after each time discontinuity (T0, T1, T2) and stores them in the normal database;
[0015] S35 Distribution Network Fault Information Storage: In the waveform recording data, apart from the data in step S34, the rest are general data. The Hash splitting method is used, which means that the data obtained by the waveform recorder is equally divided into each distributed database according to a set ratio.
[0016] Furthermore, step S2 specifically includes:
[0017] S21, Determine the model category and features based on the properties of the original data;
[0018] S22, Organize the features and store them as vectors required by the model;
[0019] S23. Construct a dataset and input it into the learning machine model to perform preliminary training on the model, thereby obtaining the tripping model.
[0020] Furthermore, the raw data includes historical fault data, remote signaling change alarm information, secondary remote signaling alarm information, remote signaling SOE data, telemetry sampling data, remote control operation record data, and expert diagnostic records.
[0021] Furthermore, preprocessing the raw data includes:
[0022] Clean the raw data;
[0023] The cleaned data is then integrated, and the dataset is obtained by transforming, refining, and integrating the cleaned data at the lowest level.
[0024] The integrated data is transformed, which involves standardizing the data and converting it into a predetermined format.
[0025] The data is stored in the basic database.
[0026] Furthermore, the data transformation is as follows:
[0027] 1) Deviation standardization is a linear transformation of the data, mapping the values to a set range [min, max]. The transformation formula is as follows:
[0028]
[0029] Where max is the maximum value of the sample data, min is the minimum value of the sample data, and X is the current data value;
[0030] 2) Standard deviation standardization, wherein the processed data has a mean of 0 and a standard deviation of 1.
[0031]
[0032] Where x is the mean of the original data, and k is the standard deviation of the original data;
[0033] Furthermore, S21 specifically involves: extracting feature categories, feature names, and labeled feature IDs from the original data, and determining model features based on the features of the original data;
[0034] Features were extracted from the raw data, namely, event features and remote signaling data features, from the event timetable, the SOE data table, and the remote control operation record table.
[0035] Furthermore, extracting event features from the timetable event specifically involves extracting attribute and tag features from the timetable event, including circuit breaker type brkType, voltage level vltyId, and reclosing status chzqk, and then recording the event occurrence time evtTime.
[0036] Beneficial effects
[0037] In power systems, factors such as malfunctions or failures of protection or circuit breakers, channel transmission interference, and protection action time deviations can affect the system. The above solutions reduce the interrupt frequency of the CPU, improve the parallelism of the CPU and I / O devices, increase throughput, enhance the data processing capability of distribution network faults, and improve the flexibility and reliability of distribution network faults. Attached Figure Description
[0038] The accompanying drawings, which form part of this application, are used to provide a further understanding of this application. The illustrative embodiments of this application and their descriptions are used to explain this application and do not constitute an undue limitation of this application.
[0039] Figure 1 This is a flowchart illustrating the model construction process of the present invention;
[0040] Figure 2 This is a schematic diagram of the architecture of the model construction process of this invention;
[0041] Figure 3 This is a schematic diagram of the structure of the normal database for recording waveform data in the embodiment;
[0042] Figure 4 This is a statistical table of tripping data within one month, as shown in the example. Detailed Implementation
[0043] This application describes a power distribution network information storage method based on big data and dual databases, the storage method comprising:
[0044] S1, Obtain raw data and preprocess the raw data;
[0045] S2, extract features from the preprocessed data and build a data feature table;
[0046] S3, establish a database, which includes a normal database and a distributed database. In the database, data is divided into normal data and distribution network fault information based on a data feature table. The normal data is stored in the distributed database, and the distribution network fault information is stored in the normal database. Specifically, in the distribution network waveform data, the event ID is used as the primary key based on the data feature table.
[0047] S31 determines the fault time T1, queries the time when the first violent fluctuation occurs, and obtains it by the time when the voltage suddenly drops from a stable value to a certain threshold.
[0048] S32 determines the fault recovery time T2, which is the time when the voltage stabilizes and returns to the normal value, and the end time when the voltage returns to the stable range and Θt no longer exceeds the threshold.
[0049] S33 determines the stable time T0 before the fault, where T0 is the time Δt before time T1;
[0050] S34 determines the total station telemetry, teleindication, and meteorological data within σt before and after each time discontinuity (T0, T1, T2) and stores them in the normal database;
[0051] S35 Distribution Network Fault Information Storage: In the waveform recording data, apart from the data in step S34, the rest are general data. The Hash splitting method is used, which means that the data obtained by the waveform recorder is equally divided into each distributed database according to a set ratio.
[0052] Furthermore, step S2 specifically includes:
[0053] S21, Determine the model category and features based on the properties of the original data;
[0054] S22, Organize the features and store them as vectors required by the model;
[0055] S23. Construct a dataset and input it into the learning machine model to perform preliminary training on the model, thereby obtaining the tripping model.
[0056] Furthermore, the raw data includes historical fault data, remote signaling change alarm information, secondary remote signaling alarm information, remote signaling SOE data, telemetry sampling data, remote control operation record data, and expert diagnostic records.
[0057] Furthermore, preprocessing the raw data includes:
[0058] Clean the raw data;
[0059] The cleaned data is then integrated, and the dataset is obtained by transforming, refining, and integrating the cleaned data at the lowest level.
[0060] The integrated data is transformed, which involves standardizing the data and converting it into a predetermined format.
[0061] The data is stored in the basic database.
[0062] Furthermore, the data transformation is as follows:
[0063] 1) Deviation standardization is a linear transformation of the data, mapping the values to a set range [min, max]. The transformation formula is as follows:
[0064]
[0065] Where max is the maximum value of the sample data, min is the minimum value of the sample data, and X is the current data value;
[0066] 2) Standard deviation standardization, wherein the processed data has a mean of 0 and a standard deviation of 1.
[0067]
[0068] Where x is the mean of the original data, and k is the standard deviation of the original data;
[0069] Furthermore, S21 specifically involves: extracting feature categories, feature names, and labeled feature IDs from the original data, and determining model features based on the features of the original data;
[0070] Features were extracted from the raw data, namely, event features and remote signaling data features, from the event timetable, the SOE data table, and the remote control operation record table.
[0071] Furthermore, extracting event features from the timetable event specifically involves extracting attribute and tag features from the timetable event, including circuit breaker type brkType, voltage level vltyId, and reclosing status chzqk, and then recording the event occurrence time evtTime.
[0072] Example 1
[0073] The following is combined with Figures 1 to 4 The scheme of this application is described in detail as follows:
[0074] Step 1: Obtain the selection:
[0075] 1) Obtain ledger data for each power plant and switch from PMS;
[0076] 2) Obtain trip data from SCADA for the past three years;
[0077] 3) Obtain auxiliary data related to the above tripping data from SCADA, D5000 and other systems on the other side, including remote signaling change alarm information, secondary remote signaling alarm information, remote signaling SOE data, telemetry sampling data, remote control operation record data, etc.
[0078] 4) Obtain the expert judgment results records of the above tripping data from the distribution network fault analysis system database.
[0079] Step 2: Data Preprocessing
[0080] 1) Data cleaning
[0081] Data cleaning is performed on the data obtained in step 1, which mainly involves deleting irrelevant data, duplicate data, smoothing noisy data in the original dataset, filtering out data that is not related to the mining topic, and handling missing values and outliers.
[0082] 2) Data Integration
[0083] When integrating data, real-world entities from multiple data sources often have different representations and may not match. Therefore, it's crucial to consider entity identification and attribute redundancy issues, transforming, refining, and integrating the source data at the lowest level. For example, both `sales_dt` in data source A and `sales_date` in data source B describe time, i.e., A.sales_dt = B.sales_date. Careful integration of different source data can reduce or even avoid data redundancy and inconsistencies, thereby improving the speed and quality of data mining. Redundant attributes should be analyzed first, and then deleted after detection.
[0084] 3) Data Transformation
[0085] Data is standardized by transforming it into an appropriate format. For example...
[0086] 1. Min-Max Normalization:
[0087] Min-max normalization, also known as deviation normalization, is a linear transformation of the original data, mapping the values to the range [0,1]. The transformation formula is as follows:
[0088]
[0089] Where `max` represents the maximum value of the sample data, and `min` represents the minimum value. `max - min` represents the range. Deviation standardization preserves the relationships present in the original data and is the simplest method to eliminate the influence of units and data range. The drawback of this method is that if the values are concentrated and a certain value is very large, the normalized values will be close to 0 and will be very similar. If values exceeding the current range of the attributes [min, max] are encountered in the future, it will cause system errors, requiring a redefinition of `min` and `max`.
[0090] 2. Zero-mean normalization:
[0091] Zero-mean normalization, also known as standard deviation normalization, results in data with a mean of 0 and a standard deviation of 1.
[0092]
[0093] Where x is the mean of the original data and k is the standard deviation of the original data.
[0094] By following the steps above, the quality of the data is greatly improved, making it ready for subsequent use.
[0095] Step 3: Data import
[0096] 1) First, create the project, then perform thorough checks before data is imported into the database, such as ensuring data uniqueness, data type, and checking for missing items. Once you are sure there are no issues, save the data to the database.
[0097] 2) After the data enters the acquisition database, it must undergo a post-entry system check. If it is spatial data, it must undergo a topology check, and then a byte-by-byte comparison check with the original data file. After all checks pass, it will proceed to manual inspection.
[0098] 3) Manual inspection and verification: Data on distribution network fault overviews, spatial element data (graphics and attributes), text data, map data, and object data that can be made into maps should be manually inspected and verified. The inspection method is manual comparison. The work content of manual inspection and verification is the same. The system requires that manual inspection and verification be performed by different personnel to strengthen data inspection and minimize errors caused by human factors.
[0099] 4) After checking and verifying, store the normal data and fault data into the normal database and the distributed database respectively for later use.
[0100] Step 4: Data Analysis
[0101] From the dataset obtained above, usable data tables include the event table, the SOE (Simultaneous Signaling and Communication Equipment) data table (soe), and the acRec (Remote Control Operation Record) table. We will now extract "event features" and "remote signaling data features" from these tables. Following this approach, the following features can be extracted:
[0102] 1) Event characteristics
[0103] First, extract attributes and tag features from the event table, such as faultType (fault type), exceptionId (system abnormal state), faultParam (fault parameter), etc., and then record evtTime (event occurrence time).
[0104] 2) Characteristics of remote information data
[0105] Remote signaling SOE data is one of the most valuable auxiliary data. It describes the time and location of the tripping event, as well as the protective actions of the switch when the event occurs. We can extract some key feature values for analysis, such as "total fault", "reclosing", "grounding", "remote", "local", and "near-control position", and label them as SOE1, SOE2, SOE3...SOEn respectively. At the same time, the remote signaling signal time is marked as Tsoe.
[0106] The features extracted above can be summarized as follows:
[0107]
[0108] Store the above features in Redis for use during model training and inference. During training, extract the event features and remote information data features respectively, and concatenate them into the vector required by the model.
[0109] These features are stored in Redis as a Map.
[0110] Step 5: Server
[0111] The process of storing data on the server first involves applying for and activating storage services, then creating a storage bucket, and finally uploading the data. The Gateway component library is used to ensure stable and controllable service capabilities. For example, in a power distribution network failure system, the daily throughput of fault data is enormous. To ensure the server operates normally and does not crash, the data needs to be split and processed.
[0112] Examples of normal data and fault data:
[0113] 1) Normal database logic for recording waveform data storage
[0114] In multiple data tables, the event ID is used as the primary key. A fault waveform diagram is shown below. Figure 3 As shown:
[0115] Determine the fault time T1, which is the moment when the voltage first fluctuates drastically. The method is to obtain the moment when the voltage suddenly drops from a stable value to a threshold value.
[0116] Determine the fault recovery time T2, which is the moment when the voltage stabilizes and returns to normal. The method for determining this time is the end time when the voltage recovers to the stable range and Θt no longer exceeds the threshold. Θt is generally taken as 60 seconds.
[0117] Determine the stable time T0 before the fault, where T0 is the time Δt before time T1, and Δt is generally taken as 30 seconds.
[0118] Determine the total station telemetry, teleindication, and meteorological data within σt before and after each time discontinuity (T0, T1, T2), and store them in the normal database. σt is generally taken as 5 seconds.
[0119] In summary, the most important analytical data for a primary distribution network consists of the waveform recordings from time T0 (Δt before the fault occurs) to time T2 (Θt after the fault is restored), as well as all relevant data within σt before and after the three important time points (T0, T1, T2).
[0120] All the data analyzed according to this logic will need to be stored in a regular database for later analysis.
[0121] The remaining time-series data are not critical data and will be stored in a distributed database for future use.
[0122] The logic of storing waveform data in a distributed database:
[0123] The fault waveform diagram is formed by countless points, with 5 microseconds as one point. One phase has 40,000 points, and three phases multiplied by 40,000 points multiplied by 10 equals one day's data, which is 1.2 million records.
[0124] like Figure 4 The image shows the power outage statistics for one month, averaging more than ten times a day.
[0125] Directly entering so much data into the database would put excessive pressure on the server, resulting in slow response times and a high risk of crashes. Therefore, it needs to be split and stored across multiple disks. Here, a hash-based partitioning method is used. Taking 11 databases as an example, the formula is orderId % 10.
[0126] 100000 % 10 = 0, assigned to db0.
[0127] 100001 % 10 = 1, assigned to db1. ...
[0129] 100010 % 10 = 0, assigned to db0.
[0130] 100011 % 10 = 1, assigned to db1.
[0131] Here, orderId refers to the fault data sequence number identifier.
[0132] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Those skilled in the art can make various modifications and adjustments within the technical scope disclosed in the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.
Claims
1. A power distribution network information storage method based on big data and dual databases, characterized in that: The storage method includes: S1, Obtain raw data and preprocess the raw data; S2, extract features from the preprocessed data and build a data feature table; S3, establish a database, which includes a normal database and a distributed database. In the database, data is divided into normal data and distribution network fault information based on a data feature table. The normal data is stored in the distributed database, and the distribution network fault information is stored in the normal database. Specifically, in the distribution network waveform data, the event ID is used as the primary key based on the data feature table. S31 determines the fault time T1, queries the time when the first violent fluctuation occurs, and obtains it by the time when the voltage suddenly drops from a stable value to a certain threshold. S32 determines the fault recovery time T2, which is the time when the voltage stabilizes and returns to the normal value, and the end time when the voltage returns to the stable range and Θt no longer exceeds the threshold. S33 determines the stable time T0 before the fault, where T0 is the time Δt before time T1; S34 determines the total station telemetry, teleindication, and meteorological data within σt before and after each time discontinuity (T0, T1, T2) and stores them in the normal database; S35 Distribution Network Fault Information Storage: In the waveform recording data, apart from the data in step S34, the rest are general data. The Hash splitting method is used, which means that the data obtained by the waveform recorder is equally divided into each distributed database according to a set ratio.
2. The power distribution network information storage method based on big data and dual databases as described in claim 1, characterized in that, Step S2 specifically involves: S21, Determine the model category and features based on the properties of the original data; S22, Organize the features and store them as vectors required by the model; S23. Construct a dataset and input it into the learning machine model to perform preliminary training on the model, thereby obtaining the tripping model.
3. The power distribution network information storage method based on big data and dual databases as described in claim 1, characterized in that, The raw data includes historical fault data, remote signaling displacement alarm information, secondary remote signaling alarm information, remote signaling SOE data, telemetry sampling data, remote control operation record data, and expert diagnostic records.
4. The power distribution network information storage method based on big data and dual databases as described in claim 1, characterized in that, Preprocessing the raw data includes: Clean the raw data; The cleaned data is then integrated, and the dataset is obtained by transforming, refining, and integrating the cleaned data at the lowest level. The integrated data is transformed, which involves standardizing the data and converting it into a predetermined format. The data is stored in the base database.
5. The power distribution network information storage method based on big data and dual databases as described in claim 4, characterized in that, The data transformation is as follows: 1) Deviation standardization is a linear transformation of the data, mapping the values to a set range [min, max]. The transformation formula is as follows: ； Where max is the maximum value of the sample data, min is the minimum value of the sample data, and X is the current data value; 2) Standard deviation standardization, wherein the processed data has a mean of 0 and a standard deviation of 1. ； Where x is the mean of the original data and k is the standard deviation of the original data.
6. The power distribution network information storage method based on big data and dual databases according to claim 2, characterized in that, S21 specifically involves: extracting feature categories, feature names, and labeled feature IDs from the original data, and determining model features based on the features of the original data; Features were extracted from the raw data, namely, event features and remote signaling data features, from the event timetable, the SOE data table, and the remote control operation record table.
7. The power distribution network information storage method based on big data and dual databases as described in claim 1, characterized in that, Extracting event features from the timetable event specifically involves extracting attribute and tag features from the timetable event, including circuit breaker type brkType, voltage level vltyId, and reclosing status chzqk, and then recording the event occurrence time evtTime.