Electric power engineering time sequence data quality inspection method and system and medium

Abstract

The invention discloses an electric power engineering time sequence data quality inspection method and system and a medium. According to the method, a complete quality guarantee system from edge side source filtering, multi-channel parallel shunting, platform side four-class deep inspection and quality whole-course tracing to theme layer service verification is constructed. Full-chain quality guarantee from source filtering to deep detection to service verification is realized through a three-layer collaborative architecture, packaging processing and deep detection responsibilities are separated, and edge-side lightweight preprocessing shares central pressure and reduces network flow. Through a remote signaling single thread, a telemetering 16 thread and a remote pulse ordered queue, platform side multi-channel parallel data processing is completed, and data logic consistency and service index accuracy are ensured in combination with theme layer service verification. Through a double-layer storage and repair marking system of an original data layer and a standard data layer, tracing and repair process auditing of quality problems are supported, and credible guarantee is provided for data governance.

Description

A method, system, and medium for quality inspection of time-series data in power engineering. Technical Field

[0001] This invention relates to the field of power data detection technology, specifically to a method, system, and medium for checking the quality of time-series data in power engineering. Background Technology

[0002] With the advancement of the global energy transition, new energy power plants such as photovoltaic and wind power generation have been widely used worldwide, becoming an important component of renewable energy. The scale of power engineering equipment is constantly expanding, and the level of intelligence is continuously improving, resulting in an exponential increase in the amount of data generated. Taking a 110MW photovoltaic power plant as an example, approximately 57 million real-time data points are generated daily, covering operational monitoring data from various equipment such as inverters (883 string inverters and 113 centralized inverters), combiner boxes (approximately 721), transformer substations (97), and data acquisition devices (37). This data is characterized by high frequency, heterogeneity, dispersion, and massive volume, posing significant challenges to data collection, storage, processing, and analysis. The quality of power plant data directly affects key business operations such as the accuracy of power generation statistics, the timeliness of equipment fault diagnosis, and the accuracy of power prediction; therefore, data quality management has become increasingly important.

[0003] Traditional methods for checking the quality of power engineering data mainly rely on manual inspections and single-device monitoring, which have the following prominent drawbacks: 1. Inaccurate time stamping leads to inaccurate time series analysis: Power plant data acquisition involves multiple protocols such as IEC104 and Modbus. Data is collected from the I-zone safety control area through isolation devices to the management area. Due to inaccurate timekeeping of the acquisition devices, the data time stamping is incorrect, resulting in a 3-5% deviation in daily power generation statistics and an error of ±15 minutes in equipment failure time recording. Existing technologies typically only perform simple time range verification at the data receiving end, lacking a system-level time synchronization mechanism with GPS / BeiDou clocks, and thus cannot fundamentally solve the problem of time stamp accuracy.

[0004] 2. Data fluctuations leading to statistical analysis distortion: Equipment maintenance or data acquisition configuration errors can cause sudden data changes, resulting in unreasonable fluctuation values. Existing technologies mainly rely on simple threshold judgments, which cannot distinguish between normal power fluctuations and abnormal fluctuations, nor can they establish dynamic change ranges based on the electrical characteristics of the equipment. They also lack intelligent repair mechanisms based on associated measurement points, making them inconvenient to use.

[0005] 3. Incomplete time series due to missing data: Network interruptions can lead to data loss, creating time series breakpoints and affecting business applications such as trend analysis, dispersion calculation, and efficiency analysis. Existing technologies have limited methods for handling missing data, often using simple linear interpolation or mean filling, without considering the differences in physical characteristics of different measurement points. This results in significant discrepancies between the completed data and the actual operating status, leading to poor performance.

[0006] 4. Dead value issues lead to fault warning failure: Abnormal acquisition channels can cause measurement point data to not refresh for extended periods, resulting in dead value data that loses its accuracy. If used for equipment status assessment, this can lead to serious misjudgments. Existing technologies lack a dead value detection mechanism based on a sliding time window, making it impossible to set differentiated judgment parameters according to the physical characteristics of the measurement points. Consequently, dead value detection is not timely, and the response speed is slow.

[0007] 5. Lack of a systematic quality inspection architecture: Existing technologies concentrate all quality inspection rules on a single level, failing to fully utilize the advantages of edge computing resources and layered processing. This makes it difficult to guarantee second-level processing performance in the face of massive scenarios, lacks a data quality traceability mechanism throughout the entire process, and results in low productivity and low pass rate. Summary of the Invention

[0008] The present invention provides a method, device, and storage medium for checking the quality of time-series data in power engineering, which can at least solve one of the technical problems in the background art.

[0009] To achieve the above objectives, the present invention adopts the following technical solution: a method for quality inspection of time-series power engineering data, comprising the following steps: S100, collecting power engineering data and performing edge preprocessing; S200, performing multi-channel splitting of the edge-preprocessed data according to the data type; S300, performing four types of quality inspection on the platform side of the multi-channel split data; S400, performing layered storage of the inspected data and the original circuit engineering data to construct a complete traceability chain; S500, performing main-layer business verification on the inspected data to complete the data quality inspection.

[0010] Among them, the main layer business verification method in step S500 includes: physical consistency verification, dispersion rate rationality verification, and equipment status association verification of the cleaned standard data; defining in-depth data logic problems; and determining data adaptation to business applications.

[0011] Further, the power engineering data edge preprocessing method in step S100 of the present invention includes: performing data type conversion, data ratio calculation, and preliminary quality marking on the original data at the data acquisition driver end, filtering out obviously abnormal data at the source; wherein, the data ratio calculation method includes: performing a linear transformation on the original acquired value to obtain the actual physical value according to the base and multiple parameters configured in the object model attribute, and the ratio calculation formula is: transformed value = (original value - base) × multiple; the preliminary quality marking method includes: performing a preliminary validity judgment on the data after ratio calculation according to the maximum, minimum, and upper limit threshold range preset in the object model attribute, and marking the data quality with a status field; the quality marking rules are: data within the range of [minimum value, maximum value] is marked as normal, i.e., status word = 1; data less than the minimum value or greater than the maximum value is marked as abnormal, i.e., status word = 0, and will not be transmitted upwards; data greater than the maximum value but less than the upper limit value will be corrected to the upper limit value and marked as corrected, i.e., status word = 2.

[0012] Furthermore, the multi-channel splitting method in step S200 of the present invention includes: allocating the data stream to three types of channels—remote signaling channel, telemetry channel, and remote pulse channel—according to the data type. The telemetry channel adopts multi-threaded parallel processing and dynamically switches the batch processing mode by monitoring the queue backlog in real time.

[0013] Furthermore, in step S300 of the present invention, the four types of quality detection on the platform side include: timescale inspection, jump inspection, missing value inspection, and dead value inspection. Among them, the timescale inspection establishes a standard time reference through GPS time synchronization and determines the validity of the timescale by an adaptive time window; the jump inspection adopts a dual judgment mechanism of preset condition judgment + interval growth judgment and executes a classification repair strategy; the missing value inspection selects differentiated completion linear interpolation, historical mean, and correlation derivation strategies according to the characteristics of measurement points with stable trends, periodic fluctuations, and strong correlations; the dead value inspection adopts a sliding window mechanism and a dual threshold judgment combining the absolute threshold of the range and the percentage threshold of the standard deviation to detect in real time and trigger hierarchical alarms.

[0014] On another front, a power engineering time-series data quality inspection system, used to execute the method described in any one of claims 1 to 4, is characterized by comprising: an edge-side preprocessing module, a multi-channel parallel processing module, a platform-side quality detection engine module, a quality traceability module, and a main-layer business verification module; the edge-side preprocessing module filters the collected initial power engineering data, the multi-channel parallel processing module classifies the filtered data, the platform-side quality detection engine module performs quality detection on the classified data, the quality traceability module constructs a complete traceability chain for the quality-detected data, and simultaneously transmits the data to the main-layer business verification module for business verification.

[0015] Furthermore, the edge preprocessing module of the present invention includes: a data type conversion unit, a data ratio calculation unit, and a preliminary quality marking unit; the data type conversion unit converts the float type output by the device into the double type, the data ratio calculation unit calculates the physical value according to the base and multiple output by the data type conversion unit, and the preliminary quality marking unit preliminarily judges the validity of the data output by the data ratio calculation unit according to the preset maximum and minimum value range.

[0016] Furthermore, the platform-side quality inspection engine module of the present invention includes four parallel quality inspection units: a time stamp inspection unit, a jump inspection unit, a missing value inspection unit, and a dead value inspection unit.

[0017] Furthermore, the quality traceability module of the present invention includes: a hierarchical storage unit, a quality identification unit, a repair method recording unit, and a traceability query interface; the hierarchical storage unit, the quality identification unit, and the repair method recording unit are connected to the traceability query interface in parallel; the hierarchical storage unit adopts a two-layer structure for data storage: the original data layer, i.e., the ODS layer, retains the collected original data including the original timestamp SOURCE_TIME, the original value SOURCE_VALUE, and the original quality code SOURCE_QUALITY; the standard data layer, i.e., the DWD layer, stores the data after quality inspection and repair, including the corrected timestamp STANDARD_TIME, the corrected value STANDARD_VALUE, the corrected quality code STANDARD_QUALITY, and the repair method recording unit. The method is marked with REPAIR_TYPE; the quality identification unit adds a quality identification field QUALITY_FLAG to each data entry, including four types of identification: VALID (valid), INVALID (invalid), SUSPECT (suspicious), and REPAIRED (repair). Business applications can selectively use data based on the quality identification. The repair method recording unit records detailed repair method codes for all automatically repaired data, including time stamp correction, jump derivation, missing value interpolation, and dead value alarms. Each repair method also records metadata such as execution time, rule version number, and confidence score. The traceability query interface supports multi-dimensional queries by test point, time period, quality problem type, and repair method, and outputs a complete traceability chain containing original data, repaired data, quality identification, repair method, and repair basis.

[0018] In another aspect, the present invention also discloses a computer-readable storage medium storing a computer program, which, when executed by a processor, causes the processor to perform the steps of the method described above.

[0019] As can be seen from the above technical solutions, this solution establishes a complete quality assurance system from edge-side source filtering, multi-channel parallel traffic splitting, four types of deep inspection on the platform side, full-process quality traceability, to topic-level business verification. Through a three-layer collaborative architecture, it achieves full-chain quality assurance from source filtering to deep inspection and then to business verification, improving processing efficiency by 40% and achieving a detection coverage rate of 98%. By separating the responsibilities of packaging processing from deep inspection, lightweight preprocessing on the edge side reduces network traffic by 30% by allocating pressure to the central processing center. The platform side's multi-channel parallel processing (single-threaded remote signaling, 16-threaded telemetry, and ordered remote pulse queue) achieves a processing capacity of 1200 records / second. Topic-level business verification ensures data logical consistency and achieves a business indicator accuracy rate of 97%. The quality traceability system supports the root cause tracing of quality issues and full auditing of the repair process through dual-layer storage and repair marking, providing reliable assurance for data governance. Attached Figure Description

[0020] Figure 1 is a flowchart of the timing data quality check for circuit engineering according to the present invention; Figure 2 is a flowchart of the edge-side preprocessing module according to the present invention; Figure 3 is a flowchart of the multi-channel parallel processing module according to the present invention; Figure 4 is a flowchart of the time scale check according to the present invention; Figure 5 is a flowchart of the jump check according to the present invention; Figure 6 is a flowchart of the missing value check according to the present invention; Figure 7 is a flowchart of the dead value check according to the present invention; Figure 8 is a flowchart of the quality traceability module according to the present invention; Figure 9 is a flowchart of the topic layer business verification module according to the present invention. Detailed Implementation

[0021] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are some embodiments of the present invention, but not all embodiments.

[0022] As shown in Figure 1, the power engineering time-series data quality inspection method described in this embodiment includes the following steps: S100, collecting power engineering data and performing edge preprocessing; S200, performing multi-channel splitting on the edge-preprocessed data according to the data type; S300, performing four types of quality inspection on the platform side on the multi-channel split data; S400, performing layered storage on the inspected data and the original circuit engineering data to construct a complete traceability chain; S500, performing main layer business verification on the inspected data to complete the data quality inspection.

[0023] The following is a detailed explanation of each step: S100, Collect power engineering data and perform edge preprocessing; Edge preprocessing refers to performing data type conversion, data ratio calculation, and preliminary quality marking on the raw data at the data acquisition driver end, filtering out obviously abnormal data at the source, and controlling the processing delay to within 10 milliseconds.

[0024] Data acquisition-driven systems define data processing rules based on object model attributes, performing initial filtering and processing on the received raw data. This includes three core processing stages: 1. Data Type Conversion: Upon receiving float-type raw data from the device, the acquisition driver converts it to double type for storage and transmission to improve data precision and subsequent calculation accuracy. For example, the inverter's output current value is converted from float (32 bits, 6-7 significant digits) to double (64 bits, 15-16 significant digits) to ensure accurate decimal calculations. 2. Transformation Ratio Calculation: Based on the base and multiplier parameters configured in the object model attributes, the raw acquired value is linearly transformed to obtain the actual physical value. The transformation ratio calculation formula is: Transformed Value = (Original Value - Base) × Multiplier. Example application: A current transformer has a transformation ratio of 100 / 5, and the device's original output value is 3.5A. The configured base... If the base value is 0 and the multiplier is 20, the actual current value is calculated as: (3.5-0)×20=70A; For a sensor with zero offset, such as a temperature sensor outputting 0-10V corresponding to -20℃ to 80℃, with a base value of 2V and a multiplier of 10, the actual temperature when the original output is 5V is: (5-2)×10=30℃. By flexibly configuring the base and multiplier parameters, the signal characteristics of different devices can be adapted; 3. Preliminary quality marking: Based on the preset maximum, minimum, and upper limit threshold ranges of the object model attributes, the validity of the data after the transformation ratio calculation is initially judged, and the data quality is marked with the status field. The quality marking rules are: data within the range of [minimum value, maximum value] is marked as normal (status word=1), data less than the minimum value or greater than the maximum value is marked as abnormal (status word=0) and will not be transmitted upwards, and data greater than the maximum value but less than the upper limit value will be corrected to the upper limit value and marked as corrected (status word=2).

[0025] Example of data changes before and after quality marking: Original data: String current = 15.8A, Timestamp = 2024-01-15 10:30:25, Status word = Unprocessed, Configuration parameters: Minimum = 0A, Maximum = 12A, Upper limit = 13A, Marking result: String current = Invalid, Timestamp = 2024-01-15 10:30:25, Status word = 0 (Abnormal data was filtered); Original data: Inverter efficiency = 102.5%, Timestamp = 2024-01-15 10:30:26, Status word = Unprocessed, Configuration parameters: Minimum = 90%, Maximum = 98%, Upper limit = 100%, Marking result: Inverter efficiency = 100%, Timestamp = 2024-01-15 10:30:26, Status word = 2 (Out-of-limit data corrected); Original data: Substation temperature = 45.3℃, Timestamp = 2024-01-15 10:30:27, Status word = Unprocessed, Configuration parameters: Minimum value = -20℃, Maximum value = 85℃, Upper limit value = 90℃, Marking result: Substation temperature = 45.3℃, Timestamp = 2024-01-15 10:30:27, Status word = 1 (Normal data retained).

[0026] Through three-step preprocessing at the edge, approximately 8-12% of obviously abnormal data can be filtered at the source, reducing network transmission traffic and central processing pressure, while retaining decimal precision (e.g., setting to retain 2 decimal places will output 45.33℃ as 45.327℃), ensuring data standardization. S200 performs multi-channel splitting on the edge-preprocessed data according to data type; multi-channel splitting refers to allocating the data stream to three types of channels—remote signaling channel, telemetry channel, and remote pulse channel—based on data type. The telemetry channel employs multi-threaded parallel processing and dynamically switches batch processing modes based on real-time queue backlog, achieving a processing speed of over 1200 records per second.

[0027] S300 performs four types of quality checks on the platform side for multi-channel split data; the four types of quality checks on the platform side refer to the quality inspection engine executing four core quality rules in parallel: time stamp check, jump check, missing value check, and dead value check.

[0028] The timescale check establishes a standard time base through GPS synchronization and determines the validity of the timescale using an adaptive time window, achieving a 100% detection rate. The jump check employs a dual judgment mechanism (preset condition judgment + interval growth judgment) and executes a classification and repair strategy, achieving a 96% detection accuracy. The missing value check selects a differentiated completion strategy (linear interpolation, historical mean, correlation derivation) based on the characteristics of the measurement points (stable trend, cyclical fluctuation, strong correlation), achieving a 95% completion success rate. The dead value check uses a sliding window mechanism and dual threshold judgment (absolute threshold for range + percentage threshold for standard deviation) to detect in real time and trigger graded alarms, achieving a 98% detection accuracy.

[0029] The dead value detection process includes: First, manually pre-setting conditions to determine which measurement points need to be processed for dead values, setting the data range and time interval parameters for judgment. The data range includes an absolute interval (the interval setting for absolute values) and a percentage interval (the interval setting for the proportion of data change to the original value). The time interval refers to the length of time for judging whether the data has become dead. During system operation, if it is found that a certain data does not exceed the specified value range (absolute interval or percentage interval) within a specified time, it is considered that the data has become dead and needs to be cleaned. This is because an anomaly in the acquisition channel causes data changes to not be sent up in time, resulting in the upper layer finding that the data has not changed for a long time, which does not conform to the actual physical characteristics of the measurement point. If it is not handled, incorrect data will be used in the statistics, thus providing users with incorrect and distorted statistical results. After a dead value is detected, a cleaning strategy is executed. The cleaning methods include two types: the derivation method and the emptying method. The derivation method mainly analyzes and infers the normal value by analyzing data that are related to the dead value data. The emptying method is used when the data cannot be recovered, so that it is not involved in various subsequent statistics. Through this mechanism, the problem of data stopping refresh caused by the abnormality of the acquisition channel can be detected and handled in time, avoiding the pollution of statistical results by erroneous data. S400: After testing, the original data of the circuit engineering is stored in layers to build a complete traceability chain. The traceability chain construction method includes: storing the original data and the repair data in layers (the original data layer ODS is retained for 7 days and the standard data layer DWD is retained for 3 years), adding quality identifiers (VALID / INVALID / SUSPECT / REPAIRED) and repair method codes (TR timescale correction / JT jump derivation / QC missing interpolation / SZ dead value alarm) to each data, and providing a multi-dimensional traceability query interface to output the complete traceability chain.

[0030] S500 performs main-layer business verification on the detected data to complete data quality testing. The main-layer business verification refers to performing physical consistency verification (e.g., active power ≈ √3 × line voltage × line current × power factor, with an allowable deviation of ±3%), dispersion rate rationality verification (e.g., current dispersion rate of similar equipment ≤ 15%), and equipment status correlation verification (e.g., power should be 0 when the machine is stopped) on the cleaned standard data to discover deep-seated data logic problems and ensure that the data can be used for business applications.

[0031] Physical consistency issues refer to discrepancies between measured data points and physical calculation relationships. For example, the product of active power on the AC side of the inverter and the current, voltage, and power factor may not conform to the power calculation formula P≈√3×U×I×cosφ (allowing ±3% deviation), or the inverter efficiency calculation value may exceed a reasonable range (efficiency = AC power / DC power should be between 90% and 98%). These issues reflect deviations in the data acquisition or calculation process and require re-verification. Abnormal dispersion issues refer to unreasonable distribution of operating parameters for similar equipment under the same operating conditions. For example, if the current dispersion rate of each string under the same inverter exceeds 15%, it indicates that a certain string may have obstruction, a fault, or abnormal data acquisition. Excessive power dispersion under the same irradiance indicates performance discrepancies or data anomalies. Dispersion analysis can identify single-point anomalies and clustered problems. Equipment status correlation issues refer to mismatches between equipment operating status and measured data. For example, an inverter may show as shut down but the power data is not zero, or the equipment may show as running but the current and voltage data are zero. Such logical contradictions indicate errors in status or data acquisition, requiring manual intervention for troubleshooting. By verifying these three types of business logic, deep-seated quality problems can be identified before data application, preventing erroneous data from misleading operation and maintenance decisions and economic analysis, and ensuring that the data meets the accuracy requirements of business applications such as power generation statistics, equipment efficiency assessment, and fault diagnosis.

[0032] To implement the above method, this invention provides a power engineering time-series data quality inspection system. The system includes: an edge-side preprocessing module, a multi-channel parallel processing module, a platform-side quality detection engine module, a quality traceability module, and a main-layer business verification module. The edge-side preprocessing module filters the collected initial power engineering data; the multi-channel parallel processing module classifies the filtered data; the platform-side quality detection engine module performs quality detection on the classified data; the quality traceability module constructs a complete traceability chain for the quality-detected data and simultaneously transmits the data to the main-layer business verification module for business verification.

[0033] As shown in Figure 2, the edge-side preprocessing module is deployed on the data acquisition driver end and is used to perform lightweight quality processing on the acquired raw data. This module includes a data type conversion unit, a data ratio calculation unit, and a preliminary quality marking unit.

[0034] The module comprises several components: a data type conversion unit that converts the device's output float type to double type to improve accuracy; a data ratio calculation unit that calculates physical values ​​based on the configured base and multiple; and a preliminary quality marking unit that preliminarily judges data validity based on a preset maximum and minimum value range, marking data outside the range as invalid. This module is simple and fast, filtering obviously abnormal data at the source, reducing the processing pressure on the central end, and keeping processing latency within 10 milliseconds.

[0035] As shown in Figure 3, the multi-channel parallel processing module uses a message queue to achieve data splitting and parallel processing. It is optimized for scenarios with 57 million data entries per day. Based on data type characteristics, the module divides the data stream into three channels: the remote signaling channel processes state-change data, which is small in volume but less sensitive to order, and is processed serially using a single thread; the telemetry channel processes continuously running data, which is large in volume and requires high processing speed, and adopts a multi-threaded parallel processing architecture, configuring 8-16 parallel threads based on the number of server CPU cores, with each thread independently executing quality check logic; and the remote pulse channel processes cumulative data, which has strict requirements for temporal order, and uses an ordered queue channel to ensure that data from the same measurement point is processed serially in chronological order.

[0036] Dynamic batch processing is implemented in each channel, and the backlog of message queues is monitored in real time. When the queue backlog exceeds the set threshold, it automatically switches to batch processing mode, reducing database connection overhead by using batch operations and increasing the processing speed to 1200-1500 messages / second. When the queue backlog decreases to a normal level, it resumes single-message real-time processing mode to ensure real-time processing during off-peak hours.

[0037] The multiple parallel threads of the telemetry channel adopt a round-robin load balancing algorithm, which distributes data to different threads according to the hash value of the measurement point ID, ensuring that the data of the same measurement point is always processed by the same thread to avoid concurrency conflicts. The CPU utilization reaches 60% during peak periods, and the memory usage is 8GB.

[0038] The platform-side quality inspection engine module is deployed on the core processing platform of the data middleware, executing four types of core quality inspection rules and serving as the core module of the entire system. This module comprises four parallel quality inspection units: a time stamp inspection unit, a jump inspection unit, a missing value inspection unit, and a dead value inspection unit. As shown in Figure 4, the time stamp inspection unit establishes a standard time reference based on GPS / BeiDou time synchronization and determines the validity of the data time stamp through an adaptive time window mechanism. A GPS / BeiDou clock module is deployed on the data middleware server to receive the standard time signal. The system performs time synchronization every 30 seconds to ensure that the deviation between the data middleware system time and the standard time is less than 100 milliseconds. This time reference serves as the standard for all data time stamp verification.

[0039] Based on the characteristics of the data transmission link, an adaptive time scale range window is set. For photovoltaic power station application scenarios, considering data acquisition time, encapsulation and transmission time, network jitter time, and processing and storage time, the time window is set as: current system time - 5 minutes, current system time + 5 minutes.

[0040] Extract the timestamp field tdata from each received data record and calculate the time deviation. If Δt ≤ 5 minutes, it is determined to be valid data and the quality bit is marked as GOOD. If Δt > 5 minutes, it is determined to be time stamp error and the quality bit is marked as BAD_TIMESTAMP. The original time stamp, system time, deviation value, and device number are recorded. Time stamp error data is rejected from entering the database to avoid polluting the standard data layer.

[0041] When time stamp errors are detected, the system automatically triggers an alarm and pushes the alarm information to the operation and maintenance platform in real time. It records the frequency and trend of time stamp errors for the device. If the same device experiences multiple time stamp errors within a short period of time (e.g., 30 minutes), the alarm is escalated to a critical alarm. This unit simultaneously retains dual timestamp records of the original time stamp and the system reception time, providing complete information for data traceability and problem localization.

[0042] As shown in Figure 5, the jump detection unit adopts a dual judgment mechanism of interval growth judgment and preset condition judgment to perform jump detection from two dimensions: dynamic change and absolute range.

[0043] The preset condition judgment sets an absolute value range based on the physical characteristics of the measuring point, such as string current 0-12A, inverter efficiency 90-98%, voltage 380-420V, etc. Data outside this range are directly judged as absolute abnormalities without the need to calculate the rate of change.

[0044] The interval growth determination is based on a dynamic change model established according to the electrical characteristics of the equipment. Taking the branch current of the combiner box as an example, the reasonable change range of the current in adjacent acquisition cycles (e.g., 5 minutes) is calculated according to the string power characteristics and the irradiance change law. The formula is |Icurrent-Iprevious| / Iprevious≤threshold (e.g., 20%). The rate of change between the current value and the previous value is calculated in real time. If it exceeds the reasonable range, it is judged as a dynamic anomaly. After a jump is detected, a classification and repair strategy is executed: For cumulative jumps (e.g., the power is reduced to zero due to the counter being cleared), the correction value is derived by associating with the measurement point data (e.g., power, running time) to maintain data continuity. The repair success rate reaches 95%, and it is marked as REPAIRED. For random jumps that cannot be derived (e.g., incorrect acquisition address), it is marked as invalid and set to empty, and marked as INVALID to avoid polluting subsequent statistics.

[0045] This unit sets differentiated rate-of-change thresholds based on the electrical characteristics of different equipment types (combiner boxes, inverters, transformer substations), such as branch current ±20%, power ±15%, and voltage ±5%, to adapt to the dynamic characteristics of the equipment.

[0046] As shown in Figure 6, the missing data detection unit establishes a standard time series model based on the measurement point acquisition frequency to identify missing data, and selects a differentiated completion strategy based on the physical characteristics of the measurement point.

[0047] This unit first establishes a standard time series model based on the sampling frequency of the measurement points (e.g., 5 seconds, 1 minute, 5 minutes). This model defines the number of samplings and timestamp sequence that each measurement point should have within a specific time period. By traversing the timestamps of the actual received data and comparing them with the standard sequence, time breakpoints and data gaps are identified, and the batch loss caused by network interruption (no data for more than 10 consecutive minutes, accounting for 70%) and sporadic loss caused by occasional packet loss (accounting for 30%) are distinguished.

[0048] The completion strategy is selected based on the type of missing data and the characteristics of the measuring points: For measuring points with stable trends (such as temperature and humidity), linear interpolation is used, and linear fitting is performed based on data points before and after the missing period, with the error controlled within ±5%, and it is marked as QC_LINEAR; For measuring points with periodic fluctuations (such as irradiance and wind speed), the historical average value filling method is used, and the average value of the same time period in the previous 3-7 days is used to fill the missing data, with an accuracy of 92%, and it is marked as QC_HISTORY; For strongly correlated measuring points (such as active power and current and voltage), the correlation derivation method is used, and the missing value is calculated through relevant measuring point data and physical formula (P=√3×U×I×cosφ), with a consistency of 94%, and it is marked as QC_DERIVE.

[0049] The completed data is marked with repair tags to facilitate traceability and quality assessment.

[0050] As shown in Figure 7, the dead value detection unit uses a sliding time window mechanism to monitor the change range of measurement point data in real time, and combines dual threshold judgment to detect dead values ​​and trigger hierarchical alarms.

[0051] During the parameter configuration phase, two types of judgment parameters are pre-set based on the physical characteristics of the measuring point: one is the time window length, which is determined according to the change cycle of the measuring point, such as a 30-minute window for slow temperature changes and a 15-minute window for rapid power changes; the other is the change threshold, including absolute thresholds (such as temperature ±0.5℃, irradiance ±10W / m²) and percentage thresholds (such as branch current change < 2% of rated value, power change < 3% of rated value).

[0052] During the real-time detection phase, a sliding window mechanism is adopted. For each new data received, the system backtracks backward from the current time to the set time window, extracts all data points of the measurement point within this window from the time series database, and calculates the range (maximum value - minimum value) and standard deviation of the data sequence. If the range is less than the set absolute threshold or the ratio of the standard deviation to the mean is less than the percentage threshold, it is determined that the measurement point has a dead value within this time window.

[0053] Upon detecting a dead value, the system immediately triggers an alarm. Based on the alarm level (a dead value lasting 1 hour is a general alarm, and a dead value lasting 3 hours is a serious alarm), the alarm is pushed to the maintenance personnel's mobile APP and the monitoring screen. The alarm information includes the measurement point name, the duration of the dead value, the current constant value, and the data statistics in the window. At the same time, the data during the dead value period is marked as invalid (quality code is set to 0) and isolated from business applications such as equipment efficiency calculation and power generation statistics. A dead value tracking record table is established to record the time of occurrence, recovery time, and scope of impact of the dead value.

[0054] Compared to fixed-time point checks, this unit reduces the time to detect dead values ​​from an average of 1-2 hours to less than 30 seconds.

[0055] As shown in Figure 8, the quality traceability module realizes full-process traceability of data quality, including a hierarchical storage unit, a quality identification unit, a repair method record unit, and a traceability query interface.

[0056] The hierarchical storage unit, quality identification unit, and repair method record unit operate in parallel and are connected to the traceability query interface. The hierarchical storage unit employs a two-layer data storage structure: the raw data layer (ODS layer) retains the original collected data, including the original timestamp SOURCE_TIME, original value SOURCE_VALUE, and original quality code SOURCE_QUALITY. This layer's data is unmodified and serves as original evidence for tracing quality issues. The standard data layer (DWD layer) stores data after quality inspection and repair, including the corrected timestamp STANDARD_TIME, corrected value STANDARD_VALUE, corrected quality code STANDARD_QUALITY, and repair method marker REPAIR_TYPE. This layer's data is used for business applications and statistical analysis.

[0057] The quality identification unit adds a quality identification field QUALITY_FLAG to each piece of data, which includes four types of identification: VALID (passes all quality check rules), INVALID (quality problem detected and cannot be automatically repaired), SUSPECT (an anomaly detected but it is uncertain whether it is a real anomaly), and REPAIRED (quality problem detected and has been repaired by automatic rules). Business applications can selectively use data based on the quality identification.

[0058] The repair method recording unit records detailed repair method codes for all automatically repaired data, including time scale correction (coded TR-TimeRepair, recording the original time scale and the corrected time scale), jump derivation (coded JT-JumpTreat, recording the original value, the derived value, and the associated measurement points on which the derivation is based), missing interpolation (coded QC-QualityComplete, recording the completion method such as LINEAR / HISTORY / DERIVE and the reference data point), and dead value alarm (coded SZ-StagnantZero, recording the dead value duration and window statistics parameters). Each repair method also records metadata such as execution time, rule version number, and confidence score.

[0059] The traceability query interface supports multi-dimensional queries by measurement point, time period, quality problem type, repair method, etc., and outputs a complete traceability chain including original data, repair data, quality identification, repair method, and repair basis.

[0060] As shown in Figure 9, the topic layer business verification module performs business logic verification on the standard data after it has been cleaned by the platform side, and discovers deep-seated data logic problems. This module includes a physical consistency verification unit, a dispersion rate rationality verification unit, and a device status association verification unit.

[0061] The physical consistency verification unit verifies the physical laws such as the calculation relationship between active power and current, voltage and power factor (active power ≈ √3 × line voltage × line current × power factor, with an allowable deviation of ±3%), and the reasonable range of inverter efficiency (efficiency = AC power / DC power, range 90%-98%).

[0062] The dispersion rate rationality verification unit verifies whether the current dispersion rate of each string under the same inverter is within a reasonable range (it should be <15%; if it exceeds this range, an abnormality in a certain string is suspected), and the dispersion rate of operating parameters of similar equipment, etc.

[0063] The device status correlation verification unit verifies the logical consistency between the device status and the operating data. For example, when the inverter is in shutdown status, the power should be 0; otherwise, it is determined to be an abnormal status or data.

[0064] This layer of verification is geared towards specific business scenarios, ensuring that data can be used in business applications through business logic consistency checks.

[0065] This solution establishes a standard time reference for GPS / BeiDou time synchronization and an adaptive time window verification mechanism to ensure time accuracy at the system level. The time stamp error detection rate reaches 100%, compared with the existing technology's detection rate of 55% and false alarm rate of 12%. It achieves full and accurate identification with zero false alarms, solving the problem of inaccurate time series analysis caused by time stamp errors.

[0066] By employing a dual judgment mechanism (interval growth judgment + preset condition judgment) and a classification and repair strategy, the system accurately identifies and processes jump data, achieving a detection accuracy rate of 96%, reducing the false alarm rate from 25% in existing technologies to 4%, and achieving a repair success rate of 95%. This effectively ensures the continuity and accuracy of power generation statistics and solves the problem of statistical analysis distortion caused by data jumps.

[0067] By classifying measurement point characteristics (stable trend, cyclical fluctuation, and strong correlation) and using differentiated completion strategies (linear interpolation, historical mean, and correlation derivation), intelligent completion is achieved, with a 100% missing data detection rate, a 95% completion success rate, and a 30% improvement in data completion accuracy compared to existing technologies, thus solving the problem of incomplete time series data caused by missing data.

[0068] By using real-time monitoring with a sliding time window and dual threshold determination (absolute threshold for range + percentage threshold for standard deviation), dead values ​​can be quickly detected and alarmed, with a detection accuracy of 98%. The false alarm rate has been reduced from 18% in existing technologies to 2%, and the detection time has been shortened from an average of 2 hours to within 30 seconds, thus solving the problem of fault warning failure caused by dead values.

[0069] By establishing a three-layer collaborative architecture of edge side-platform side-topic layer and a multi-channel parallel processing mechanism, real-time processing of 57 million massive data entries per day is achieved, with a processing speed of 1200 entries / second. This represents a 40% improvement over the centralized processing of existing technologies, meeting the requirements for second-level processing performance and solving the problem of lacking a systematic quality inspection architecture.

[0070] Through a dual-layer storage and repair tagging system consisting of a raw data layer and a standard data layer, it supports the tracing of the root causes of quality problems and the auditing of the repair process, providing a reliable guarantee for data governance.

[0071] In summary, this technical solution establishes a complete quality assurance system, encompassing edge-side source filtering, multi-channel parallel traffic splitting, four types of deep inspection on the platform side, end-to-end quality traceability, and topic-level business verification. Through a three-layer collaborative architecture, it achieves end-to-end quality assurance from source filtering to deep inspection and business verification, improving processing efficiency by 40% and achieving 98% detection coverage. Separating packaging processing from deep inspection responsibilities, lightweight preprocessing on the edge side alleviates the pressure on the central processing center, reducing network traffic by 30%. The platform side's multi-channel parallel processing (single-threaded remote signaling, 16-threaded telemetry, and ordered remote pulse queue) achieves a processing capacity of 1200 records per second. Topic-level business verification ensures data logical consistency and achieves 97% accuracy in business metrics. The quality traceability system, through dual-layer storage and repair marking, supports the root cause tracing of quality issues and end-to-end auditing of the repair process, providing reliable assurance for data governance.

[0072] In another aspect, the present invention also discloses a computer-readable storage medium storing a computer program, which, when executed by a processor, causes the processor to perform the steps of the method described above.

[0073] In another aspect, the present invention also discloses a computer device, including a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor performs the steps of the method described above.

[0074] In another embodiment provided in this application, a computer program product containing instructions is also provided, which, when run on a computer, causes the computer to execute any of the power engineering time-series data quality inspection methods described above.

[0075] It is understood that the systems, devices, and storage media provided in the embodiments of the present invention correspond to the methods provided in the embodiments of the present invention, and the explanations, examples, and beneficial effects of the relevant content can be referred to the corresponding parts of the above methods.

[0076] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially as a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid state disk (SSD)).

[0077] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0078] The various embodiments in this specification are described in a related manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the system embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0079] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for quality inspection of time-series data in power engineering, characterized in that, The process includes the following steps: S100, collecting power engineering data and performing edge preprocessing; S200, splitting the preprocessed data into multiple channels according to data type; S300, performing four types of quality checks on the split data from the platform; S400, storing the checked data and the original circuit engineering data in layers to build a complete traceability chain; S500, performing main-layer business verification on the checked data to complete data quality testing. Specifically, the main-layer business verification method in step S500 includes: performing physical consistency verification, dispersion rate rationality verification, and equipment status association verification on the cleaned standard data, defining deep-level data logic issues, and determining data compatibility with business applications.

2. The method for checking the quality of time-series data in power engineering according to claim 1, characterized in that, The power engineering data edge preprocessing method in step S100 includes: performing data type conversion, data ratio calculation, and preliminary quality marking on the raw data at the data acquisition driver end, filtering out obviously abnormal data at the source; wherein, the data ratio calculation method includes: performing a linear transformation on the raw acquired value to obtain the actual physical value according to the base and multiple parameters configured in the object model attribute, and the ratio calculation formula is: transformed value = (raw value - base) × multiple; the preliminary quality marking method includes: performing an initial validity judgment on the data after ratio calculation according to the maximum, minimum, and upper limit threshold range preset in the object model attribute, and marking the data quality with a status field; the quality marking rules are: data within the range of [minimum value, maximum value] is marked as normal, i.e., status word = 1; data less than the minimum value or greater than the maximum value is marked as abnormal, i.e., status word = 0, and will not be transmitted upwards; data greater than the maximum value but less than the upper limit value will be corrected to the upper limit value and marked as corrected, i.e., status word = 2.

3. The method for quality inspection of time-series data in power engineering according to claim 1, characterized in that, The multi-channel splitting method in step S200 includes: allocating the data stream to three types of channels, namely remote signaling channel, remote measurement channel, and remote pulse channel, according to the data type. Among them, the remote measurement channel adopts multi-threaded parallel processing and dynamically switches the batch processing mode by monitoring the queue backlog in real time.

4. The method for quality inspection of time-series data in power engineering according to claim 1, characterized in that, Step S300 involves four types of quality checks on the platform side: timescale check, jump check, missing value check, and dead value check. The timescale check establishes a standard time reference using GPS synchronization and determines timescale validity using an adaptive time window. The jump check employs a dual-judgment mechanism combining preset condition judgment and interval growth judgment, and executes a classification and repair strategy. The missing value check selects differentiated completion linear interpolation, historical mean, and correlation derivation strategies based on the characteristics of measurement points: trend-stable, periodic fluctuation, and strongly correlated. The dead value check uses a sliding window mechanism and a dual-threshold judgment combining absolute range threshold and percentage standard deviation threshold for real-time detection and triggers tiered alarms.

5. A power engineering time-series data quality inspection system, used to perform the method as described in any one of claims 1 to 4, characterized in that, include: Edge-side preprocessing module, multi-channel parallel processing module, platform-side quality detection engine module, quality traceability module, and main layer business verification module; The edge-side preprocessing module filters the initial power engineering data collected, the multi-channel parallel processing module classifies the filtered data, the platform-side quality detection engine module performs quality detection on the classified data, and the quality traceability module builds a complete traceability chain for the data after quality detection, while transmitting the data to the main layer business verification module for business verification.

6. The power engineering time-series data quality inspection system according to claim 5, characterized in that, The edge preprocessing module includes: a data type conversion unit, a data ratio calculation unit, and a preliminary quality marking unit. The data type conversion unit converts the float type output by the device to the double type. The data ratio calculation unit calculates the physical value based on the base and multiple output by the data type conversion unit. The preliminary quality marking unit makes a preliminary judgment on the validity of the data output by the data ratio calculation unit based on the preset maximum and minimum value range.

7. The method for checking the quality of time-series data in power engineering according to claim 5, characterized in that, The platform-side quality inspection engine module includes four parallel quality inspection units: a time stamp inspection unit, a jump inspection unit, a missing value inspection unit, and a dead value inspection unit.

8. The method for checking the quality of time-series data in power engineering according to claim 5, characterized in that, The quality traceability module includes: a hierarchical storage unit, a quality identification unit, a repair method record unit, and a traceability query interface. The hierarchical storage unit, quality identification unit, and repair method record unit operate in parallel and are connected to the traceability query interface. The hierarchical storage unit employs a two-layer data storage structure: the raw data layer (ODS layer) retains the collected raw data, including the raw timestamp (SOURCE_TIME), raw numerical value (SOURCE_VALUE), and raw quality code (SOURCE_QUALITY); the standard data layer (DWD layer) stores the data after quality inspection and repair, including the corrected timestamp (STANDARD_TIME), corrected numerical value (STANDARD_VALUE), corrected quality code (STANDARD_QUALITY), and repair method marker. The REPAIR_TYPE and QUALITY_FLAG fields are added to each data entry in the quality identification unit. These fields include four types of flags: VALID (valid), INVALID (invalid), SUSPECT (suspect), and REPAIRED (repairable). Business applications can selectively use data based on these quality flags. The repair method recording unit records detailed repair method codes for all automatically repaired data, including time-stamp correction, jump derivation, missing value interpolation, and dead value alarms. Each repair method also records metadata such as execution time, rule version number, and confidence score. The traceability query interface supports multi-dimensional queries by test point, time period, quality problem type, and repair method, outputting a complete traceability chain containing the original data, repaired data, quality flags, repair methods, and repair basis.

9. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, the processor performs the method as described in any one of claims 1 to 4.

Images