A method and system for calculating industrial time-series data with instantaneous high-frequency characteristics

Abstract

This invention discloses a method and system for calculating industrial time-series data with instantaneous high-frequency characteristics, belonging to the field of real-time computing for power system dispatch automation. The method employs a cache-based out-of-order data correction technique, a streaming-based dynamic delay calculation method, and a reentrant calculation window verification method to adaptively optimize instantaneous high-frequency time-series data. This achieves high-precision, high-frequency, and high-performance industrial time-series data calculation, effectively improving the power grid's operation monitoring level and ensuring its operational safety. This invention is particularly applicable to high-frequency data calculation and analysis of key equipment such as IGBTs and fault recording devices in flexible DC substations, improving the accuracy and real-time performance of power system fault analysis and diagnosis.

Description

Technical Field

[0001] This invention belongs to the field of real-time computing for power system dispatch automation, and specifically relates to a method and system for calculating industrial time-series data with instantaneous high-frequency characteristics. Background Technology

[0002] Industrial time-series data computing refers to the technical process of cleaning, analyzing, mining, and applying structured data continuously generated in a time sequence in industrial scenarios through specific algorithms, models, or tools, ultimately extracting business value (such as equipment failure early warning, production efficiency optimization, and quality anomaly detection).

[0003] With the continuous expansion of the power grid, flexible DC transmission technology, as an important piece of equipment for building smart grids, has gradually become a key component of modern power systems. Flexible DC transmission stations and other critical facilities play a vital role in islanded power supply, urban distribution network capacity expansion and upgrading, and efficient power transmission and distribution. In flexible DC station systems, the operating status of key equipment such as IGBTs (Insulated Gate Bipolar Transistors) directly affects the stability of the entire power grid. When these devices fail, fault recorders collect critical data such as voltage and current at extremely high frequencies, down to milliseconds or even microseconds. This high-frequency data is extremely important for the stable operation of the power grid and fault analysis. By using industrial time-series data calculation methods to extract and analyze features from high-frequency fault data of flexible DC stations, strong support can be provided for accurate fault location and type identification, while also providing effective information for subsequent operation and maintenance optimization and fault prevention. Furthermore, with the promotion and application of flexible DC transmission systems, as new energy sources (such as wind power and photovoltaics) are connected and consumed on a large scale, the randomness of new energy output will cause instantaneous high-frequency fluctuations in grid power, voltage and other parameters. How to accurately capture and quantify these instantaneous high-frequency characteristics in the process of industrial time-series data calculation, and thus provide data basis for real-time control of new energy output fluctuations, is also the key to promoting better integration of new energy into the grid and improving the grid's ability to absorb new energy.

[0004] In the prior art, patent CN118779104A proposes a real-time computing method for an industrial internet platform based on device objects. This method utilizes device objects to establish a hierarchical abstract device model and a physical device model. By dragging and dropping different nodes on the process canvas to form different process tasks, users can create complex computing processes by dragging and dropping different types of operator nodes. Each node represents an operator with a specific function, and the constructed process tasks can be submitted to the Flink cluster for computation, thereby obtaining different data results and providing data support for other services on the platform. However, this technology has the following drawbacks, making it unable to meet the real-time computing requirements of power dispatch automation:

[0005] 1. This method relies on Flink's computation periodic function, achieving a minimum computational accuracy at the millisecond level. It lacks consideration for high-frequency data with sampling frequencies reaching the microsecond level. Furthermore, data generated by noise interference or network transmission errors still participates in the overall calculation as sub-items, significantly impacting computational efficiency and the accuracy of the results.

[0006] 2. This method only considers the measurement points of the device object as the calculation unit, and lacks consideration for the time alignment factor of the measurement point data. Data timestamps are an important indicator of the accuracy of high-frequency feature data calculation results. Flink-based calculations perform calculations on data in periods or windows. In this case, the time granularity is coarse for high-frequency feature data calculations. Massive amounts of timestamp data with uneven levels are used for calculations on a unified time section, and the accuracy of the calculation results needs to be improved.

[0007] 3. Existing technologies lack the ability to backtrack and locate problems in recalculation processes. Real-time computation based on Flink's streaming computing characteristics and batch processing using historical data cannot reflect the evolution of computation results. Furthermore, it is difficult to capture errors in sub-item data during the computation process, and it is impossible to verify the real-time computation results of the same value with historical computation results. This also easily leads to a waste of computing resources.

[0008] Therefore, there is an urgent need for a new method for calculating industrial time-series data that is oriented towards instantaneous high-frequency characteristics. Summary of the Invention

[0009] Purpose of the invention: The purpose of this invention is to overcome the shortcomings of the prior art and provide a method and system for calculating industrial time-series data based on instantaneous high-frequency characteristics. It constructs an architecture for processing instantaneous high-frequency characteristic data and, based on this, effectively improves the power grid's acquisition and processing of high-frequency data, enhances the power system's fault diagnosis level, and ensures the safe operation of the power grid.

[0010] Technical solution: To achieve the above-mentioned objectives, the present invention adopts the following technical solution:

[0011] Firstly, a method for calculating industrial time-series data with instantaneous high-frequency characteristics includes:

[0012] Create a memory-based data buffer to store measurement point data. All measurement point data is written in a loop according to the following method: data from the start to the archiving period is compressed and stored on disk, and data from the archiving period to the end is migrated to the memory at the start position.

[0013] Create a memory-based computation buffer to store the computational results generated over time, composed of different measurement points;

[0014] The data to be written to the data buffer is preprocessed as follows: the filtering parameters are automatically adjusted according to the real-time change characteristics and rate of change of the device data values ​​to remove noise interference and abnormal data points caused by transmission errors; for finite frame data within a local time, the data is sorted according to the timestamp of the data tag; and the missing data is filled with the corresponding fitted values ​​according to the configured strategy.

[0015] The calculation task is initiated, and the scale of the measurement point data for the calculation sub-items is estimated. The calculation task is then broken down into multiple calculation sub-tasks. For each sub-task, data is read from a memory-based measurement point data buffer, and the resulting calculation items are stored in the calculation buffer. Once the calculations are completed, they are merged into the total calculation result. The time delay characteristics between the transmission and calculation nodes and the original calculation sub-item data are analyzed, and the difference is dynamically processed based on real-time changes to ensure the consistency of data calculation timestamps. A memory-based cache pool of the latest calculation results is constructed, and heavyweight calculation units are calibrated based on the magnitude assessment results. Incremental recalculation is performed by combining the sub-item parameter status codes or calculation formula change markers.

[0016] Based on the timestamp of the received data and combined with the general data delay time, the latest data timestamp of the entire system per second is determined, and the proportion of measurement points is calculated cumulatively from newest to oldest according to the latest data timestamp. When the cumulative proportion reaches or exceeds the set threshold for the first time, a cross-section calculation alignment time stamp is generated as the core reference benchmark for dynamically adjusted calculation start time.

[0017] The system specifies a backtracking point as required, recovers the computational input data and intermediate results of the specified backtracking point from the data buffer or historical data storage, compares the real-time computational results with the historical computational results of the corresponding backtracking point, identifies the differences in computational quantum term parameters and the differences in the final computational results, marks the computational result status code according to the preset threshold, and performs recalculation in case of errors or abnormalities.

[0018] Furthermore, the data buffer employs a double-buffering technique, comprising an archiving buffer and an auxiliary buffer. The data writing process is as follows: When the timestamp of the measured data reaches the preset archiving time in the buffer, the archiving buffer performs batch lossless compression on the data within the current window, forming data blocks and writing them to the disk file. After compression, the memory space occupied by the archiving buffer is released. Uncompressed data in the auxiliary buffer, i.e., new data written after the archiving time, is moved to the beginning of the buffer. Subsequent data continues to be written to the buffer in a queue, and this process is repeated.

[0019] Furthermore, the structure of a data block includes a data block header and a data pointer to the compressed package. The data block header contains the data block type, the corresponding measurement point, the data submission frequency, the start time of the data block, and the size of the compressed block. The measurement point data consists of the data time, data value, and data status code.

[0020] Furthermore, the computation buffer stores multiple computation quantities, the result of which varies with time. Each computation quantity consists of multiple sub-items, which are individual timestamped measurement point data, including status code, data time, data value, measurement point name, and measurement point ID. In addition to the sub-items, the computation impact factors of the computation quantity also include real-time time, computation cycle, computation sub-item completeness rate, and computation method. The computation sub-item completeness rate is the proportion of measurement point data participating in the computation to the total measurement point data that should participate. According to the data structure required by the computation quantity, a computation space for the above data structure is established in memory as a computation buffer.

[0021] Furthermore, the calculation methods include real-time calculation and full ratio calculation. In real-time calculation, when the time reaches the trigger condition of the calculation cycle, the data of each calculation sub-item is directly used without considering the data timestamp to generate the corresponding calculation result. In full ratio calculation, it is determined whether the data proportion of each calculation sub-item at the current moment reaches or exceeds the set threshold. If the set threshold is not reached, the latest data timestamp of the entire system every second is used to calculate and generate the result.

[0022] Furthermore, in actual calculations, a collaborative processing method is used, based on the completeness ratio and supplemented by real-time calculation. First, a target completeness ratio is set, and a maximum waiting time is built in. During calculation, priority is given to waiting for the calculated sub-item data to reach the set completeness ratio. If the requirement is met within the maximum waiting time, the calculation is performed based on the qualified data. If the target completeness ratio is not reached after the timeout, the latest data timestamp of the entire system is used for calculation.

[0023] Furthermore, for a finite number of frames of data within a local time period, sorting is performed based on the timestamps of the data markers, including:

[0024] The timestamp of each data write is limited to half the time span of the data buffer, and the data is sorted from oldest to newest according to the timestamps it carries.

[0025] Secondly, an industrial time-series data computing system oriented towards instantaneous high-frequency characteristics includes:

[0026] The cache-based out-of-order data correction module includes a buffer construction unit and a data preprocessing unit. The buffer construction unit creates a memory-based data buffer and a computation buffer. The data buffer stores measurement point data, which is written cyclically according to the following method: data is compressed and stored on disk from the start to the archive period, and data from the archive to the end period is migrated to the memory at the start position. The computation buffer stores the computational results generated over time, which are composed of different measurement points. The data preprocessing unit is configured to perform the following preprocessing on the data to be written to the data buffer: automatically adjust the filtering parameters according to the real-time change characteristics and rate of change of the device data values ​​to remove noise interference and abnormal data points caused by transmission errors; sort the limited frame data within a local time period according to the timestamp of the data tag; and fill in the corresponding fitted values ​​for missing data according to the configured strategy.

[0027] The dynamic latency calculation module based on streaming processing includes a calculation processing unit, an incremental calculation unit, and a real-time calculation unit at startup. The calculation processing unit is configured to estimate the scale of the measurement point data for each calculation sub-item after the calculation task starts, and decompose the calculation task into multiple calculation sub-tasks. For each sub-task, data is read from a memory-based measurement point data buffer, and the resulting calculation items are stored in the calculation buffer. These sub-items are then merged into the total calculation result upon completion. The module analyzes the time delay characteristics between the transmission and calculation nodes and the original calculation sub-item data, dynamically processing the difference based on real-time changes to ensure timely data calculation. Consistency of time stamps; the incremental calculation unit is configured to build a memory-based cache pool of the latest calculation results, calibrate heavyweight calculation units based on the magnitude evaluation results, and perform incremental recalculation in combination with the status code of sub-item parameters or the change marker of calculation formula; the real-time calculation unit at the start of calculation is configured to determine the latest data timestamp of the entire system per second based on the timestamp of the received data and the general data delay time, and calculate the proportion of measurement points cumulatively from newest to oldest according to the latest data timestamp. When the cumulative proportion reaches or exceeds the set threshold for the first time, a cross-section calculation alignment time stamp is generated as the core reference benchmark for dynamically adjusting the calculation start time;

[0028] The reentrant computation window verification module includes a backtracking point selection unit and a recomputation verification unit. The backtracking point selection unit is used to specify a backtracking point as needed and recover the computation input data and intermediate results of the specified backtracking point from the data buffer or historical data storage. The recomputation verification unit is configured to compare the real-time computation results with the historical computation results of the corresponding backtracking point, find the differences in computation quantum term parameters and the differences in the final computation results, mark the computation result status code according to the preset threshold, and perform recomputation in case of error or abnormality.

[0029] Thirdly, the present invention provides an electronic device comprising: one or more processors; a memory; and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, wherein when the programs are executed by the processors, they implement the steps of the industrial time-series data calculation method for instantaneous high-frequency characteristics as described in the first aspect.

[0030] Fourthly, the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the industrial time-series data calculation method for instantaneous high-frequency characteristics as described in the first aspect.

[0031] Fifthly, the present invention provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the industrial time-series data calculation method for instantaneous high-frequency characteristics as described in the first aspect.

[0032] Compared with the prior art, the present invention has the following beneficial effects:

[0033] (1) The following methods were used to realize the calculation and analysis of time-series data with instantaneous high-frequency characteristics: First, the out-of-order data correction technology based on caching can effectively handle the out-of-order problem in data transmission, ensuring the accuracy and integrity of the data. Second, the dynamic delay calculation method based on streaming processing can process the data stream in real time, dynamically adjust the delay, and improve the calculation efficiency. In addition, the reentrant calculation window verification method can verify the calculation results multiple times to ensure the reliability of the calculation. This invention overcomes the shortcomings of existing real-time calculation methods based on Flink industrial Internet platforms, such as insufficient coverage of microsecond-level high-frequency data, interference of calculation accuracy by noise data, lack of time alignment of measurement point data, and weak backtracking and result verification capabilities, thus meeting the real-time calculation needs of power dispatch automation.

[0034] (2) It adopts a storage and computing separation architecture to process data, combining the advantages of streaming computing and batch processing computing, which can efficiently process high-frequency data and is suitable for computing key equipment in flexible DC power stations. In addition, in the promotion and application of flexible DC power transmission systems, the use of this invention can effectively solve the fluctuation problem of new energy access and consumption, better integrate new energy into the power grid, reduce dependence on diesel power generation, and reduce environmental pollution.

[0035] (3) This invention not only improves the real-time performance and accuracy of fault diagnosis, but also evolves the changes in equipment status and power flow across the entire network, providing solid data support for in-depth research into the causes of faults. This invention enhances the level of fault diagnosis in power systems and ensures the safe operation of the power grid. Attached Figure Description

[0036] Figure 1 This is a flowchart illustrating the implementation of the high-frequency time-series data calculation method disclosed in this embodiment of the invention.

[0037] Figure 2 This is a schematic diagram of the high-frequency time-series data buffer disclosed in an embodiment of the present invention;

[0038] Figure 3 This is a schematic diagram of the high-frequency time-series data disorder correction preprocessing disclosed in the embodiments of the present invention;

[0039] Figure 4 This is a schematic diagram of the dynamic delay calculation framework for high-frequency time-series data disclosed in an embodiment of the present invention;

[0040] Figure 5 This is a schematic diagram of high-frequency time-series data section time alignment disclosed in an embodiment of the present invention;

[0041] Figure 6 This is a schematic diagram of high-frequency time-series data recalculation disclosed in an embodiment of the present invention. Detailed Implementation

[0042] The technical solution of the present invention will be further described below with reference to the accompanying drawings.

[0043] In practical applications, due to differences in data acquisition and processing speeds among different devices and network transmission delays, local out-of-order data often occurs during data access. This out-of-order phenomenon not only increases the complexity of data processing but also seriously affects the accuracy of data analysis. Traditional periodic calculation methods suffer from significant inefficiency and insufficient real-time performance when processing high-frequency real-time data. The lag in periodic calculation results further reduces the accuracy of the calculation results, making fault diagnosis even more difficult.

[0044] Furthermore, cross-sectional calculations of massive amounts of data consume significant computing resources. To accurately capture subtle changes in the power system's operating status, monitor anomalies in the power grid in real time, promptly detect potential faults, and help maintenance personnel quickly and accurately locate fault points, it is essential to improve computational efficiency and conserve computing resources. This not only helps provide more reliable, timely, and accurate data for power system power flow calculations and grid security analysis but also ensures the safe and stable operation of the power grid. This invention improves the accuracy of calculations by adaptively optimizing instantaneous high-frequency time series, repairing missing, duplicate, or abnormal timestamps and data points, automatically adjusting the start time of cross-sectional data calculations, and verifying real-time and historical calculation results. This provides reliable data support for the stability and reliability of the power grid and strong support for the optimized scheduling of the power system.

[0045] This invention discloses an industrial time-series data calculation method for instantaneous high-frequency characteristics. It utilizes a cache-based out-of-order data correction technique, a streaming-based dynamic delay calculation method, and a reentrant calculation window verification method. Employing a computation-storage separation architecture, it completes high-frequency data calculation and analysis for key equipment such as IGBTs and fault recordings in flexible DC substations, improving the accuracy and real-time performance of power system fault analysis and diagnosis. (Refer to...) Figure 1 Out-of-order data correction technology based on caching includes the following steps:

[0046] Step 1: A data buffer is created using a double-buffering technique to store the measurement point data. The data buffer is divided into two functional buffers: an archive data buffer and an auxiliary buffer. Data is written cyclically, compressing and storing the data from the beginning to the archive period on disk, and migrating the data from the archive period to the end to the beginning memory location. The data block header consists of data block type, data block commit frequency, data measurement point description, and data block size.

[0047] Step 2: Create a memory-based buffer of computational results composed of different measurement points, which grows over time; calculate and store the computational results according to the start time of the calculation, and the start time of real-time calculation is based on the pre-set data stamp delay time and sub-item completeness rate; calculate the impact factor including real-time time, calculation cycle, calculation sub-item completeness rate, calculation sub-item status code, calculation result sampling cycle, and calculation method.

[0048] Step 3, preprocessing of the data to be written to the data buffer: using the traditional out-of-order correction method for power equipment, the filtering parameters are automatically adjusted according to the real-time change characteristics and rate of change of the equipment data values ​​to remove noise interference and abnormal data points caused by transmission errors; for finite frame data within a local time, the data is sorted from oldest to newest according to the timestamp of the data tag; for missing data, the corresponding fitted values ​​are filled in according to the configured strategy.

[0049] The dynamic delay calculation method based on streaming processing includes the following steps:

[0050] Step 4: Start the computation task, analyze the computational workload including the scale of measurement points for computational sub-items, read data from the memory-based measurement point data buffer using multiple computational sub-tasks, and store the generated computational workload items in the computational workload result buffer. After the computation is completed, merge them into the total computational workload result; analyze the time delay characteristics between the transmission and computation nodes and the original computational sub-item data, and dynamically process the difference based on real-time changes to ensure the consistency of data computation timestamps; construct a memory-based latest computational result cache pool, calibrate heavyweight computational units based on the magnitude evaluation results, and perform incremental recalculation in combination with sub-item parameter status codes or calculation formula change markers.

[0051] Step 5: Generate cross-section calculation alignment time scale. Based on the timestamps of the received data, using the general data delay time, and simultaneously counting the timestamps of data per second for the entire system, the percentage of measurement points is calculated cumulatively from newest to oldest time scale. When the cumulative percentage first reaches or exceeds the set threshold, the cross-section calculation alignment time scale is generated, serving as the core reference benchmark for dynamically adjusted calculation start time.

[0052] Reentrant calculation window verification includes the following steps:

[0053] Step 6: Specify the backtracking point according to the requirements, obtain historical data, compare the real-time calculation results of the same amount of computation with the historical calculation results, compare the differences in the quantum term parameters and the differences in the final results, and mark the status code of the calculation result according to the set threshold.

[0054] According to an embodiment of the present invention, the out-of-order data correction technology based on caching divides the calculation results and data storage into different memory regions in a storage-computation separation architecture, corresponding to steps 1 to 3 above. The specific implementation details are as follows.

[0055] A memory-based data buffer is created, with a specified unit time span as the data buffer window to cache real-time data. The size of the data buffer adaptively adjusts according to the scale of data written by the measurement points within a unit time. The data buffer uses a double-buffering technique, including an archive buffer and an auxiliary buffer. When the timestamp of data written by a measurement point reaches the preset archive time of the buffer: 1) The archive buffer is responsible for batch lossless compression of the data within the current window (from the start to the archive time), forming data blocks and writing them to the disk file; 2) After compression, the memory space occupied by the archive buffer is released; 3) Uncompressed data in the auxiliary buffer, i.e., new data written after the archive time, is moved to the beginning position of the buffer; 4) Subsequent data continues to be written to the buffer in a queue, and this process is repeated.

[0056] Each data point collected carries a timestamp (i.e., the time identifier when the data was written). Assuming the data processing cycle is divided into "minutes", the start time of the current minute cycle can be automatically obtained (i.e., the first second of this minute, such as xx minutes 00 seconds). The theoretical archiving reference time for each minute cycle is the end time of that cycle (such as xx minutes 59 seconds). Due to potential delays in data transmission (e.g., data from the xx minutes 00 seconds to xx minutes 59 seconds cycle will not be fully received in the buffer until (xx+1) minutes 05 seconds), to ensure data integrity, the actual time of "data compression and writing to disk" (such as (xx+1) minutes 10 seconds) is set as the official archiving time of the data buffer.

[0057] like Figure 2 As shown in the data buffer diagram, when the timestamp of cached data at a certain measurement point reaches the archiving time of the data buffer, batch lossless compression is performed on the cached data of a single measurement point from the start position to the archiving time to form a data block, which is then written to the disk data file. Next, the memory space occupied by the compressed cached data is reset, and the uncompressed data in memory is moved to the start position. The latest timestamp data is then written to the data buffer using a queue data structure, and subsequent data writing cycles through this process. The data buffer structure includes the latest data time, the number of data entries, a data validity flag, and a pointer to a data block pointing to a single data value structure. The data block structure includes a data block header and a pointer to a compressed package. The data block header contains the data block type, the measurement point corresponding to the data block, the data submission frequency, the start time corresponding to the data block, and the compressed block size. The measurement point data consists of data time, data value, and data status code.

[0058] like Figure 2As shown in the calculation buffer, the calculation buffer stores multiple computational quantities, and the calculation result of each quantity varies with time. Each computational quantity consists of multiple sub-items, i.e., individual timestamped measurement point data. In addition to using sub-items for calculation, the computational quantity also needs to consider "real-time time", "calculation cycle", "computation sub-item completeness rate (the proportion of measurement point data participating in the calculation to the total measurement point data that should participate)" and "calculation method". The calculation sub-item includes "status code", "data time", "data value", "measurement point name", and "measurement point ID (i.e., measurement point definition keyword)". According to the data structure required by the computational quantity, the above data structure is established in memory for computational space. The calculation method is divided into real-time calculation and calculation based on completeness ratio. Each computational quantity is calculated and stored in its own independent calculation cycle. The status code includes "fit", "normal", "acquisition", "not currently complete", and "manual setting" statuses, used to identify the numerical result of the computational quantity in various situations. Among the factors affecting computational load, the computation cycle will affect the accuracy of the computation; an excessively wide computation cycle will prevent the capture of changing data. The completeness rate of computational sub-items will affect the completeness of the computational load data values. If the current completeness rate is 99%, but the data delay is more than 2 seconds, it is highly likely that the computational load data calculation will occur within this delay, and at this time only 90% of the computational sub-items can participate in the calculation, thus affecting the completeness of the computational load. The status code affects the accuracy of the data values; in most cases, the status code "normal" or "collection" should be used.

[0059] The data to be written to the data buffer is preprocessed as follows: Traditional out-of-order correction methods for power equipment (based on fuzzy control algorithms) are used to automatically adjust filtering parameters according to the real-time changes and rates of change of the equipment data values. This simulates the experience and fuzzy logic of human experts, using fuzzy rules to infer and remove noise interference and abnormal data points caused by transmission errors (e.g., the range of active power values ​​for a generator at a certain voltage level; data values ​​exceeding this range are marked as "suspicious"). To reduce communication overhead and improve CPU, memory, and I / O utilization, the timestamp of each data write is within half the time span of the data buffer, i.e., the start and end timestamps do not exceed 1 / 2 * (end_time - start_time), and are sorted from oldest to newest timestamps. For missing data during transmission or values ​​filtered out according to optimization strategies, corresponding fitted values ​​are filled in according to the strategy. Optimization strategies include interpolation query, step query, and step nearest value. The current data status code is marked as "fitted," different from the "acquisition" status code for normal status codes. Figure 3As shown, in the data access stage, the data of measurement point 1 has out-of-order timestamps (such as t+2 followed by t+1). After sorting, it is normalized by time from farthest to nearest (t→t+1→t+2) and stored in the data buffer. The data of measurement point N has an outlier at time t+2. After data cleaning and treatment (discarding outliers), the fitted value at that time is generated according to the fitting strategy. Finally, the normalized and completed data (including values ​​and status codes) is stored in the data buffer according to the rules, providing orderly and complete measurement point data for subsequent processing.

[0060] According to an embodiment of the present invention, the dynamic delay calculation method based on streaming processing corresponds to steps 4 and 5 above. It obtains the computational result through dynamic delay calculation of high-frequency time-series data and determines the real-time computation start time based on data section time alignment. The specific implementation method is as follows.

[0061] High-frequency time-series data dynamic delay calculation framework, such as Figure 4 As shown, after starting the calculation task, the data volume of the calculation sub-items and measurement points is analyzed to calculate the number of data packets to be assembled. A calculation task (corresponding to a certain calculation quantity, whose calculation formula contains several calculation sub-items) can be decomposed into multiple calculation sub-tasks. For example, if the calculation formula of a certain calculation task contains 100 calculation sub-items, it can be divided into groups of 20 calculation sub-items, with each group corresponding to an independent calculation sub-task. For multiple computational subtasks, data is read from a memory-based measurement point data buffer. The data is temporarily stored in the access buffer and awaits reading into the computation buffer. Measurement points in each subtask are allocated according to a measurement point aggregation strategy (the measurement point aggregation strategy can be divided according to measurement points under the jurisdiction of the plant, measurement points under the jurisdiction of the line, operation and maintenance responsibility area, etc.). The generated computational quantities are stored in the computation buffer and merged into the total computational quantity result after the calculation is completed. The time delay characteristics between the transmission and computation nodes and the original computational sub-item data are analyzed, the difference is calculated in real time, and the calculation is performed at the moment with the best consistency of the data computation timestamp. A memory-based latest computational result cache pool is constructed. The computational unit scale is evaluated according to the number of sub-item parameters, the nesting level of sub-item parameters, and the synchronization of sub-item parameters. Based on the scale evaluation results, heavyweight computational units are calibrated (evaluated according to the number of computational measurement points, the division of computational subtasks, and the currently available computational resources). The latest computational result cache pool is used for caching. Incremental recalculation is performed in combination with sub-item parameter status codes or calculation formula change markers. Incremental recalculation is based on the quality code of the calculated sub-item. For example, if the status code of a calculated sub-item in the calculation result at time t is marked as "fitting," then after time t+n, since all delayed data has arrived, the status code of the calculated sub-item changes to "normal." During calculation, it is not necessary to recalculate all calculated sub-items in this computational quantity; only those with status codes other than "normal" need to be recalculated. The results of other calculated sub-items can be used from the results stored in memory. Figure 4As shown, the ordered and complete measurement point data in the data buffer is divided into multiple data packets (corresponding to different calculation sub-tasks task1~taskN) according to the calculation sub-items, and distributed to each sub-task in the calculation buffer for calculation. After each sub-task is completed, the results (task-val, task-stat) are merged into the final calculation result cache area, and the final output is a complete result containing the corresponding values ​​of each timestamp, status code and calculation time, realizing the split calculation and result integration of high-frequency data.

[0062] In the above description, the time delay characteristics between the transmission and computing nodes and the original computing sub-item data are analyzed. The difference is calculated in real time, and the moment with the best consistency between the data computing timestamps is selected for calculation. Specifically, this means first analyzing the delay pattern between the system time of the transmission / computing nodes and the acquisition timestamps inherent in the computing sub-item data, and calculating the time difference between the two in real time; then, based on these time differences, the moment with the smallest deviation between the acquisition timestamps of most computing sub-item data and the processing time of the transmission / computing nodes is selected, and this moment is taken as the final computing execution moment. The computing execution moment is also the moment when the computing starts, meaning that the computing start moment is dynamically adjusted. This invention uses a high-frequency time series data segment time alignment mechanism to determine this moment. High-frequency time series data segment time alignment is as follows: Figure 5 As shown, the first step is to collect the latest data timestamps for the entire system every second. Using the current time of the local system as the time point (m seconds), and combining this with the general delay time (usually set to 2 seconds), the latest data timestamps for the entire system every second are determined. Therefore, the data timestamps for the entire system every second are m-2 seconds. Then, the proportion of measurement points is calculated cumulatively from newest to oldest based on these timestamps. When the cumulative proportion first reaches or exceeds a set threshold, a cross-section calculation alignment time stamp is generated. This time stamp is the core reference benchmark for the "moment with the best data calculation timestamp consistency," and the calculation of the computational workload is initiated. This also requires consideration of the "calculation cycle," "calculation sub-item completeness rate," and "calculation sub-item status code," and execution is differentiated according to the set calculation method. Specifically, in the "real-time calculation" method, the trigger condition of reaching the "calculation cycle" must be met. During calculation, the data of each calculation sub-item is directly used without additional consideration of the data timestamps to generate the corresponding calculation results. In the "completeness ratio" method, it is necessary to first determine whether the data proportion of each sub-item of the computational workload at a certain moment reaches or exceeds the set threshold. If the data timestamp completeness ratio does not reach the set threshold, the latest data timestamps for the entire system every second are used to calculate and generate the results. Regardless of the calculation method used, the calculation results of the cross-sectional data must be marked with their own status codes according to the "calculation sub-item status codes". The final output includes a complete calculation result containing the "calculation data timestamp", "data value", "status code", and "calculation time".

[0063] It should be noted that the two calculation methods mentioned above are not mutually exclusive, but rather a collaborative logic based on the "completeness ratio" and supplemented by the other method. The "completeness ratio" is a crucial consideration during the calculation process (this metric directly relates to the data latency of the calculated sub-items, ensuring the integrity of the data involved in the calculation): First, a target completeness ratio (e.g., 95%) is set, along with a built-in maximum waiting time; during calculation, priority is given to waiting for the calculated sub-item data to reach the set completeness ratio. If the requirement is met within the maximum waiting time, the calculation is performed based on the compliant data; if the target completeness ratio is not reached after the timeout, to avoid calculation stagnation affecting the overall process, it will automatically switch to using the latest data timestamps from the entire system every second for calculation, thus achieving a balance between data integrity and calculation timeliness.

[0064] According to an embodiment of the present invention, the reentrant computation window verification method corresponds to step 6 above. Since data fluctuations, system failures, or parameter changes may occur during the computation process, differences may exist between real-time computation results and historical computation results. Based on system requirements and the complexity of the computation task, a suitable backtracking point is selected. The backtracking point can be a specific point in time, or the start or end point of a computation phase. The computation input data and intermediate results of the specified backtracking point are recovered from the data buffer or historical data storage to ensure data integrity and consistency. During recomputation, the real-time computation results of the current computation task are obtained, including the final result of the computation and the parameter values ​​of each computational sub-item. Historical computation results of the same computation task at the specified backtracking point are extracted from the historical data storage. The computational quantum term parameters in the real-time computation results and historical computation results are compared one by one, and the differences are recorded. These parameters may include input data, intermediate computation results, variables in the computation formula, etc. After obtaining the data, the final values ​​of the real-time computation results and historical computation results are compared. A reasonable difference threshold is set according to the actual application scenario and system accuracy requirements. Based on the comparison result between the difference value and the set threshold, the status code of the computation result is marked. For computational results marked as "abnormal" or "error", the computation task is re-executed, starting from the backtracking point and recalculated using the recovered data.

[0065] like Figure 6 The recalculation process is as follows: Due to data delay, the status code associated with a certain sub-item of the computational quantity x is "fitted" (indicating that the data value is not actually collected, but generated by estimating the data value before t+2). At this time, the computation cycle time arrives, and a calculated value is generated based on the fitted data value. Waiting for t+7, the delayed data arrives, and the data value status code changes to "collected" (indicating that the data is actually collected). At this time, the computational quantity x is recalculated, generating a new calculation result.

[0066] The method of this invention utilizes a cache-based out-of-order data correction technique, a streaming-based dynamic delay calculation method, and a reentrant calculation window verification method. Employing a computation- and storage-separated architecture, it performs high-frequency data calculation and analysis on key equipment such as IGBTs and fault recording devices in flexible DC power stations. This improves the accuracy and real-time performance of power system fault analysis and diagnosis, leverages the important role of flexible DC transmission systems in traditional power transmission, promotes the in-depth application of industrial time-series data calculation methods and systems oriented towards instantaneous high-frequency characteristics, and drives technological development and application expansion in related technical fields.

[0067] It should be noted that the descriptions of each step in this document are only intended to clearly present the core aspects of the technical solution, and are not the sole limitation on the execution order of the steps. In practical applications, those skilled in the art can reasonably adjust the execution order of the above steps according to the specific data processing scenario, system resource configuration, and computational requirements—for example, some non-related steps can be executed in parallel, or the order of the steps can be changed without affecting the core technical effects such as data timestamp consistency and the accuracy of calculation results. As long as the adjusted execution method does not change the core logic of "changing the calculation alignment timescale of the generated cross-section as the calculation benchmark, dynamically adjusting the calculation time to ensure data alignment, and optimizing the calculation by combining the integrity rate and timeout mechanism," and does not result in substantial differences in technical functions or effects, it falls within the protection scope of this technical solution.

[0068] Based on the same technical concept as the method, this invention also provides an industrial time-series data computing system oriented towards instantaneous high-frequency characteristics, comprising:

[0069] The cache-based out-of-order data correction module includes a buffer construction unit and a data preprocessing unit. The buffer construction unit creates a memory-based data buffer and a computation buffer. The data buffer stores measurement point data, which is written cyclically according to the following method: data is compressed and stored on disk from the start to the archive period, and data from the archive to the end period is migrated to the memory at the start position. The computation buffer stores the computational results generated over time, which are composed of different measurement points. The data preprocessing unit is configured to perform the following preprocessing on the data to be written to the data buffer: automatically adjust the filtering parameters according to the real-time change characteristics and rate of change of the device data values ​​to remove noise interference and abnormal data points caused by transmission errors; sort the limited frame data within a local time period according to the timestamp of the data tag; and fill in the corresponding fitted values ​​for missing data according to the configured strategy.

[0070] The dynamic latency calculation module based on streaming processing includes a calculation processing unit, an incremental calculation unit, and a real-time calculation unit at startup. The calculation processing unit is configured to estimate the scale of the measurement point data for each calculation sub-item after the calculation task starts, and decompose the calculation task into multiple calculation sub-tasks. For each sub-task, data is read from a memory-based measurement point data buffer, and the resulting calculation items are stored in the calculation buffer. These sub-items are then merged into the total calculation result upon completion. The module analyzes the time delay characteristics between the transmission and calculation nodes and the original calculation sub-item data, dynamically processing the difference based on real-time changes to ensure timely data calculation. Consistency of time stamps; the incremental calculation unit is configured to build a memory-based cache pool of the latest calculation results, calibrate heavyweight calculation units based on the magnitude evaluation results, and perform incremental recalculation in combination with the status code of sub-item parameters or the change marker of calculation formula; the real-time calculation unit at the start of calculation is configured to determine the latest data timestamp of the entire system per second based on the timestamp of the received data and the general data delay time, and calculate the proportion of measurement points cumulatively from newest to oldest according to the latest data timestamp. When the cumulative proportion reaches or exceeds the set threshold for the first time, a cross-section calculation alignment time stamp is generated as the core reference benchmark for dynamically adjusting the calculation start time;

[0071] The reentrant computation window verification module includes a backtracking point selection unit and a recomputation verification unit. The backtracking point selection unit is used to specify a backtracking point as needed and recover the computation input data and intermediate results of the specified backtracking point from the data buffer or historical data storage. The recomputation verification unit is configured to compare the real-time computation results with the historical computation results of the corresponding backtracking point, find the differences in computation quantum term parameters and the differences in the final computation results, mark the computation result status code according to the preset threshold, and perform recomputation in case of error or abnormality.

[0072] The present invention also provides an electronic device, the device comprising: one or more processors; a memory; and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the programs, when executed by the processors, implementing the steps of the industrial time-series data calculation method for instantaneous high-frequency characteristics as described above.

[0073] The present invention also provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the industrial time-series data calculation method oriented towards instantaneous high-frequency characteristics as described above.

[0074] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the industrial time-series data calculation method oriented towards instantaneous high-frequency characteristics as described above.

[0075] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, apparatus (systems), electronic devices, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0076] This invention is described with reference to a flowchart of a method according to embodiments of the invention. It should be understood that each step in the flowchart and combinations thereof can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing device, generate instructions for implementing the process. Figure 1 A device for performing a specified function in one or more processes. The processor involved in each embodiment may be a general-purpose processor, a central processing unit, a graphics processing unit, a digital signal processor, a programmable logic device, a quantum computing-based data processing logic device, an artificial intelligence processor, etc., and is not limited thereto.

[0077] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 The function specified in one or more processes.

[0078] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 Steps of a specified function in one or more processes.

Claims

1. A method for calculating industrial time-series data with instantaneous high-frequency characteristics, characterized in that, The method includes: Create a memory-based data buffer to store measurement point data. All measurement point data is written in a loop according to the following method: data from the start to the archiving period is compressed and stored on disk, and data from the archiving period to the end is migrated to the memory at the start position. Create a memory-based computation buffer to store the computational results generated over time, composed of different measurement points; The data to be written to the data buffer is preprocessed as follows: the filtering parameters are automatically adjusted according to the real-time change characteristics and rate of change of the device data values ​​to remove noise interference and abnormal data points caused by transmission errors; for finite frame data within a local time, the data is sorted according to the timestamp of the data tag; and the missing data is filled with the corresponding fitted values ​​according to the configured strategy. The calculation task is initiated, and the scale of the measurement point data for the calculation sub-items is estimated. The calculation task is then broken down into multiple calculation sub-tasks. For each sub-task, data is read from a memory-based measurement point data buffer, and the resulting calculation items are stored in the calculation buffer. Once the calculations are completed, they are merged into the total calculation result. The time delay characteristics between the transmission and calculation nodes and the original calculation sub-item data are analyzed, and the difference is dynamically processed based on real-time changes to ensure the consistency of data calculation timestamps. A memory-based cache pool of the latest calculation results is constructed, and heavyweight calculation units are calibrated based on the magnitude assessment results. Incremental recalculation is performed by combining the sub-item parameter status codes or calculation formula change markers. Based on the timestamp of the received data and combined with the general data delay time, the latest data timestamp of the entire system per second is determined, and the proportion of measurement points is calculated cumulatively from newest to oldest according to the latest data timestamp. When the cumulative proportion reaches or exceeds the set threshold for the first time, a cross-section calculation alignment time stamp is generated as the core reference benchmark for dynamically adjusted calculation start time. The system specifies a backtracking point as required, recovers the computational input data and intermediate results of the specified backtracking point from the data buffer or historical data storage, compares the real-time computational results with the historical computational results of the corresponding backtracking point, identifies the differences in computational quantum term parameters and the differences in the final computational results, marks the computational result status code according to the preset threshold, and performs recalculation in case of errors or abnormalities.

2. The method according to claim 1, characterized in that, The data buffer uses a double-buffering technique, comprising an archive buffer and an auxiliary buffer. The data writing process is as follows: When the timestamp of the data written by the measurement point reaches the preset archive time in the buffer, the archive buffer performs batch lossless compression on the data within the current window, forming data blocks and writing them to the disk file. After compression, the memory space occupied by the archive buffer is released. Uncompressed data in the auxiliary buffer, i.e., new data written after the archive time, is moved to the beginning of the buffer. Subsequent data continues to be written to the buffer in a queue, and this process is repeated.

3. The method according to claim 2, characterized in that, The structure of a data block includes a data block header and a pointer to the compressed package. The data block header contains the data block type, the corresponding measurement point, the data submission frequency, the start time of the data block, and the size of the compressed block. The measurement point data consists of the data time, data value, and data status code.

4. The method according to claim 1, characterized in that, The computation buffer stores multiple computation quantities, and the result of each computation quantity varies with the time domain. Each computation quantity consists of multiple sub-items, which refer to a single timestamped measurement point data, including status code, data time, data value, measurement point name, and measurement point ID. In addition to the sub-items, the computation impact factors of the computation quantity also include real-time time, computation cycle, computation sub-item completeness rate, and computation method. Among them, the computation sub-item completeness rate is the proportion of measurement point data that participates in the computation to the total measurement point data that should participate. Based on the data structure required for computation, a computation space for the aforementioned data structure is established in memory as a computation buffer.

5. The method according to claim 4, characterized in that, The calculation methods include real-time calculation and full ratio calculation. In real-time calculation, when the time reaches the trigger condition of the calculation cycle, the data of each calculation sub-item is used directly without considering the data timestamp to generate the corresponding calculation result. In full ratio calculation, it is determined whether the data proportion of each calculation sub-item at the current time reaches or exceeds the set threshold. If the set threshold is not reached, the latest data timestamp of the entire system every second is used to calculate and generate the result.

6. The method according to claim 5, characterized in that, In actual calculations, a collaborative processing method is used, based on the completeness ratio and supplemented by real-time calculation. First, a target completeness ratio is set, and a maximum waiting time is built in. During calculation, priority is given to waiting for the calculated sub-item data to reach the set completeness ratio. If the requirement is met within the maximum waiting time, the calculation is performed based on the qualified data. If the target completeness ratio is not reached after the timeout, the latest data timestamp of the entire system is used for calculation.

7. The method according to claim 1, characterized in that, For a finite number of frames of data within a local time period, sorting is performed based on the timestamps of the data markers, including: The timestamp of each data write is limited to half the time span of the data buffer, and the data is sorted from oldest to newest according to the timestamps it carries.

8. An industrial time-series data computing system oriented towards instantaneous high-frequency characteristics, characterized in that, include: The cache-based out-of-order data correction module includes a buffer construction unit and a data preprocessing unit. The buffer construction unit creates a memory-based data buffer and a calculation buffer. The data buffer is used to store measurement point data. The data is written in a loop according to the method of compressing and storing the data from the start to the archiving period and migrating the data from the archiving period to the end to the starting position memory. The computation buffer is used to store the computational results generated over time, which consist of different measurement points; The data preprocessing unit is configured to preprocess the data to be written to the data buffer as follows: automatically adjust the filtering parameters according to the real-time change characteristics and rate of change of the device data values ​​to remove noise interference and abnormal data points caused by transmission errors; sort the limited frame data within a local time according to the timestamp of the data marker. For missing data, fill in the corresponding fitted values ​​according to the configured strategy; The dynamic delay calculation module based on streaming processing includes a calculation processing unit, an incremental calculation unit, and a real-time calculation unit at the start of the calculation. The calculation processing unit is configured to estimate the scale of the measurement point data of the calculation sub-item after the calculation task starts, decompose the calculation task into multiple calculation sub-tasks, read data from the memory-based measurement point data buffer for multiple calculation sub-tasks, and save the generated calculation items in the calculation buffer. After the calculation is completed, they are merged into the total calculation result. Analyze the time delay characteristics between transmission and computing nodes and the original computing sub-item data, and dynamically process the difference based on real-time changes to ensure the consistency of data computing timestamps; The incremental calculation unit is configured to build a memory-based cache pool of the latest calculation results, calibrate heavyweight calculation units based on the magnitude evaluation results, and perform incremental recalculation in combination with the sub-item parameter status code or calculation formula change marker. The real-time calculation unit at the start of the calculation is configured to determine the latest data timestamp of the entire system per second based on the timestamp of the received data and the general data delay time, and to calculate the proportion of measurement points cumulatively from newest to oldest according to the latest data timestamp. When the cumulative proportion reaches or exceeds the set threshold for the first time, a cross-section calculation alignment time scale is generated as the core reference benchmark for dynamically adjusting the calculation start time. The reentrant calculation window verification module includes a backtracking point selection unit and a recalculation verification unit. The backtracking point selection unit is used to specify a backtracking point as needed and restore the calculation input data and intermediate results of the specified backtracking point from the data buffer or historical data storage. The calculation verification unit is configured to compare the real-time calculation results with the historical calculation results at the corresponding backtracking point, identify the differences in the calculation quantum term parameters and the differences in the final calculation results, mark the status codes of the calculation results according to the preset threshold, and recalculate in case of errors or abnormalities.

9. An electronic device, characterized in that, The device includes: one or more processors; a memory; and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, wherein when the programs are executed by the processors, they implement the steps of the industrial time-series data calculation method for instantaneous high-frequency characteristics as described in any one of claims 1-7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the industrial time-series data calculation method for instantaneous high-frequency characteristics as described in any one of claims 1-7.

11. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the steps of the industrial time-series data calculation method for instantaneous high-frequency characteristics as described in any one of claims 1-7.

Images