A data timestamp self-calibration and trusted fusion method and system

Abstract

The application provides a data timestamp self-calibration and credible fusion method and system, comprising: obtaining energy consumption sampling data of at least two heterogeneous data sources; identifying link abnormalities based on gateway forwarding timestamps and server receiving timestamps, and processing sampling data with link abnormalities to obtain an effective sample set; estimating time calibration parameters of each data source based on the effective sample set, and compensating and correcting original sampling timestamps of devices according to the time calibration parameters to obtain calibrated timestamps; time aligning multi-source heterogeneous energy consumption data based on the calibrated timestamps; calculating credibility of each data source, and generating fusion weights according to the credibility, wherein the credibility is determined based on data quality related indexes; weighting and fusing aligned sequences of each data source based on the fusion weights to output fusion results; and feeding back the fusion results and link abnormality statistical information for updating the time calibration parameters and the credibility, so as to realize online closed-loop self-calibration and credible fusion.

Description

Technical Field

[0001] This invention belongs to the field of trusted data processing technology for integrated energy management and industrial automation, and specifically relates to a data timestamp self-calibration and trusted fusion method, system and device. Background Technology

[0002] In integrated energy stations, integrated power generation, grid, load and storage parks and energy storage power stations, it is usually necessary to connect multiple devices such as electricity meters, PCS, BMS, cold and heat source units, circulating pumps, air conditioning terminals, weather stations and various environmental sensors at the same time. The data sources cover multiple protocol forms such as Modbus, BACnet, OPCUA, IEC104, private TCP and edge gateway cache upload.

[0003] Due to differences in sampling periods, clock accuracy, and network transmission paths among different devices, the following problems commonly occur in actual operation: 1) The timestamps from different data sources come from different sources, resulting in offsets ranging from seconds to minutes and long-term drift; 2) Network latency, cache retransmission, and packet loss can cause data to be out of order or duplicated; 3) The server's receiving time is inconsistent with the device's sampling time, causing the multi-source data to be unable to be accurately aligned; 4) Erroneous data directly enters the prediction, optimization and control modules, which can easily lead to scheduling misjudgments or control oscillations.

[0004] In practical engineering, the aforementioned time misalignment problem directly leads to the inability of multiple power sources, energy, and state variables to form physically consistent state vectors within the same control cycle. This, in turn, causes a chain reaction of effects such as distortion of prediction model inputs, oscillations in optimized scheduling results, frequent charging and discharging of energy storage, and instability of the control closed loop. Especially in second-level or even sub-second-level scheduling scenarios, time errors are amplified into power errors, becoming a key bottleneck factor affecting the accuracy and stability of integrated energy control.

[0005] Therefore, there is an urgent need for a time synchronization and reliable fusion technology for multi-source heterogeneous devices to achieve automatic alignment and quality assessment of data timelines, thereby improving the accuracy of integrated energy scheduling and control from the source. Summary of the Invention

[0006] This invention provides a data timestamp self-calibration and trusted fusion method, system and device, which can at least solve the technical problem that time error is amplified into power error in second-level or even sub-second-level scheduling scenarios, becoming a key bottleneck factor affecting the accuracy and stability of integrated energy control.

[0007] On the one hand, the present invention provides a data timestamp self-calibration and trusted fusion method, including: Acquire energy consumption sampling data from at least two heterogeneous data sources, and record the original sampling timestamp of the device, the forwarding timestamp of the gateway, the receiving timestamp of the server for each sample data, as well as generate an event identifier for unique identification and cross-link association; Link anomalies are identified based on the gateway forwarding timestamp and the server receiving timestamp, and the sampled data of the link anomalies are processed to obtain a valid sample set; Based on the effective sample set, time calibration parameters for each data source are estimated. The time calibration parameters include at least clock offset and clock drift rate. The original sampling timestamp of the device is compensated and corrected according to the time calibration parameters to obtain the calibration timestamp. Based on the calibration timestamp, the multi-source heterogeneous energy consumption data is time-aligned to obtain the aligned sequence corresponding to each data source. For each data source, a credibility score is calculated, and a fusion weight is generated based on the credibility score, which is determined based on data quality-related indicators. The aligned sequences from each data source are weighted and fused based on the fusion weights, and the fusion result is output. When the credibility meets the preset conditions, the control command is limited or speed is limited, and the fusion result and link anomaly statistics are fed back to update the time calibration parameters and credibility, so as to realize online closed-loop self-calibration and trusted fusion.

[0008] In one optional implementation, identifying link anomalies based on the gateway forwarding timestamp and the server receiving timestamp includes: Determine whether the gateway forwarding timestamp sequence of the same data source is monotonically increasing; if it is not monotonically increasing, it is determined to be out of order. Calculate the difference between the server's received timestamp and the gateway's forwarded timestamp. If the difference exceeds the preset maximum allowable delay threshold, it is determined to be an abnormal delay. Determine whether the sampled data of the same event identifier appears multiple times within a preset time window; if so, it is determined to be a duplicate upload. Sampling data that exhibits any of the above-mentioned anomalies will be marked as anomalies.

[0009] In an optional implementation, the time calibration parameter further includes link round-trip time; the clock offset and link round-trip time are obtained through a four-time bidirectional handshake mechanism, which includes: The server sends a time synchronization request to the device or gateway where the data source is located and records the time of sending. After receiving a request, the device or gateway records the time of receipt and then includes that time in the response message. Before sending a response message, the device or gateway records the sending time and includes that sending time in the response message; The server records the time of receipt after receiving the response message; Based on the above four time points, the link round-trip delay is calculated as half the sum of the uplink and downlink transmission times, and the handshake offset is calculated as half the difference between the uplink and downlink transmission times.

[0010] In one optional implementation, the clock drift rate is estimated as follows: within a sliding time window, a linear function is fitted with the original sampling timestamp of the device at each sampling point as the independent variable and the reference time after delay compensation on the server side as the dependent variable; the difference between the slope of the linear function and 1 is the clock drift rate.

[0011] In one optional implementation, the original sampling timestamp of the device is compensated and corrected to obtain a calibration timestamp, including: Multiply the original sampling timestamp of the device by the drift correction factor, which is equal to 1 plus the clock drift rate multiplied by one million, to obtain the drift-compensated time; Add the clock offset to the time after drift compensation, and then subtract the link delay compensation term to obtain the calibration timestamp; The link delay compensation term is calculated using a first-order low-pass filter: the link delay compensation term at the current moment is equal to the smoothing coefficient multiplied by the link delay compensation term at the previous moment, plus a subtracted smoothing coefficient multiplied by the difference between the server receiving timestamp and the gateway forwarding timestamp at the current sampling point; the initial value of the link delay compensation term is set to the link round-trip delay.

[0012] In one optional implementation, the step of time-aligning the multi-source heterogeneous energy consumption data based on the calibration timestamp to obtain the aligned sequence corresponding to each data source includes: Construct a global timeline using the minimum and maximum values ​​of the calibration timestamps from all data sources; A series of equally spaced time points are generated on the global timeline according to a preset fusion cycle; For each data source, the calibration timestamp of each sampling point is matched to the nearest global time point. If there is no corresponding sampling point at a certain global time point, interpolation or the method of keeping the previous valid value is used to fill it, so as to obtain the aligned sequence of each data source on a unified time axis.

[0013] In one optional implementation, the data quality-related indicators include at least two of the following: time continuity, latency stability, jitter stability, packet loss rate, out-of-order or duplicate proportion, and amplitude exceeding limit proportion; wherein: The time continuity is defined as the ratio of the number of valid sampling points within the evaluation window to the theoretical number of points that should be reached. The theoretical number of points that should be reached is determined based on the rated sampling period of the data source or the mode of the actual number of points reached within the window. The latency stability is defined as: 1 minus the ratio of the average link latency within the evaluation window to the maximum allowable latency threshold, and this value is not less than 0, where the link latency is the difference between the server receiving timestamp and the gateway forwarding timestamp; The jitter stability is defined as: 1 minus the ratio of the standard deviation of link delay within the evaluation window to the maximum allowable jitter threshold, and this value is not less than 0; The packet loss rate is equal to 1 minus the time continuity; The disorder or repetition ratio is the ratio of the disorder or repetition count within the evaluation window to the theoretically expected number of points; The amplitude exceeding the limit ratio is the ratio of the number of sampled values ​​exceeding the preset normal operating limits to the number of valid sampled points.

[0014] In one optional implementation, the confidence level is calculated as follows: Assign a weight between 0 and 1 to each of the following: time continuity, delay stability, jitter stability, packet loss rate, out-of-order or repetitive proportion, and amplitude exceeding the limit proportion, and the sum of all weights is 1. The credibility score is obtained by multiplying each of the six index values ​​by its corresponding weight and then summing the results. The credibility score ranges from 0 to 1.

[0015] In one optional implementation, generating fusion weights based on credibility includes: The credibility of each data source is exponentially calculated, with the exponent being a preset value greater than or equal to 1, to obtain the fusion weight of that data source. Within the same fusion cycle, the fusion weights of all data sources are summed, and the fusion weight of each data source is divided by the sum to obtain the normalized fusion weight.

[0016] In one optional implementation, the weighted fusion of the aligned sequences from each data source based on the fusion weight, and the output of the fusion result, includes: The weighted value is obtained by multiplying the aligned sequence value of each data source by the normalized fusion weight of that data source. The numerator is obtained by summing the weighted values ​​of all data sources; Sum the combined weights of all data sources before normalization, and add a very small positive number to get the denominator; Divide the numerator by the denominator to obtain the fusion result.

[0017] In one optional implementation, the preset conditions include at least one of the following: The proportion of data sources with credibility below the preset lower limit to the total number of data sources reaches or exceeds the preset proportion threshold. The credibility of any pre-specified key data source is lower than the preset lower limit and the duration exceeds the preset time threshold.

[0018] In one optional implementation, the step of limiting the amplitude or speed of the control command includes: When the preset conditions are met, the amplitude of the control command is limited to a rated value that does not exceed a preset multiple, wherein the preset multiple is between 0.3 and 0.7. At the same time, the change in the control command per unit time is limited to not exceed the preset maximum rate of change; When the preset conditions are no longer met and are not met within a consecutive preset number of fusion cycles, the original control command amplitude limit and speed limit boundaries are restored.

[0019] In one optional implementation, the step of feeding back the fusion result and link anomaly statistics to update the time calibration parameters and confidence level includes: The fusion result of each fusion output, along with the corresponding calibration timestamp, alignment error, and link anomaly statistics, is stored in the feedback cache. Every preset update cycle, the estimation and confidence calculation of time calibration parameters are re-executed based on the data in the feedback cache; The updated time calibration parameters are used for timestamp compensation correction of subsequent sampled data, and the updated confidence level is used for subsequent fusion weight generation.

[0020] This invention provides a complete data foundation for separating time errors by recording three levels of timestamps (original timestamps from the device, forwarded timestamps from the gateway, and received timestamps from the server) and event identifiers. It avoids network disturbances contaminating calibration parameters by identifying link anomalies and removing abnormal samples. By estimating the clock offset and drift rate of each data source and compensating for and correcting the original timestamps of the devices, it eliminates the system errors inherent in the device clocks themselves, achieving millisecond-level time synchronization. Global time alignment and resampling based on calibration timestamps solve the problem of direct fusion of heterogeneous sampling periods. Reliability calculation and weighted fusion based on multiple quality indicators dynamically suppress the negative impact of low-quality data, outputting physically consistent optimal estimates. Through reliability-triggered control commands for amplitude / rate limiting and a closed-loop feedback update mechanism, it ensures safe system operation under extreme data quality conditions and achieves long-term adaptive calibration. Attached Figure Description

[0021] Figure 1 A schematic diagram of the data timestamp self-calibration and trusted fusion method provided by the present invention.

[0022] Figure 2 A schematic diagram of the data timestamp self-calibration and trusted fusion system framework provided by the present invention. Detailed Implementation

[0023] The present invention will now be described in detail with reference to the accompanying drawings and embodiments. It should be noted that, unless otherwise specified, the embodiments and features described herein can be combined with each other.

[0024] The following detailed description is exemplary and intended to provide further detailed explanation of the invention. Unless otherwise specified, all technical terms used in this invention have the same meaning as commonly understood by one of ordinary skill in the art. The terminology used in this invention is for describing particular embodiments only and is not intended to limit the scope of exemplary embodiments according to the invention.

[0025] In integrated energy systems (such as integrated power generation, grid, load, and storage projects in industrial parks, energy storage power stations, and integrated energy stations), it is necessary to simultaneously connect devices such as electricity meters, power storage converters (PCS), battery management systems (BMS), chiller and heater units, circulating pumps, air conditioning terminals, weather stations, and various environmental sensors. These devices use different communication protocols (Modbus, BACnet, OPCUA, IEC104, proprietary TCP, etc.), have different sampling periods (from sub-second to minute-level), significant differences in clock crystal oscillator accuracy, and suffer from uncertain delays, buffer retransmissions, and packet loss in network transmission.

[0026] The resulting practical engineering problems include: different timestamp sources from different data sources, resulting in fixed offsets and long-term drifts ranging from seconds to minutes; network latency, buffer retransmission, and packet loss leading to data out-of-order or duplication; and inconsistencies between server reception time and device sampling time, causing inaccurate alignment of multi-source data. These time misalignments directly cause the inability of multi-source power, energy, and state variables to form physically consistent state vectors within the same control cycle, leading to a chain reaction of effects such as distortion of prediction model inputs, oscillations in optimized scheduling results, frequent charging and discharging of energy storage, and instability of the control closed loop. Especially in second-level or even sub-second-level scheduling scenarios, time errors are amplified into power errors, becoming a key bottleneck affecting the accuracy and stability of integrated energy control.

[0027] Therefore, the core technical problem to be solved by this invention is: how to automatically calibrate the timestamps of multi-source heterogeneous data, eliminate time misalignment caused by clock offset, drift and network uncertainty, identify and downweight low-quality data, and output physically consistent and reliable fused data, thereby improving the scheduling and control accuracy of the integrated energy system from the source.

[0028] Please see Figure 1 This invention provides a data timestamp self-calibration and trusted fusion method, comprising: Step S1: Obtain energy consumption sampling data from at least two heterogeneous data sources, and record the original sampling timestamp of the device, the forwarding timestamp of the gateway, the receiving timestamp of the server for each sampling data, as well as generate an event identifier for unique identification and cross-link association.

[0029] Step S2: Identify link anomalies based on gateway forwarding timestamps and server receiving timestamps, and process the sampled data of link anomalies to obtain a valid sample set.

[0030] Step S3: Estimate the time calibration parameters for each data source based on the effective sample set. The time calibration parameters include at least the clock offset and clock drift rate. Then, compensate and correct the original sampling timestamp of the device according to the time calibration parameters to obtain the calibration timestamp.

[0031] Step S4: Time-align the multi-source heterogeneous energy consumption data based on the calibration timestamp to obtain the aligned sequence corresponding to each data source.

[0032] Step S5: Calculate the credibility of each data source and generate a fusion weight based on the credibility. The credibility is determined based on data quality-related indicators.

[0033] Step S6: Perform weighted fusion on the aligned sequences of each data source based on the fusion weight, and output the fusion result.

[0034] Step S7: When the credibility meets the preset conditions, the control command is limited or the speed is limited, and the fusion result and link anomaly statistics are fed back to update the time calibration parameters and credibility, so as to realize online closed-loop self-calibration and trusted fusion.

[0035] This invention provides a complete data foundation for separating time errors by recording three levels of timestamps (original timestamps from the device, forwarded timestamps from the gateway, and received timestamps from the server) and event identifiers. It avoids network disturbances contaminating calibration parameters by identifying link anomalies and removing abnormal samples. By estimating the clock offset and drift rate of each data source and compensating for and correcting the original timestamps of the devices, it eliminates the system errors inherent in the device clocks themselves, achieving millisecond-level time synchronization. Global time alignment and resampling based on calibration timestamps solve the problem of direct fusion of heterogeneous sampling periods. Reliability calculation and weighted fusion based on multiple quality indicators dynamically suppress the negative impact of low-quality data, outputting physically consistent optimal estimates. Through reliability-triggered control commands for amplitude / rate limiting and a closed-loop feedback update mechanism, it ensures safe system operation under extreme data quality conditions and achieves long-term adaptive calibration.

[0036] The synergistic effect of the above-mentioned technical features of this invention eliminates the amplified impact of time misalignment on the prediction and control closed loop from the source, and significantly improves the operational stability and control reliability of the integrated energy system.

[0037] Step S1: Obtain energy consumption sampling data from at least two heterogeneous data sources, and record the original sampling timestamp of the device, the forwarding timestamp of the gateway, the receiving timestamp of the server for each sampling data, as well as generate an event identifier for unique identification and cross-link association.

[0038] This invention first connects to at least two different types of energy consumption data sources (e.g., an electricity meter and a weather station). For each sampled data point (e.g., a power value output by the electricity meter every second), the system does not only record the value itself, but also records timestamps for three different stages: the original sampling timestamp of the device; the timestamp stamped by the clock of the front-end device (e.g., the electricity meter, PCS) at the moment of sampling; and the timestamp that the device believes was when the data was generated, but the device clock may be inaccurate (with offsets and drift).

[0039] Gateway forwarding timestamps are time stamps added by the gateway's local clock when data passes through a field gateway (such as an edge computing gateway or protocol converter). This time reflects the moment the data enters the network transmission entry point and can be used to analyze the transmission latency from the device to the gateway (if the device is directly connected to the gateway, this latency is usually small and stable; if it goes through multiple levels of aggregation, it may fluctuate).

[0040] The server receives the timestamp, and when the data finally arrives at the central server or cloud platform, it is tagged with a high-precision clock (such as NTP synchronization) on the server. This time is the server's standard reference time, with the highest accuracy, but it lags behind the device's sampling time.

[0041] The three timestamps represent the temporal characteristics of the data at different stages. The original device timestamp includes the inherent error of the device clock, the gateway timestamp reflects the time boundary of data entering the network, and the server timestamp serves as a reference for subsequent calibration. By combining the three, time errors from different sources, such as device clock skew, clock drift, network transmission latency, and jitter, can be isolated.

[0042] As an example, in a certain integrated power generation, grid, load, and storage park, there is an energy storage converter (PCS) that reports real-time power at a period of 500ms. In a certain sampling: the original sampling timestamp of the device: the internal clock of the PCS shows 2025-05-15 08:00:00.000 (but the PCS clock is about 0.5 seconds fast every day, and there is drift).

[0043] Gateway forwarding timestamp: The edge gateway records 2025-05-15 08:00:00.085 when it receives data (the gateway clock has been synchronized with the server via NTP, ensuring high accuracy). Server receiving timestamp: The cloud server records 2025-05-15 08:00:00.205 when it receives data. Event identifier: PCS01_3A2F_1234567 is generated by combining "Data Source Identifier (PCS_01)", "Device Frame Sequence Number (0x3A2F)", and "Sampling Period Index (1234567)".

[0044] This invention records three levels of timestamps and unique event identifiers, providing a complete time trajectory and identity information for subsequent link anomaly detection, latency statistics, clock skew and drift estimation, and data deduplication. Without this information, subsequent self-calibration and trusted fusion cannot distinguish between two different types of time deviations: device clock inaccuracy and network latency.

[0045] Step S2: Identify link anomalies based on gateway forwarding timestamps and server receiving timestamps, and process the sampled data of link anomalies to obtain a valid sample set.

[0046] Because network transmission can be affected by buffer queuing, link congestion, and retransmission mechanisms, data may exhibit out-of-order delivery (later data arrives first), sudden increases in latency (a particular piece of data is exceptionally slow), and duplicate uploads (the same data is retransmitted multiple times due to timeout). If these anomalous data are directly used for subsequent clock calibration, they will severely pollute the parameter estimation results. Therefore, this invention utilizes the difference sequence between the gateway timestamp and the server's received timestamp, along with event identifiers, to automatically identify these anomalous patterns. Anomalous data is then marked, removed, or assigned extremely low weights, retaining only clean, valid samples to form a valid sample set.

[0047] The identification rules of this invention typically include: (1) checking whether the gateway forwarding timestamp sequence of the same data source is strictly monotonically increasing. If a bounce occurs (the gateway time of the later one is earlier than the previous one), it is determined to be out of order; (2) calculating the difference between the server receiving timestamp and the gateway forwarding timestamp. If the difference exceeds the preset maximum allowable delay threshold (e.g., 500ms or 2000ms, depending on the scenario), it is determined to be an abnormal delay surge; (3) within a preset time window (e.g., 5 seconds), if the same event identifier appears twice or more, it is determined to be a duplicate upload. When any of the above anomalies occur, the sampled data is marked as abnormal and will not enter the valid sample set, or will be automatically downweighted in subsequent credibility calculations.

[0048] This invention effectively avoids data contamination caused by network anomalies, ensuring that the sample set used for clock calibration and time alignment is clean and reliable. This step is equivalent to the denoising stage in data preprocessing, significantly improving the robustness of subsequent parameter estimation and preventing a single network jitter from ruining the overall time synchronization accuracy.

[0049] Step S3: Estimate the time calibration parameters for each data source based on the effective sample set. The time calibration parameters include at least the clock offset and clock drift rate. Then, compensate and correct the original sampling timestamp of the device according to the time calibration parameters to obtain the calibration timestamp.

[0050] After cleaning in step (two), the system obtains a series of valid sampling points for each data source. Each sampling point has a known original sampling timestamp from the device (denoted as...). t dev ) and the corresponding server high-precision reference time (denoted as ) t ref This can be approximated by subtracting the estimated network latency from the timestamp received by the server. Since the device clock may have a fixed offset (e.g., always 0.5 seconds slow) and a cumulative drift over time (e.g., 1 second fast per day, i.e., a drift rate of approximately 11.6 ppm), the system needs to establish a mathematical model to... t dev Mapped to actual physical time. This model is typically a linear model: t ref =(1+ drift ) tdev + offsettref ,in drift It is the clock drift rate (unit: ppm). offset It is a fixed offset. Using linear regression or other estimation algorithms, the unique offset for each data source is determined from the effective sample set. drift and offset Then, for each piece of original sampled data, a calibrated timestamp is calculated using this model. t cal This ensures that data from all data sources is mapped to the same high-precision time base (i.e., the server clock domain).

[0051] Clock drift rate exhibits a long-term trend, typically changing slowly over minutes to hours; a fixed offset, on the other hand, updates rapidly with each bidirectional handshake. The calibrated timestamp eliminates systematic errors caused by inherent device clock inaccuracies, but the impact of network transmission latency still requires separate compensation (this compensation is refined into Δ in subsequent steps or weight 5). net ).

[0052] This invention achieves high-precision mapping of device timestamps to a unified time domain by estimating the independent clock offset and drift of each data source online, eliminating time misalignment caused by differences in the accuracy of device clock crystals and accumulated errors over long-term operation. This is the most critical technical means to achieve physical consistency of multi-source data.

[0053] Step S4: Time-align the multi-source heterogeneous energy consumption data based on the calibration timestamp to obtain the aligned sequence corresponding to each data source.

[0054] Even though the timestamps of each data source are calibrated to a unified time domain, the original sampling periods of different devices are still different (e.g., electricity meters once per second, PCS once every 500ms, and weather stations once every 5 seconds). For subsequent fusion calculations (such as weighted averaging), the data from all data sources needs to be resampled onto the same time grid. This step first determines a global time axis (e.g., from the earliest data moment to the latest data moment) based on the calibration timestamps of all data sources. Then, it generates a series of equally spaced time points according to a preset fusion period (e.g., 200ms, 500ms, or 1 second). For each data source, the value corresponding to its original calibration timestamp is mapped to the nearest global time point. If a time point lacks original data, interpolation (e.g., linear interpolation) or the previous valid value is used to fill it, ultimately obtaining the aligned sequence of each data source on a unified time axis. .

[0055] Time alignment is a prerequisite for fusion. Alignment errors (such as approximation errors introduced by interpolation) will be part of the quality feedback and affect the reliability calculation. Common resampling methods include nearest neighbor, linear interpolation, or spline interpolation, and the specific choice depends on the data characteristics (preservation is suitable for discrete state quantities, and linear interpolation is suitable for continuous analog quantities).

[0056] This invention solves the problem of data that cannot be directly fused due to inconsistent sampling periods of heterogeneous devices. By resampling, all data are projected onto the same time grid, providing a directly computable input for subsequent weighted fusion.

[0057] Step S5: Calculate the credibility of each data source and generate a fusion weight based on the credibility. The credibility is determined based on data quality-related indicators.

[0058] Data quality can vary significantly across different data sources and time periods. For example, a single electricity meter might experience unstable latency due to communication jitter, or a sensor might exhibit abnormal fluctuations due to low battery power. This step calculates a comprehensive reliability index for each data source. CiThis metric reflects the "reliability" of the data source within the current evaluation window. Reliability is calculated based on multiple data quality indicators, such as temporal continuity (whether data is lost), latency stability (whether latency is stable), jitter stability (whether latency variance is small), data duplication / out-of-order ratio, and whether the values ​​exceed physical limits. After obtaining the reliability score, the system converts it into a fusion weight. αi Generally, the higher the credibility, the greater the weight. The weight can be the credibility itself, or it can be a power function of the credibility (to amplify the contribution of high-credibility sources).

[0059] The credibility level is dynamic and changes over time. The system periodically (e.g., every minute) recalculates the credibility of each data source within a sliding window. The credibility value typically ranges from 0 to 1, where 0 represents completely untrustworthy (and should be discarded) and 1 represents completely trustworthy. This credibility mechanism enables the system to dynamically identify and suppress the negative impact of low-quality data sources on the fusion results, avoiding errors in overall control decisions due to temporary sensor failures or network fluctuations.

[0060] Step S6: Perform weighted fusion on the aligned sequences of each data source based on the fusion weight, and output the fusion result.

[0061] After time alignment and reliability assessment, the system has a set of aligned values ​​for each fusion cycle (e.g., every 1 second). and the corresponding normalized fusion weights αi ( t This step performs a weighted fusion calculation: Fusion result y ( t The sum of weighted values ​​from all data sources is equal to the sum of the weights (a very small positive number is added to prevent the denominator from being zero). This formula is essentially a weighted average, but allows the total weight to be greater than 1 or equal to 1 after normalization. The fused result represents the optimal estimate of the physical quantity (such as total power, temperature, pressure) at the current moment, integrating information from all available data sources, with higher-quality sources contributing more.

[0062] Fusion can be applied to a single physical quantity (such as multiple sensors measuring the same temperature) or to a combination of different physical quantities (but usually it is the same physical quantity that is fused). In practical systems, multiple redundant sensors are often deployed at the same measuring point, or the same electrical quantity (such as grid connection point power) measured by different types of equipment (such as electricity meters and PCS) is fused.

[0063] This invention, through weighted fusion, outputs more accurate and robust estimates than any single data source. Even if some data sources are temporarily abnormal, the fusion result remains stable, avoiding control malfunctions caused by single data fluctuations.

[0064] Step S7: When the credibility meets the preset conditions, the control command is limited or the speed is limited, and the fusion result and link anomaly statistics are fed back to update the time calibration parameters and credibility, so as to realize online closed-loop self-calibration and trusted fusion.

[0065] When the system detects a significant decline in overall data quality (e.g., low-reliability data sources account for more than 30%, or critical data sources such as the main electricity meter remain consistently low-reliability), it considers the current fusion results potentially unreliable. Continuing to issue commands according to normal control logic could lead to dangers (such as overcharging of energy storage or equipment overload). In this case, the system automatically triggers protective measures: limiting the amplitude (maximum power) and speed (rate of change) of control commands issued to the executing devices (such as PCS and air conditioning units), operating in a conservative mode until data quality is restored.

[0066] The fusion results, time alignment errors, and link anomaly statistics (such as packet loss count and latency peaks) are not discarded after use, but are fed back to the preceding modules of the system. This information is used to periodically update time calibration parameters (e.g., re-estimate drift rate) and threshold values ​​in reliability calculations. This enables the system to adapt to long-term drift caused by equipment aging and changes in the network environment, maintaining high accuracy at all times.

[0067] Preset conditions are configurable, for example: the number of data sources with a confidence level below 0.7 exceeds 20% of the total; or the main meter's confidence level is below 0.5 for more than 3 seconds. The limiting factor is typically set between 0.3 and 0.7, indicating a limit of 30% to 70% of the rated power. The feedback update cycle can be on the order of minutes or hours.

[0068] The closed-loop feedback mechanism of this invention makes the method adaptive, enabling it to track changes in devices and networks online and maintain high accuracy over a long period. The safety linkage mechanism ensures that even under extremely poor data quality conditions, the system will not output dangerous control commands, achieving synergistic protection of functional safety and data quality.

[0069] In one optional implementation, identifying link anomalies based on gateway forwarding timestamps and server receiving timestamps includes: Determine whether the gateway forwarding timestamp sequence of the same data source is monotonically increasing; if it is not monotonically increasing, it is determined to be out of order. Calculate the difference between the server's received timestamp and the gateway's forwarded timestamp. If the difference exceeds the preset maximum allowable delay threshold, it is determined to be an abnormal delay. Determine whether the sampled data of the same event identifier appears multiple times within a preset time window; if so, it is determined to be a duplicate upload. Sampling data that exhibits any of the above-mentioned anomalies will be marked as anomalies.

[0070] In this invention, out-of-order delivery detection is achieved by checking the monotonicity of the gateway timestamp, because under normal circumstances, data generated earlier should have a smaller gateway timestamp. Delay anomaly detection determines whether data is severely delayed due to network congestion by comparing the server reception time minus the gateway forwarding time with a preset threshold (e.g., 500ms in industrial scenarios, which can be relaxed to 2000ms in wireless scenarios). Duplicate upload detection utilizes the uniqueness of event identifiers; if the same identifier is received within a sliding window (e.g., 2 seconds), it indicates that a retransmission or mistransmission has occurred.

[0071] The three rules in this invention cover the most common link-layer problems in practical engineering, are simple to implement, and have low computational overhead. The maximum allowable latency threshold and window size can be configured by the system or dynamically adjusted based on historical data.

[0072] In an optional implementation, the time calibration parameter further includes link round-trip time; the clock offset and link round-trip time are obtained through a four-time bidirectional handshake mechanism, which includes: The server sends a time synchronization request to the device or gateway where the data source is located and records the time of sending. After receiving a request, the device or gateway records the time of receipt and then includes that time in the response message. Before sending a response message, the device or gateway records the sending time and includes that sending time in the response message; The server records the time of receipt after receiving the response message; Based on the above four time points, the link round-trip delay is calculated as half the sum of the uplink and downlink transmission times, and the handshake offset is calculated as half the difference between the uplink and downlink transmission times.

[0073] This invention's mechanism does not rely on the accuracy of the device clock; it achieves this by exchanging four timestamps (the time the client request was sent). t 1. Server request receiving time t 2. Server response sending time t 3. Client response reception time t 4) The round-trip time (RTT) of the link and the relative clock offset between the server and the device can be calculated. The formula derivation is based on the assumption of symmetrical uplink and downlink transmission delays (which can be corrected statistically if the actual transmission delays are asymmetrical).

[0074] This invention presents a simplified implementation of the classic Network Time Protocol (NTP), suitable for industrial real-time control scenarios. The handshake can occur between the server and gateway, or directly with smart devices. The order and inclusion relationship of the four time points are clearly defined.

[0075] The bidirectional handshake mechanism of this invention can quickly obtain the relative clock offset and link round-trip delay between the server and the device / gateway without relying on GPS or other external clock sources, providing a real-time and accurate reference for timestamp self-calibration. This method has low equipment requirements, is applicable to most existing industrial gateways and smart devices, and has excellent engineering compatibility.

[0076] In one alternative implementation, the clock drift rate is estimated as follows: within a sliding time window, a linear function is fitted with the original sampling timestamp of the device at each sampling point as the independent variable and the reference time after delay compensation on the server side as the dependent variable; the difference between the slope of the linear function and 1 is the clock drift rate.

[0077] Drift rate is a long-term cumulative effect and needs to be estimated using historical data from multiple sampling points. The method involves collecting valid samples from the data source within a sliding time window (e.g., 10 minutes), each sample bearing the device's original sampling timestamp. t dev and the corresponding server reference time t ref (This reference time can be obtained by subtracting the link delay compensation term from the server reception time.) Since the device clock model can be approximated as linear: t ref =(1+ r ) tdev + i ,in r This is the drift rate (unit: ppm). A straight line is fitted using linear regression (such as least squares method), and the slope of the line is... k =1+ r ,therefore r = k 1. This allows us to extract the frequency deviation of the device clock from long-term observation data.

[0078] The length of the sliding window needs to be carefully considered: too short and it's susceptible to noise, too long and the response is slow. It's typically set to 5–30 minutes. Linear fitting can utilize recursive algorithms to reduce computation, eliminating the need to process all data points at once.

[0079] As an example, a certain electricity meter accumulated 600 valid sampling points (period of 1 second) within a 10-minute window. The original timestamp sequence of the device is as follows: t dev =[0,1,2,...,599] seconds (relative to the start of the window), the corresponding server reference time after latency compensation is... club=[0.005,1.015,2.025,...,599.025] seconds. The equation of the straight line is obtained by least squares fitting. club =1.00001 tdev +0.005. The slope 1.00001 - 1.00001 equals 0.00001 - 0.00001, which is a drift rate of 10¹⁰ ppm. This means that the meter clock accumulates a 1-second error every 1 million seconds (approximately 11.6 days). The server uses this drift rate to pre-correct future data.

[0080] This method uses a large amount of historical data to statistically estimate the frequency drift of the device clock, compensating for the limitation of a single bidirectional handshake, which can only obtain instantaneous offsets and cannot obtain long-term drift. Through drift compensation, high time synchronization accuracy can be maintained over a long period between two handshakes, reducing the handshake frequency and lowering communication overhead. This is especially important for widely deployed low-cost devices (whose crystal oscillator stability is poor).

[0081] In one optional implementation, the original sampling timestamp of the device is compensated and corrected to obtain a calibration timestamp, including: Multiply the original sampling timestamp of the device by the drift correction factor. The drift correction factor is equal to 1 plus the clock drift rate multiplied by one million to obtain the drift-compensated time. Add the clock offset to the time after drift compensation, and then subtract the link delay compensation term to obtain the calibration timestamp; The link delay compensation term is calculated using a first-order low-pass filter: the link delay compensation term at the current moment is equal to the smoothing coefficient multiplied by the link delay compensation term at the previous moment plus a minus smoothing coefficient multiplied by the difference between the server receiving timestamp and the gateway forwarding timestamp at the current sampling point; the initial value of the link delay compensation term is set to the link round-trip delay.

[0082] Calibration timestamp = Drift compensation + Offset compensation - Delay compensation. Drift compensation: Multiply the original timestamp of the device by (1 + ... drift ×10 6) To eliminate errors accumulated over time due to clock frequency deviations. Offset compensation: Adding the clock offset obtained through handshaking. offset (This offset can be the handshake offset or the residual offset after drift correction.) Delay compensation: Subtract the link delay compensation term Δ net This compensation term reflects the transmission latency of data from the gateway to the server, but instead of directly using a single difference (because a single difference may have jitter), it uses a first-order low-pass filter (exponentially weighted moving average) to smooth the estimate. Smoothing coefficient. αThe larger the value, the smoother the filtering result, but the slower the response; conversely, the smaller the value, the faster the response but the greater the fluctuation. The initial value can be half of the round-trip delay obtained from the handshake (approximately the one-way delay) or the round-trip delay can be used directly.

[0083] The formula for a first-order low-pass filter is Δ net ( k )= α Δ net ( k 1)+(1 α ) d ( k ),in d ( k )= tsrv ( k ) tgw ( k This filter can suppress transient jitter while tracking the average trend of latency. Smoothing coefficient. α The value is usually between 0.8 and 0.99, with 0.95 recommended.

[0084] This invention processes link delay through first-order low-pass filtering, which can smooth out random jitter and follow slowly changing network trends, making it more robust than simply using the original difference. The resulting calibration timestamp enables heterogeneous data to be mapped onto a unified, high-precision timeline, which is the core algorithmic detail for achieving physical consistency.

[0085] In one optional implementation, the multi-source heterogeneous energy consumption data is time-aligned based on the calibration timestamp to obtain the aligned sequence corresponding to each data source, including: Construct a global timeline using the minimum and maximum values ​​of the calibration timestamps from all data sources; A series of equally spaced time points are generated on the global timeline according to a preset fusion cycle; For each data source, the calibration timestamp of each sampling point is matched to the nearest global time point. If there is no corresponding sampling point at a certain global time point, interpolation or the method of keeping the previous valid value is used to fill it, so as to obtain the aligned sequence of each data source on a unified time axis.

[0086] First, determine the start and end points of the global timeline: take the minimum calibration timestamp among all sample points from all data sources as the start point and the maximum as the end point (or truncate the most recent time window). Then, generate a series of equally spaced time labels between the start and end points using a fixed fusion period (e.g., 200ms, 500ms, or 1s). Finally, for each data source, map the calibration timestamps and values ​​of its original sample points onto this grid: if a grid point happens to have an original sample point, use it directly; otherwise, use interpolation (e.g., linear interpolation) or directly use the previous most recent valid value (zero-order preservation). In this way, all data sources produce sequences under the same time labels.

[0087] The choice of the fusion period should balance real-time performance and data volume, and is usually set to an integer multiple or close to the minimum sampling period among all data sources. For discrete signals such as switch signals, interpolation may result in meaningless values, so hold-over is preferable. For analog signals, linear interpolation is more reasonable.

[0088] This invention solves the fundamental problem of asynchronous sampling periods from heterogeneous devices by using a global time axis and resampling, enabling point-by-point comparison and fusion of data from different time granularities on the same time base. This method is suitable for both real-time online processing (sliding window) and offline analysis, and is simple to implement with low computational cost.

[0089] In one optional implementation, the data quality-related indicators include at least two of the following: time continuity, latency stability, jitter stability, packet loss rate, out-of-order or duplicate rate, and amplitude exceeding limit rate; wherein: Temporal continuity is defined as the ratio of the number of valid sampling points within the evaluation window to the theoretical number of points that should be reached. The theoretical number of points that should be reached is determined based on the rated sampling period of the data source or the mode of the actual number of points reached within the window. Delay stability is defined as: 1 minus the ratio of the average link delay within the evaluation window to the maximum allowable delay threshold, and this value is not less than 0, where the link delay is the difference between the server's receiving timestamp and the gateway's forwarding timestamp; Jitter stability is defined as: 1 minus the ratio of the standard deviation of link delay within the evaluation window to the maximum allowable jitter threshold, and this value is not less than 0; Packet loss rate equals 1 minus temporal continuity; The disorder or repetition ratio is the ratio of the disorder or repetition count within the evaluation window to the theoretically expected number of points; The amplitude exceeding the limit ratio is the ratio of the number of sampled values ​​exceeding the preset normal operating upper and lower limits to the number of valid sampling points.

[0090] These indicators in this invention reflect the reliability of the data source from different dimensions. Temporal continuity reflects whether data arrives at the expected frequency. The theoretical expected number of points is determined by the rated sampling period (e.g., 1 second) and window length (e.g., 60 seconds). If the rated period is unknown, the interval with the highest frequency within the window (mode) is used as an estimate. Packet loss rate complements temporal continuity, directly representing the proportion of missing data. Delay stability measures whether the average transmission delay is within an acceptable range. If the average delay approaches or exceeds the maximum allowable delay threshold, stability approaches 0. Jitter stability measures the degree of delay fluctuation. If the standard deviation is large (indicating inconsistent network speed), stability is low. Out-of-order or duplicate event ratio statistics show the percentage of out-of-order or duplicate events, reflecting the correctness of network transmission. Amplitude exceeding limits statistics show whether the physical quantity value exceeds a reasonable range (e.g., power cannot be negative or exceeds twice the rated value), reflecting whether the sensor or device itself is faulty.

[0091] The calculation of these metrics relies on an evaluation window (such as the most recent 60 seconds or 100 sampling points). The system can be configured to use only a subset of these metrics (e.g., not applicable for non-numerical state variables where amplitude exceeds limits). Maximum permissible jitter threshold. J max It is typically set to 5~50ms.

[0092] This invention quantifies quality through multi-dimensional indicators, enabling a precise depiction of the real-time health status of each data source, thus making credibility assessment more comprehensive and accurate. Compared to using only packet loss rate or a single indicator, this solution can effectively distinguish between different fault modes such as "high latency caused by network congestion" and "abnormal values ​​caused by equipment failure," providing a more scientific basis for subsequent fusion weight allocation.

[0093] In one alternative implementation, the credibility is calculated as follows: Assign a weight between 0 and 1 to each of the following: time continuity, delay stability, jitter stability, packet loss rate, out-of-order or repetitive proportion, and amplitude exceeding the limit proportion, and the sum of all weights is 1. The credibility score is obtained by multiplying each of the six index values ​​by its corresponding weight and then summing the results. The credibility score ranges from 0 to 1.

[0094] This invention employs a weighted linear summation model. Each indicator is first assigned a positive value (packet loss rate, out-of-order ratio, and out-of-limit ratio are converted to positively correlated indicators by subtracting 1), then multiplied by a preset weight coefficient, and finally summed. The sum of the weight coefficients is 1, and each weight is between 0 and 1, reflecting the relative importance of different indicators in the credibility evaluation. For example, in real-time control scenarios, latency stability and jitter stability may have higher weights; in data recording and analysis scenarios, continuity and packet loss rate have higher weights. This model is simple, transparent, and easy to implement in engineering and adjust online.w 1 to w 6 can be fixed or dynamically adjusted according to the device type (for example, for switch signals, the weight for exceeding the amplitude limit is set to 0). The final confidence value is between 0 and 1, and can be directly used as the base of the fusion weight.

[0095] The weighted summation method compresses multidimensional quality information into an intuitive credibility scalar, making subsequent weight generation and degradation judgment simple and efficient. Furthermore, due to the configurable weights, this method is highly flexible and can adapt to the varying sensitivities of different quality indicators in different industrial scenarios. Compared to complex fuzzy logic or neural networks, this method has extremely low computational cost, making it suitable for high-frequency operation on edge gateways or real-time servers.

[0096] In one alternative implementation, generating fusion weights based on credibility includes: The credibility of each data source is exponentially calculated, with the exponent being a preset value greater than or equal to 1, to obtain the fusion weight of that data source. Within the same fusion cycle, the fusion weights of all data sources are summed, and the fusion weight of each data source is divided by the sum to obtain the normalized fusion weight.

[0097] The embodiments of the present invention provide a basis for reliability Ci To actual fusion weight αi The conversion method involves first exponentially calculating the credibility. αi ′= Cig ,in c ≥1. The property of this function is: when c When = 1, the weight and credibility are linearly related; when c When the value is greater than 1, the weights of high confidence are further amplified, while the weights of low confidence are further compressed (e.g., C =0.9 squared is 0.81, and C =0.5 squared becomes 0.25, the gap widens from 0.4 to 0.56). This helps to highlight the dominant role of high-quality sources in the fusion process. After obtaining the non-normalized weights, to ensure the averageity of the fusion result (total weight normalization), these weights are divided by the sum of the non-normalized weights of all sources, so that the sum of the weights used for the final weighted average is 1. Power exponent c It is a hyperparameter that can be adjusted according to the application scenario. It is commonly used in real-time control scenarios. c =2 or 3, commonly used in redundancy averaging scenarios. c =1.

[0098] This invention, through power-law adjustment, can flexibly control the aggressiveness of the fusion strategy. When the data quality is generally good, it can be used... c=1 Retains all information; Use when there is suspicion that part of the data source is unreliable. c >1 can more effectively suppress interference from low-quality sources. Normalization ensures the correctness of the fused output in terms of physical dimensions (e.g., the power after weighted averaging will not produce scaling errors due to the sum of weights not being equal to 1).

[0099] In one optional implementation, the aligned sequences from each data source are weighted and fused based on fusion weights, and the fusion result is output, including: The weighted value is obtained by multiplying the aligned sequence value of each data source by the normalized fusion weight of that data source. The numerator is obtained by summing the weighted values ​​of all data sources; Sum the combined weights of all data sources before normalization, and add a very small positive number to get the denominator; Divide the numerator by the denominator to obtain the fusion result.

[0100] It's important to note that the denominator here is not the sum of the normalized weights (because the sum after normalization is 1), but rather the sum of the unnormalized weights plus a very small positive number. This is done to preserve the physical meaning of the fusion result: if the unnormalized weights of all data sources are very small (e.g., all confidence levels are close to 0), then the denominator is also small, but the numerator is even smaller, and the result still approaches 0. Simultaneously, the very small positive number... e This prevents division by zero errors caused by a denominator of 0. In fact, this formula is equivalent to a weighted average: y =∑ αi ′+ e ∑ αi ′ xi ,in αi ′ represents the unnormalized weights. This is consistent with the traditional weighted average formula because the normalized weights... with = αi ′ / ∑ αi ',but y =∑ wixi =∑ αi ′∑ αi ′ xi This has been added. e This is solely for numerical stability. In this invention, the fusion weights before normalization... Cig Minimal positive number e Usually 10 is taken. 9 10 6 .

[0101] This invention preserves the influence of the absolute value of the weights on the fusion result by using the sum of non-normalized weights as the denominator (for example, when all weights are very small, the result is less sensitive to outliers).

[0102] In one optional implementation, the preset conditions include at least one of the following: The proportion of data sources with credibility below the preset lower limit to the total number of data sources reaches or exceeds the preset proportion threshold. The credibility of any pre-specified key data source is lower than the preset lower limit and the duration exceeds the preset time threshold.

[0103] This embodiment defines the "preset conditions" for triggering control constraint strategies (amplitude limiting / rate limiting). The conditions are divided into two types: one is an overall decline in data quality, i.e., a low-reliability data source (reliability). Ci < C If the percentage of (min) errors is too high (e.g., exceeding 20%), it indicates a large-scale system anomaly, and a conservative mode should be entered. Secondly, if a critical data source is malfunctioning, such as the main meter or main PCS, which are essential data sources for the control loop, even if only this one source fails for more than a few seconds, constraints should be triggered to prevent control instability. These two conditions can be used independently or in combination (logical OR).

[0104] C min It is usually set to 0.6 to 0.8. or The (proportion threshold) is typically set between 10% and 30%. T The min time is typically set to 1–30 seconds. These parameters can all be configured by the system.

[0105] For example, a certain energy storage power station has 10 data sources and is set up C min =0.7, proportional threshold or =0.2, the key source is the "grid connection point electricity meter", time threshold. T min =3 seconds. Scenario A: System monitoring detects that the credibility of 3 data sources is below 0.7 (30%), exceeding the 20% threshold, triggering a downgrade. Scenario B: The overall credibility of all data sources is good (low credibility percentage 10%), but the credibility of the grid-connected meter suddenly drops to 0.5 and lasts for 3 seconds. The system detects that the duration has reached the threshold, also triggering a downgrade. Scenario C: The credibility of the grid-connected meter briefly drops to 0.5 but only lasts for 1 second before recovering, thus not triggering a downgrade to avoid frequent fluctuations.

[0106] This embodiment considers both the overall health status of the system and single critical points of failure, comprehensively covering scenarios requiring safety intervention. By introducing a duration threshold, frequent degradation / recovery oscillations caused by transient jitter are avoided, thus improving the stability of the control system.

[0107] In one alternative implementation, the control command is subjected to amplitude or speed limiting, including: When the preset conditions are met, the amplitude of the control command will be limited to a rated value that does not exceed a preset multiple, which is between 0.3 and 0.7. At the same time, the change in the control command per unit time is limited to not exceed the preset maximum rate of change; When the preset conditions are no longer met and are not met within a consecutive preset number of fusion cycles, the original control command amplitude limit and speed limit boundaries are restored.

[0108] Limiting refers to reducing the upper limit of control commands. For example, if the original allowed command range was -100kW to 100kW, it is limited to -50kW to 50kW after degradation (assuming a multiplier of 0.5). Rate limiting refers to limiting the rate of change of commands. For example, if the original allowed change of 20kW every 100ms, it is limited to a change of 5kW every 100ms after degradation. These two measures work together to prevent sudden changes or excessively large control commands when the data is unreliable. The recovery mechanism requires that the non-degradation conditions be met for several consecutive fusion cycles (e.g., 5 consecutive cycles) before exiting the degradation mode and restoring the original boundary.

[0109] Preset multiplier β The value is typically between 0.3 and 0.7, depending on the system's risk tolerance. The maximum rate of change can be set based on the equipment's dynamic response capability. The number of consecutive cycles is usually between 3 and 10.

[0110] As an example, in an energy storage power station, the power command range of the energy storage PCS during normal operation is [-500kW, 500kW], with a rate of change limited to 50kW per 100ms. When a degradation condition is triggered, the system limits the command range to [-250kW, 250kW]. β =0.5), with the rate of change limited to 10kW per 100ms. This ensures safety by preventing drastic charging and discharging of the PCS even if errors exist in the fusion result. During degradation, the system continuously monitors conditions. If, for five consecutive fusion cycles (200ms each, 1 second total), a low-confidence percentage of less than 10% is detected and the confidence level of the critical source returns to normal, the degradation process exits, and the original amplitude and rate limits are restored.

[0111] This invention achieves a fail-safe mode by limiting amplitude and speed. Even under the worst data quality conditions, it will not output instructions that exceed the equipment's tolerance or change too rapidly, thus avoiding equipment damage or power grid impact. The recovery mechanism ensures that the system automatically returns to normal operation after the anomaly disappears, without manual intervention.

[0112] In one optional implementation, the fusion results and link anomaly statistics are fed back to update the time calibration parameters and confidence level, including: The fusion result of each fusion output, along with the corresponding calibration timestamp, alignment error, and link anomaly statistics, is stored in the feedback cache. Every preset update cycle, the estimation and confidence calculation of time calibration parameters are re-executed based on the data in the feedback cache; The updated time calibration parameters are used for timestamp compensation correction of subsequent sampled data, and the updated confidence level is used for subsequent fusion weight generation.

[0113] The various modules of the system (anomaly identification, calibration parameter estimation, and credibility calculation) are not static but continuously optimized based on historical fusion results. Specifically, a feedback cache is set up to store the output results of each fusion (including fusion values, calibration timestamps, alignment errors, link anomaly statistics, etc.). At regular update intervals (e.g., 5 minutes, 1 hour), the system retrieves this batch of historical data from the cache and reruns the parameter estimation algorithm (e.g., recalculating drift rate and offset) and credibility calculation (e.g., updating the statistical baseline). Then, the newly obtained parameters replace the currently used parameters for subsequent real-time processing. In this way, the system can adaptively track long-term changes such as equipment aging, crystal oscillator drift, and network topology changes.

[0114] The choice of update cycle should balance adaptation speed and computational overhead. A circular queue can be used for the feedback cache, retaining data from the most recent N cycles (e.g., data from the most recent hour).

[0115] The closed-loop feedback of this invention endows the method with self-learning and adaptive capabilities, eliminating the need for frequent manual parameter tuning. Especially during long-term equipment operation, slowly varying factors such as crystal oscillator aging and network environment changes are automatically compensated for, consistently maintaining high time synchronization quality and fusion accuracy. This is a key innovation distinguishing it from open-loop calibration schemes, ensuring the long-term effectiveness of this invention.

[0116] On the other hand, see Figure 2 This invention provides a data timestamp self-calibration and trusted fusion system 200, comprising: The acquisition module 201 is used to acquire energy consumption sampling data from at least two heterogeneous data sources, and to record the original sampling timestamp of the device, the gateway forwarding timestamp, the server receiving timestamp, and generate an event identifier for unique identification and cross-link association for each piece of sampling data.

[0117] The anomaly identification and sample purification module 202 is used to identify link anomalies based on the gateway forwarding timestamp and the server receiving timestamp, and to process the sampled data of the link anomaly to obtain a valid sample set.

[0118] The calibration parameter estimation and timestamp self-calibration module 203 is used to estimate the time calibration parameters of each data source based on the effective sample set. The time calibration parameters include at least clock offset and clock drift rate. The original sampling timestamp of the device is compensated and corrected according to the time calibration parameters to obtain the calibration timestamp.

[0119] The time alignment and resampling module 204 is used to perform time alignment on multi-source heterogeneous energy consumption data based on the calibration timestamp to obtain the aligned sequence corresponding to each data source.

[0120] The data quality assessment and credibility calculation module 205 is used to calculate the credibility of each data source, and the credibility is determined based on data quality-related indicators.

[0121] The fusion weight generation and trusted fusion module 206 is used to generate fusion weights based on trustworthiness, and to perform weighted fusion on the aligned sequences of each data source based on the fusion weights, and output the fusion result. The control constraint and closed-loop feedback module 207 is used to execute amplitude or speed limit on the control command when the credibility meets the preset conditions, and to feed back the fusion result and link anomaly statistics to the anomaly identification and sample purification module, the calibration parameter estimation and timestamp self-calibration module, and the data quality assessment and credibility calculation module, so as to update the anomaly judgment rules, time calibration parameters and credibility.

[0122] The method provided by the embodiments of the present invention will be further described below through specific examples.

[0123] A data timestamp self-calibration and trusted fusion method includes: Step 1: Collect energy consumption data from at least two types of data sources, and record the data as follows: i The first data source k The sample value is Record the original sampling timestamps of the devices respectively. Gateway forwarding timestamp With server receive timestamp At the same time, an event identifier is generated. Used for cross-link association and deduplication.

[0124] Step 2: Identify and process link anomalies based on the gateway forwarding timestamp and the server receiving timestamp. The link anomalies include at least one or more of the following: latency surge caused by cache queuing, out-of-order delivery, time bounce caused by duplicate uploads and retransmissions. The abnormal sampled data is then marked, removed, or downweighted to obtain an effective sample set.

[0125] Step 3, estimate the first based on the effective sample set. iThe time calibration parameters of the data source include at least the link round-trip time. Clock offset With clock drift rate The four-time bidirectional handshake is used to obtain And calculate according to the following relationship:

[0126]

[0127] A calibration model is constructed to map device timestamps to a unified time domain, resulting in the calibration timestamp:

[0128] in This is the offset. This is a link delay compensation term, and At least by Or by The statistical value was determined; in, For server-side timing, The time is at the device side or gateway side; the round-trip delay and offset are calculated based on the assumption that the uplink and downlink delays are equivalent and symmetrical, or if symmetry is not satisfied, the window statistical mean / median is used to compensate for the asymmetric error.

[0129] Step 4: Construct a global timeline based on the calibration timestamp, perform time alignment and resampling on the multi-source data, and obtain the aligned sequence. .

[0130] Step 5: Calculate the credibility of each data source. And generate fusion weights based on credibility. The credibility is calculated using at least one of the following weighting relationships:

[0131] in and The fusion weight is defined as follows:

[0132] The fusion weights are normalized within the same fusion period, wherein ; Step 6: Perform trusted weighted fusion on the aligned multi-source data and output the fusion result. It satisfies:

[0133] in The number of data sources participating in the integration. This is used to avoid extremely small positive numbers with a denominator of 0; Step 7: When the credibility meets the preset degradation conditions, a control constraint strategy is triggered. The control constraint strategy includes at least one or more of the following: amplitude limiting, speed limiting, freeze holding, switching to conservative control mode, or enabling redundant data sources. The fusion result, alignment error, and anomaly statistics are fed back to update the time calibration parameters and credibility, thereby realizing online closed-loop self-calibration and trusted fusion.

[0134] The event identifier mentioned in step 1 It is generated by a combination of at least two of the following: data source identifier ID, device frame sequence number, sampling period index, and timestamp hash.

[0135] In step 2, link anomalies must be identified by at least one of the following rules: 1) The gateway forwarding timestamp sequence is not monotonically increasing; 2) Satisfy This means that the difference between the server's received timestamp and the gateway's forwarded timestamp exceeds the maximum allowable delay threshold; 3) The same event identifier appears more than once within the preset window; in The value range is 100ms to 5000ms.

[0136] The parameter value range in step 3 satisfies: 1) When expressed in ms, the range of values ​​is: ms; 2) Expressed in ppm and with a value range of ppm; 3) The range of values ​​is ms; 4) The range of values ​​is ms.

[0137] The definitions of each indicator in step 5 should include at least the following: 1) Time continuity law ,in The theoretical number of points that should be reached within the window. Valid points; 2) Packet loss rate ; 3) Disorder / Repetition Ratio ,in This is for random or repeated counting; 4) Over-limit ratio ,in This is for counting beyond the limit.

[0138] And the The range of values ​​is .

[0139] Furthermore, the delay stability Defined as ,in To evaluate the average link delay within the window; the jitter stability Defined as ,in The standard deviation of link delay. The maximum allowable jitter threshold is preferably 1 to 50 ms.

[0140] The confidence threshold in step 5 is ,when When the data source is set to a downgraded or excluded state, The value range is 0.6 to 0.8, and The value range is 1 to 3.

[0141] The preset degradation conditions in step 7 include at least one of the following: 1) Low proportion of trusted data sources meets the requirements ,in The value range is 10% to 30%; 2) The duration of credibility being below the threshold satisfies ,in The value range is 1s to 30s; And when the degradation conditions are met, the control commands are... Implement a limit to make it meet the requirements. ,in The value range is 0.3 to 0.7.

[0142] In another aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method described in any one of the above.

[0143] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the specific implementation of the present invention. Any modifications or equivalent substitutions that do not depart from the spirit and scope of the present invention should be covered within the scope of protection of the claims of the present invention.

Claims

1. A method for self-calibration and trusted fusion of data timestamps, characterized in that, include: Acquire energy consumption sampling data from at least two heterogeneous data sources, and record the original sampling timestamp of the device, the forwarding timestamp of the gateway, the receiving timestamp of the server for each sample data, as well as generate an event identifier for unique identification and cross-link association; Link anomalies are identified based on the gateway forwarding timestamp and the server receiving timestamp, and the sampled data of the link anomalies are processed to obtain a valid sample set; Based on the effective sample set, time calibration parameters for each data source are estimated. The time calibration parameters include at least clock offset and clock drift rate. The original sampling timestamp of the device is compensated and corrected according to the time calibration parameters to obtain the calibration timestamp. Based on the calibration timestamp, the multi-source heterogeneous energy consumption data is time-aligned to obtain the aligned sequence corresponding to each data source. For each data source, a credibility score is calculated, and a fusion weight is generated based on the credibility score, which is determined based on data quality-related indicators. The aligned sequences from each data source are weighted and fused based on the fusion weights, and the fusion result is output. When the credibility meets the preset conditions, the control command is limited or speed is limited, and the fusion result and link anomaly statistics are fed back to update the time calibration parameters and credibility, so as to realize online closed-loop self-calibration and trusted fusion.

2. The method according to claim 1, characterized in that, Link anomalies are identified based on the gateway forwarding timestamp and the server receiving timestamp, including: Determine whether the gateway forwarding timestamp sequence of the same data source is monotonically increasing; if it is not monotonically increasing, it is determined to be out of order. Calculate the difference between the server's received timestamp and the gateway's forwarded timestamp. If the difference exceeds the preset maximum allowable delay threshold, it is determined to be an abnormal delay. Determine whether the sampled data of the same event identifier appears multiple times within a preset time window; if so, it is determined to be a duplicate upload. Sampling data that exhibits any of the above-mentioned anomalies will be marked as anomalies.

3. The method according to claim 1, characterized in that, The time calibration parameters also include link round-trip time; the clock offset and link round-trip time are obtained through a four-time bidirectional handshake mechanism, which includes: The server sends a time synchronization request to the device or gateway where the data source is located and records the time of sending. After receiving a request, the device or gateway records the time of receipt and then includes that time in the response message. Before sending a response message, the device or gateway records the sending time and includes that sending time in the response message; The server records the time of receipt after receiving the response message; Based on the above four time points, the link round-trip delay is calculated as half the sum of the uplink and downlink transmission times, and the handshake offset is calculated as half the difference between the uplink and downlink transmission times.

4. The method according to claim 3, characterized in that, The method for estimating the clock drift rate is as follows: within a sliding time window, a linear function is fitted using the original sampling timestamp of the device at each sampling point as the independent variable and the reference time after delay compensation on the server side as the dependent variable; the difference between the slope of the linear function and 1 is the clock drift rate.

5. The method according to claim 3, characterized in that, The original sampling timestamp of the device is compensated and corrected to obtain a calibration timestamp, including: Multiply the original sampling timestamp of the device by the drift correction factor, which is equal to 1 plus the clock drift rate multiplied by one million, to obtain the drift-compensated time; Add the clock offset to the time after drift compensation, and then subtract the link delay compensation term to obtain the calibration timestamp; The link delay compensation term is calculated using a first-order low-pass filter: the link delay compensation term at the current moment is equal to the smoothing coefficient multiplied by the link delay compensation term at the previous moment, plus a subtracted smoothing coefficient multiplied by the difference between the server receiving timestamp and the gateway forwarding timestamp at the current sampling point; the initial value of the link delay compensation term is set to the link round-trip delay.

6. The method according to claim 1, characterized in that, The step of time-aligning the multi-source heterogeneous energy consumption data based on the calibration timestamp to obtain the aligned sequence corresponding to each data source includes: Construct a global timeline using the minimum and maximum values ​​of the calibration timestamps from all data sources; A series of equally spaced time points are generated on the global timeline according to a preset fusion cycle; For each data source, the calibration timestamp of each sampling point is matched to the nearest global time point. If there is no corresponding sampling point at a certain global time point, interpolation or the method of keeping the previous valid value is used to fill it, so as to obtain the aligned sequence of each data source on a unified time axis.

7. The method according to claim 1, characterized in that, The data quality-related indicators include at least two of the following: time continuity, latency stability, jitter stability, packet loss rate, out-of-order or duplicate proportion, and amplitude exceeding limit proportion; wherein: The time continuity is defined as the ratio of the number of valid sampling points within the evaluation window to the theoretical number of points that should be reached. The theoretical number of points that should be reached is determined based on the rated sampling period of the data source or the mode of the actual number of points reached within the window. The latency stability is defined as: 1 minus the ratio of the average link latency within the evaluation window to the maximum allowable latency threshold, and this value is not less than 0, where the link latency is the difference between the server receiving timestamp and the gateway forwarding timestamp; The jitter stability is defined as: 1 minus the ratio of the standard deviation of link delay within the evaluation window to the maximum allowable jitter threshold, and this value is not less than 0; The packet loss rate is equal to 1 minus the time continuity; The disorder or repetition ratio is the ratio of the disorder or repetition count within the evaluation window to the theoretically expected number of points; The amplitude exceeding the limit ratio is the ratio of the number of sampled values ​​exceeding the preset normal operating upper and lower limits to the number of valid sampling points.

8. The method according to claim 7, characterized in that, The credibility is calculated as follows: Assign a weight between 0 and 1 to each of the following: time continuity, delay stability, jitter stability, packet loss rate, out-of-order or repetitive proportion, and amplitude exceeding the limit proportion, and the sum of all weights is 1. The credibility score is obtained by multiplying each of the six index values ​​by its corresponding weight and then summing the results. The credibility score ranges from 0 to 1.

9. The method according to claim 1, characterized in that, The generation of fusion weights based on credibility includes: The credibility of each data source is exponentially calculated, with the exponent being a preset value greater than or equal to 1, to obtain the fusion weight of that data source. Within the same fusion cycle, the fusion weights of all data sources are summed, and the fusion weight of each data source is divided by the sum to obtain the normalized fusion weight.

10. A data timestamp self-calibration and trusted fusion system, comprising: The acquisition module is used to acquire energy consumption sampling data from at least two heterogeneous data sources, and to record the original sampling timestamp of the device, the gateway forwarding timestamp, the server receiving timestamp, and generate an event identifier for unique identification and cross-link association for each sample data. The anomaly identification and sample purification module is used to identify link anomalies based on the gateway forwarding timestamp and the server receiving timestamp, and to process the sampled data of the link anomaly to obtain a valid sample set. The calibration parameter estimation and timestamp self-calibration module is used to estimate the time calibration parameters of each data source based on the effective sample set. The time calibration parameters include at least clock offset and clock drift rate. The module also compensates and corrects the original sampling timestamp of the device according to the time calibration parameters to obtain the calibration timestamp. The time alignment and resampling module is used to perform time alignment on the multi-source heterogeneous energy consumption data based on the calibration timestamp to obtain the aligned sequence corresponding to each data source. The data quality assessment and credibility calculation module is used to calculate the credibility of each data source, and the credibility is determined based on data quality-related indicators. The fusion weight generation and trusted fusion module is used to generate fusion weights based on trustworthiness, and to perform weighted fusion on the aligned sequences of each data source based on the fusion weights, and output the fusion result; The control constraint and closed-loop feedback module is used to execute amplitude or speed limiting on the control command when the credibility meets the preset conditions, and to feed back the fusion result and link anomaly statistics to the anomaly identification and sample purification module, the calibration parameter estimation and timestamp self-calibration module, and the data quality assessment and credibility calculation module, so as to update the anomaly judgment rules, time calibration parameters and credibility.

Images