Energy data acquisition and processing method and system based on edge calculation

Abstract

The invention relates to the technical field of data processing, in particular to an energy data collecting and processing method and system based on edge computing, and the method comprises the steps: executing the following core operations on an edge computing node deployed at an energy site: automatically scanning a network and constructing a three-level topological structure of a site, a loop and equipment; plug and play and digital image management of the equipment are realized; secondly, dynamically adjusting a data acquisition strategy based on a service demand and a network state, and carrying out priority shunting and preprocessing on data; carrying out on-site calculation, fusion analysis and quality treatment on the collected data to generate a standardized high-value data object; working modes are intelligently switched according to network conditions, cooperative uploading with the cloud is carried out during normal, local autonomous control is executed based on topology and real-time data during interruption, and breakpoint continuous transmission and state synchronization are carried out according to priorities after recovery. According to the method, the intelligentization of data acquisition, the toughness of system operation and the high efficiency of edge cloud collaboration are realized.

Description

Technical Field

[0001] This application relates to the field of data processing technology, specifically to a method and system for energy data acquisition and processing based on edge computing. Background Technology

[0002] In large-scale distributed energy scenarios such as highways, to achieve zero-carbon and refined energy management, real-time monitoring and data collection are required for a vast number of diverse and heterogeneous devices (such as meters, sensors, and photovoltaic inverters) along the route, including tunnels, service areas, and substations. Currently, mainstream solutions mainly rely on centralized cloud platforms for centralized data processing and control.

[0003] However, existing technologies have significant shortcomings: complex device access, with varying brands and communication protocols of field equipment leading to long system integration cycles, difficulties in later expansion, and challenges in achieving "plug-and-play" functionality. Inflexible data acquisition strategies fail to dynamically adjust sampling rates and transmission strategies based on network congestion and business importance, easily resulting in delays or loss of critical event uploads due to the conflict between massive data volumes and limited bandwidth. The cloud-based centralized processing model is highly dependent on network continuity; once the network is interrupted, not only does data upload cease, but the edge side also lacks local analysis and decision-making capabilities, hindering effective control and increasing the operational risks of the energy system. Summary of the Invention

[0004] To address the aforementioned problems, this invention provides a method and system for energy data acquisition and processing based on edge computing.

[0005] In a first aspect, the technical solution of the present invention provides an energy data acquisition and processing method based on edge computing, wherein the following operations are performed at the edge computing node deployed on the energy site side: S1. By constructing a unified abstraction layer and an adaptive protocol conversion engine, the network is automatically scanned, and a three-level topology structure from site, loop to end device is identified and constructed. When a new device is connected, the device type and data point table are automatically identified and included in the management, and a digital image of the site device containing the three-level topology structure, device attributes and data point table is generated and maintained. S2. Based on the digital image of the site equipment and the preset service level agreement, and combined with the real-time network status, dynamically determine the sampling rate of various energy monitoring data; perform priority management on the collected raw data stream, divide it into a high-priority instant processing queue and a low-priority local aggregation queue, and perform aggregation or compression on the data in the low-priority local aggregation queue. S3. Perform on-site computation and fusion analysis on the raw data from the high-priority real-time processing queue of S2 and the aggregated or compressed data from the low-priority local aggregation queue to generate standardized data objects containing key quantities, derived quantities, data quality identifiers and anomaly identifiers. S4. Based on the three-level topology constructed in S1 and the standardized data object generated in S3, perform the following operations: When the network is functioning normally, the standardized data objects are uploaded to the cloud management platform via an edge-cloud collaborative transmission mechanism; In the event of a network outage, offline control is executed based on locally preset security policies associated with the three-tier topology and real-time data generated by S3 to maintain the operation of core energy services. After the network is restored, the cached data and running status are synchronized through a breakpoint resume and status reconciliation protocol based on the three-level topology node ID.

[0006] Through a unified abstraction layer and an adaptive protocol conversion engine, three-level topology identification and digital image management have completely changed the traditional solution where device access required manual configuration of protocols and mapping of data points for each device. This has enabled devices to be "plug and play," significantly shortening the system integration cycle for large-scale distributed energy scenarios, reducing the technical threshold and manpower costs for later device expansion, and solving the compatibility and adaptation problems of different brands and multi-protocol devices. Based on business SLA and real-time network status, dynamic sampling rate adjustment and priority queue management break the rigidity of traditional fixed sampling strategies. It realizes differentiated processing of high-frequency collection and real-time reporting of key data and low-frequency aggregation and compressed transmission of steady-state data. Under limited network bandwidth, it ensures the real-time and reliability of key event data such as power surges and over-limit alarms, while reducing redundant data transmission, improving network resource utilization, and avoiding the risk of delay or loss of key information under the impact of massive data. As a further limitation of the technical solution of the present invention, S1 specifically includes the following steps: S11. Deploy a unified abstraction layer on the edge computing node. The unified abstraction layer is pre-configured with protocol parsing driver modules for multiple industrial protocols. Based on the unified abstraction layer, construct an adaptive protocol conversion engine. S12. By actively probing or listening to network packets, the network within the energy site is scanned to identify the communication addresses of online devices and capture their response packets to extract protocol-related feature information. S13. For each device identified in S12, the adaptive protocol conversion engine matches the extracted feature information with the preset protocol feature library, automatically loads the corresponding protocol parsing driver module, and then reads the device attribute information; based on the logical relationship between the device attribute information and the communication address, it automatically infers and constructs a three-level topology structure from the site, loop to the end device. S14. Based on the three-level topology, the data point structure of each device is automatically parsed through the loaded protocol parsing driver module; for each parsed raw data point, a corresponding unified variable identifier is generated, thereby forming the data point mapping relationship set of the device. The data point mapping relationship set defines the correspondence between the raw data points and the unified variable identifier; the device and its data point mapping relationship set are registered in the initial site device digital image. S15. When a new device is subsequently detected to be connected, repeat steps S12 to S14 to generate a corresponding data point mapping relationship set for the new device, and dynamically update the new device and its data point mapping relationship set to the digital image of the site device.

[0007] The automatic triggering update mechanism for new device access enables real-time dynamic maintenance of the digital image of site equipment, ensuring the consistency between the image data and the status of on-site equipment. This provides accurate basic equipment data for upper-level dynamic sampling and edge control, avoiding the problem of the traditional static equipment ledger being out of touch with the actual situation on site.

[0008] As a further limitation of the technical solution of the present invention, in S13, the step of matching the extracted feature information with a preset protocol feature library and automatically loading the corresponding protocol parsing driver module includes: S131. Calculate the similarity between the extracted feature information and each predefined protocol feature template in the protocol feature library to generate a candidate protocol list; S132. According to the priority of the candidate protocols, load the corresponding protocol parsing driver modules in sequence to conduct exploratory communication; S133. If the exploratory communication successfully receives valid response data that conforms to the protocol specification, then confirm the protocol type and bind the corresponding protocol parsing driver module; if it fails, then try the next candidate protocol in the candidate protocol list.

[0009] By generating a candidate protocol list through similarity calculation and probing it according to priority, the inefficiency of blind protocol testing is avoided, and the recognition accuracy and adaptation speed of multi-protocol devices are greatly improved. Even when facing devices with unknown protocols, log retention can provide a basis for subsequent adaptation, solving the problems of poor compatibility and low success rate of traditional protocol adaptation.

[0010] As a further limitation of the technical solution of the present invention, in S13, the step of automatically inferring and constructing the three-level topology includes: S134. Parse the device attribute information to determine whether the device belongs to the site-level, loop-level, or terminal-level device. S135. Based on the subordination rules of identifiers or communication addresses in the device attribute information, infer the connection relationship between devices; S136. Based on the hierarchy and connection relationships of all devices, generate a three-level topology in the form of a graph structure or a hierarchical tree.

[0011] By using the hierarchical rules of device attributes and communication addresses, the automatic hierarchical division and connection mapping of the site-loop-end device is realized, forming a visualized topology map or hierarchical tree, making the device networking relationship of the energy site clear at a glance, and providing accurate topology data support for subsequent differentiated data collection and hierarchical autonomous control.

[0012] As a further limitation of the technical solution of the present invention, step S2 includes: S21. Read the digital image of the site equipment and analyze the type, parameter category, and topological position of each equipment in the three-level topology; S22. According to the preset service level agreement, set the baseline sampling rate for different parameters and mark the parameters as high or low priority. S23. Monitor the network link status between edge computing nodes and the cloud in real time, obtain the current available bandwidth, round-trip latency and packet loss rate indicators, and evaluate the real-time network carrying capacity. S24. Combining equipment information, service level agreements, and real-time network carrying capacity, dynamically calculate and issue actual sampling rate instructions for each parameter; S25. Collect the raw data stream according to the instruction, and inject it into the corresponding queue according to the priority flag; S26. Perform time window aggregation or lossy / lossless compression on data in the low-priority local aggregation queue.

[0013] By employing a two-factor sampling rate calculation logic that dynamically adjusts the service SLA baseline sampling rate and network carrying capacity, precise matching of sampling frequency with service importance and network status is achieved.

[0014] By dividing data into high / low priority queues and aggregating and compressing low-priority data, the system achieves separation of high-priority and low-priority data transmission. High-priority abnormal data can preempt bandwidth for immediate reporting, and steady-state data is significantly reduced in transmission volume after aggregation and compression. This reduces the amount of uplink data without affecting critical business operations and alleviates the transmission pressure of massive data in distributed energy scenarios.

[0015] As a further limitation of the technical solution of the present invention, step S3 includes: S31. Read raw data from the high-priority real-time processing queue and read data from the low-priority local aggregation queue. If it is a compressed data packet, perform decompression. If it is aggregated data, extract the raw data points or statistical features contained therein. S32. At the edge computing node, a pre-set primary calculation model is invoked, and based on the data obtained from S31, power calculation is performed locally to generate key quantities for characterizing the energy state; the power calculation includes at least one of three-phase voltage / current RMS value calculation, active / reactive power calculation, power factor calculation, harmonic analysis, phase-locked loop tracking, and power trend calculation. S33. Based on the key quantities, generate derived quantities that reflect equipment energy efficiency, load characteristics, or operational anomalies through predefined derived calculation rules; S34. The key quantities and derived quantities are spatiotemporally aligned with the context data from environmental sensors to construct a timestamp-synchronized multivariate joint analysis dataset. S35. Perform quality governance on the multivariate joint analysis dataset, including: detecting and labeling missing values, anomalous jumps and outliers, performing data imputation, time-scale correction and smoothing, and desensitizing sensitive fields involving privacy; S36. Based on the treated multivariate joint analysis dataset, apply threshold judgment to perform multivariate joint analysis, identify abnormal states that do not conform to the physical laws of equipment operation, and generate abnormal alarms with preliminary cause inferences. S37. Encapsulate the processing results of steps S35 and S36, including key quantities, derived quantities, data quality identifiers and anomaly identifiers, into standardized data objects.

[0016] By offloading the three-phase effective value, harmonic analysis and other power calculations to the edge, key quantities and derived quantities are directly generated, avoiding the massive uploading of original sampling data, greatly reducing the computing and storage pressure on the cloud, and at the same time realizing low-latency output of core energy indicators, meeting the latency requirements of real-time monitoring and control. By conducting multivariate joint analysis of environmental context data, combined with governance methods such as data interpolation, time-stamping correction, and desensitization, we not only solved the problem of high false alarm rate in traditional single-threshold detection, but also improved the credibility and standardization of the data, reduced the workload of secondary data cleaning in the cloud, and provided high-quality data support for upper-level energy management and decision-making. By generating abnormal alarms with cause inference, early detection and location of energy equipment failures are achieved, helping maintenance personnel to quickly troubleshoot problems, reducing the troubleshooting cost and processing time of equipment failures in remote scenarios such as highways, and improving the operation and maintenance efficiency of energy systems.

[0017] As a further limitation of the technical solution of the present invention, step S4 includes: S41. Continuously monitor the network connection status between edge computing nodes and the cloud management platform; when the network quality index is higher than the preset normal threshold, it is determined to be in normal network mode; when the network is completely interrupted or the quality index is lower than the preset fault threshold, it is determined to be in network interruption mode. S42. Under normal network conditions, standardized data objects are uploaded via the edge-cloud collaborative transmission mechanism. S43. In network interruption mode, trigger the offline autonomous control routine; S44. After the network is restored, perform breakpoint resume and status reconciliation based on topology priority.

[0018] As a further limitation of the technical solution of the present invention, the execution logic of the offline autonomous control routine includes: S431. Load the local security policies and safeguard list related to the current topology; S432. Based on the standardized data objects generated by S3, assess the operating status of each energy subsystem within the current site; S433. Based on the assessment results and strategies, generate control commands for energy storage systems, adjustable loads, and distributed power sources, and issue them for execution to maintain the continuity and security of core energy services.

[0019] By using topology-based local security policies and energy subsystem status assessments, control commands such as energy storage charging and discharging and load tiering can be automatically generated when the network is interrupted. This ensures the continuous operation of core energy services such as charging piles and tunnel lighting, minimizing the impact of network interruptions on the energy system.

[0020] As a further limitation of the technical solution of the present invention, in S44, the steps of performing breakpoint resumption and status reconciliation based on topology priority include: S441. After the network is restored, the edge computing nodes initiate a synchronization handshake with the cloud management platform. Taking the nodes in the three-level topology as the basic unit, the last successful upload timestamp / sequence number of the cached data on the edge side is compared with the last confirmed reception timestamp / sequence number recorded in the cloud. The data difference segments under each node are identified, and a data retransmission task is generated for each node. S442. Assign a topology priority weight to each data retransmission task; among which, the weight of the site-level node is the highest, followed by the loop-level node, and the weight of the end device-level node is the lowest; based on the topology priority weight, determine the execution order of each retransmission task and allocate the corresponding uplink bandwidth quota. S443. Execute the retransmission tasks of each node in the order of execution. For each task, starting from the breakpoint of the data in that node, transmit the difference data segment to the cloud using the allocated bandwidth quota. During the transmission process, verify and confirm each data packet. S444. Reconcile the sequence of control actions and key state snapshots executed on the edge side during the network outage with the cloud, and update the state images of both ends.

[0021] Bandwidth is allocated and retransmissions are performed based on the priority of site level, loop level, and end device level, ensuring the priority synchronization of core topology node data. CRC32 checksum and status reconciliation are used to achieve consistency of data and operating status between the edge and the cloud, avoiding data loss or status confusion, and solving the problems of disordered and easily missed traditional breakpoint resume transmission.

[0022] Secondly, the present invention also provides an energy data acquisition and processing system based on edge computing, used to implement the method described in the first aspect, wherein the system includes edge computing nodes deployed on the energy site side; the edge computing node includes: The device image building module includes: A unified abstraction layer and protocol conversion engine are used to automatically scan the network, identify and build a three-level topology, and automatically identify the device type and data point table when a new device is connected. The image management unit is used to generate and maintain digital images of site equipment that include the three-level topology, device attributes, and data point table. The strategic data acquisition and preprocessing module includes: The strategy configuration unit is used to determine the baseline sampling rate and priority label of various types of data based on the digital image of the site equipment and the preset service level agreement. The network sensing unit is used to monitor network status and assess network carrying capacity in real time. A dynamic sampling scheduler is used to dynamically calculate and issue actual sampling rate instructions by combining the output of the policy configuration unit and the evaluation results of the network perception unit. The data acquisition and distribution unit is used to acquire raw data streams according to the instructions and to perform priority queue management; The data preprocessing unit is used to perform aggregation or compression on data in the low-priority local aggregation queue; The edge intelligence processing module includes: A data preprocessing interface is used to read data from the strategy-based acquisition and preprocessing module and to unpack or restore compressed or aggregated data. The core computing engine performs power calculations based on preprocessed data to generate key quantities; A derivative analysis engine is used to generate derivative quantities based on the key quantities; The data fusion and governance unit is used to perform spatiotemporal alignment, quality governance, and anomaly detection of key quantities, derived quantities, and environmental data; The result encapsulation unit is used to encapsulate the processing results into standardized data objects containing key quantities, derived quantities, data quality identifiers, and anomaly identifiers. The edge-cloud collaboration and autonomous control module includes: The network status monitoring and mode decision unit is used to continuously monitor the network connection status and switch between normal network mode and network interruption mode. The edge-cloud collaborative transmission engine is used to upload the standardized data objects to the cloud management platform under normal network conditions. An offline autonomous control engine is used to perform offline control based on local security policies and the standardized data objects in network interruption mode; The intelligent synchronization unit is used to perform breakpoint resume and status reconciliation based on topology priority after the network is restored.

[0023] As can be seen from the above technical solutions, this application has the following advantages: This application possesses the ability to adaptively identify multi-source heterogeneous devices and construct digital images, completely changing the outdated mode of traditional system integration that relies heavily on manual protocol configuration and point-to-point mapping, greatly reducing deployment difficulty and subsequent maintenance costs. It can dynamically adjust the collection strategy according to real-time business needs and network conditions. This resolves the contradiction between massive data and limited bandwidth, ensures the priority of collecting and uploading critical event data, and guarantees the real-time performance and reliability of the system.

[0024] By employing edge-side on-site computing, offline autonomous control, and breakpoint resume mechanisms, the system overcomes the strong dependence on network continuity inherent in traditional cloud-based centralized processing. During network outages, edge nodes can autonomously maintain core energy services; upon network recovery, they can achieve precise synchronization of data and operational status, upgrading the energy system from unipolar cloud management to edge-cloud collaborative autonomy. This reduces system operational risks under network failures and ensures the continuity and security of energy supply in critical scenarios such as highway tunnels and service areas. Attached Figure Description

[0025] To more clearly illustrate the technical solution of this application, the accompanying drawings used in the description will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0026] Figure 1 This is a flowchart illustrating the method provided in an embodiment of the present invention.

[0027] Figure 2 A block diagram of a system provided in an embodiment of the present invention. Detailed Implementation

[0028] To make the purpose, features, and advantages of this application more apparent and understandable, specific embodiments and accompanying drawings will be used to clearly and completely describe the technical solution protected by this application. Obviously, the embodiments described below are only some embodiments of this application, and not all embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0029] Unless otherwise defined, all technical and scientific terms used in this application have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used in this application and in the specification of this invention is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention.

[0030] like Figure 1 As shown, this embodiment of the invention provides a method for energy data acquisition and processing based on edge computing, which performs the following operations at the edge computing node deployed on the energy site side: S1. By constructing a unified abstraction layer and an adaptive protocol conversion engine, the network is automatically scanned, and a three-level topology structure from site, loop to end device is identified and constructed. When a new device is connected, the device type and data point table are automatically identified and included in the management, and a digital image of the site device containing the three-level topology structure, device attributes and data point table is generated and maintained. This step specifically includes: S11. Deploy a unified abstraction layer on the edge computing node. The unified abstraction layer is pre-configured with protocol parsing driver modules for multiple industrial protocols. Based on the unified abstraction layer, construct an adaptive protocol conversion engine. S12. By actively probing or listening to network packets, the network within the energy site is scanned to identify the communication addresses of online devices and capture their response packets to extract protocol-related feature information. S13. For each device identified in S12, the adaptive protocol conversion engine matches the extracted feature information with a pre-set protocol feature library, automatically loads the corresponding protocol parsing driver module, and then reads the device attribute information; based on the logical relationship between the device attribute information and the communication address, it automatically infers and constructs a three-level topology structure from the site, loop to the end device; the steps of matching the extracted feature information with the pre-set protocol feature library and automatically loading the corresponding protocol parsing driver module in this step include: S131. Calculate the similarity between the extracted feature information and each predefined protocol feature template in the protocol feature library to generate a candidate protocol list; S132. According to the priority of the candidate protocols, load the corresponding protocol parsing driver modules in sequence to conduct exploratory communication; S133. If the exploratory communication successfully receives valid response data that conforms to the protocol specification, then confirm the protocol type and bind the corresponding protocol parsing driver module; if it fails, then try the next candidate protocol in the candidate protocol list.

[0031] The steps for automatically inferring and constructing a three-level topology include: S134. Parse the device attribute information to determine whether the device belongs to the site-level, loop-level, or terminal-level device. S135. Based on the subordination rules of identifiers or communication addresses in the device attribute information, infer the connection relationship between devices; S136. Based on the hierarchy and connection relationships of all devices, generate a three-level topology in the form of a graph structure or a hierarchical tree.

[0032] S14. Based on the three-level topology, the data point structure of each device is automatically parsed through the loaded protocol parsing driver module; for each parsed raw data point, a corresponding unified variable identifier is generated, thereby forming the data point mapping relationship set of the device. The data point mapping relationship set defines the correspondence between the raw data points and the unified variable identifier; the device and its data point mapping relationship set are registered in the initial site device digital image. S15. When a new device is subsequently detected to be connected, repeat steps S12 to S14 to generate a corresponding data point mapping relationship set for the new device, and dynamically update the new device and its data point mapping relationship set to the digital image of the site device.

[0033] Within the software framework of edge computing nodes, a unified device access interface is designed, and protocol parsing driver modules for mainstream industrial protocols such as Modbus, IEC 61850, and OPC UA are developed or integrated. Based on this, an adaptive protocol conversion engine is constructed. This engine has a built-in protocol feature library that stores feature templates for various protocols, such as: Modbus TCP: Characterized by TCP port 502, response messages conform to the format [transaction identifier][protocol identifier][length][unit identifier][function code]...

[0034] IEC 61850 MMS: Characterized by TCP port 102, using ASN.1 BER encoding.

[0035] After the engine starts, it performs network discovery. For example, it sends ICMP Ping or a broadcast frame of a specific protocol to subnet 192.168.1.0 / 24. For responding devices, it captures their first response packet and extracts the feature vector F=(protocol_port,packet_header,response_delay).

[0036] The extracted feature vector F is compared with the template in the feature library. Perform similarity calculation:

[0037] Protocols with a similarity higher than a threshold are selected as candidates. The candidate protocol drivers are then loaded sequentially for testing. For example, when loading the Modbus driver, a request to read the holding register is sent to device address 1. If a valid response frame is received, the protocol binding is confirmed to be successful.

[0038] After binding the driver, read the device attribute information A (such as device name, model, and manufacturer identifier). Based on the identifier in A and the device's IP address, infer the hierarchical relationship.

[0039] Example of topology inference rules: If the device model is a station-level monitoring unit, it is marked as a station-level device.

[0040] If A contains a parentDeviceID field, then a parent-child connection is established directly.

[0041] If the IP addresses are 192.168.1.101-110 and belong to the same VLAN, then it is inferred that they belong to the same circuit, and their parent node is the gateway device (site level) with IP address 192.168.1.1.

[0042] Finally, a three-level topology G=(V,E) is generated, stored in the form of an adjacency list or attribute graph, where V is a vertex (device) and E is an edge (connection).

[0043] For each device, its data point table is read via the bound driver. For example, for a Modbus device, its predefined register mapping table is read; for an IEC 61850 device, its SCD file is read or the DataObject under the LogicalNode is obtained via the MMS service.

[0044] For each raw data point Generate a uniform variable identifier Establish a mapping relationship set. .

[0045] Device Node Its attributes and mapping relationship set Register to the site's digital image of the device. This image can be stored as nodes and relationships in JSON or a graph database.

[0046] S2. Based on the digital image of the site equipment and the preset service level agreement, and combined with the real-time network status, dynamically determine the sampling rate for various types of energy monitoring data; prioritize the collected raw data streams, dividing them into a high-priority immediate processing queue and a low-priority local aggregation queue, and perform aggregation or compression on the data in the low-priority local aggregation queue; this step specifically includes: S21. Read the digital image of the site equipment and analyze the type, parameter category, and topological position of each equipment in the three-level topology; S22. According to the preset service level agreement, set the baseline sampling rate for different parameters and mark the parameters as high or low priority. S23. Monitor the network link status between edge computing nodes and the cloud in real time, obtain the current available bandwidth, round-trip latency and packet loss rate indicators, and evaluate the real-time network carrying capacity. S24. Combining equipment information, service level agreements, and real-time network carrying capacity, dynamically calculate and issue actual sampling rate instructions for each parameter; S25. Collect the raw data stream according to the instruction, and inject it into the corresponding queue according to the priority flag; S26. Perform time window aggregation or lossy / lossless compression on data in the low-priority local aggregation queue.

[0047] Configure SLAs for different parameters based on digital imaging. For example: High priority: Voltage sag (SLA_target_rate=100Hz, priority=1) Low priority: Indoor temperature (SLA_target_rate=1 / 300Hz, priority=5) Use the ping command or TCP connection test to measure the round-trip time (RTT) and packet loss rate (Loss) in the cloud. Available bandwidth It can be estimated using the following formula:

[0048] This represents the total number of bytes (or bits) of all data packets sent within the test window. This represents the number of packets successfully received within the test window (the number of ACK packets). The duration of the test window (in seconds), and Loss is the packet loss rate, (1 Loss is a probability correction factor for successful transmission.

[0049] Define a dynamic sampling rate adjustment function. When network bandwidth is sufficient, the SLA target sampling rate is adopted. When bandwidth is limited, the sampling rate for low-priority data is reduced using the following formula. :

[0050] in The adjustment coefficient (0 < <1), The target sampling rate for low-priority data under ideal network conditions is set as the preset target sampling rate. Currently available bandwidth, High-priority data is preserved based on bandwidth threshold. .

[0051] according to Data is collected periodically. After collection, the data is placed into the corresponding memory queue according to the priority flag in the SLA. For data in the low-priority queue, when the time window or data volume threshold is reached, aggregation or lossless / lossy compression using algorithms such as ZIP or LZ4 is performed.

[0052] S3. Perform on-site computation and fusion analysis on the raw data from the high-priority real-time processing queue of S2 and the aggregated or compressed data from the low-priority local aggregation queue to generate standardized data objects containing key quantities, derived quantities, data quality identifiers, and anomaly identifiers. This step specifically includes: S31. Read raw data from the high-priority real-time processing queue and read data from the low-priority local aggregation queue. If it is a compressed data packet, perform decompression. If it is aggregated data, extract the raw data points or statistical features contained therein. S32. At the edge computing node, a pre-set primary calculation model is invoked, and based on the data obtained from S31, power calculation is performed locally to generate key quantities for characterizing the energy state; the power calculation includes at least one of three-phase voltage / current RMS value calculation, active / reactive power calculation, power factor calculation, harmonic analysis, phase-locked loop tracking, and power trend calculation. S33. Based on the key quantities, generate derived quantities that reflect equipment energy efficiency, load characteristics, or operational anomalies through predefined derived calculation rules; S34. The key quantities and derived quantities are spatiotemporally aligned with the context data from environmental sensors to construct a timestamp-synchronized multivariate joint analysis dataset. S35. Perform quality governance on the multivariate joint analysis dataset, including: detecting and labeling missing values, anomalous jumps and outliers, performing data imputation, time-scale correction and smoothing, and desensitizing sensitive fields involving privacy; S36. Based on the treated multivariate joint analysis dataset, apply threshold judgment to perform multivariate joint analysis, identify abnormal states that do not conform to the physical laws of equipment operation, and generate abnormal alarms with preliminary cause inferences. S37. Encapsulate the processing results of steps S35 and S36, including key quantities, derived quantities, data quality identifiers and anomaly identifiers, into standardized data objects.

[0053] Read raw sampled values ​​from the high-priority queue. Compressed packages are read from the low-priority queue and decompressed using the corresponding algorithm; if the data is aggregated, its statistical features are extracted directly.

[0054] For electricity data, standard calculations are performed: Three-phase active power:

[0055] , , These are the RMS effective values ​​of the three-phase voltages (unit: V). , , These are the RMS effective values ​​of the three-phase currents (unit: A). , , These are the phase differences between the voltage and current of each phase (unit: radians or degrees), and P is the total active power (unit: W or kW).

[0056] Harmonic distortion rate:

[0057] This is the effective value of the fundamental voltage. The effective value of the h-th harmonic voltage. To analyze the maximum harmonic order, Total harmonic distortion (%).

[0058] Phase-locked loop (PLL): An algorithm based on a second-order generalized integrator (SOGI) is used to analyze the grid voltage signal (input grid voltage signal). Real-time tracking of phase angle (SOGI output phase angle) and frequency estimation .

[0059]

[0060] The generated derivatives include: Equipment energy efficiency

[0061] The useful power output by the device. The electrical power input to the device.

[0062] load rate

[0063] This refers to the actual operating power of the equipment. This refers to the rated power of the equipment.

[0064] Health score

[0065] , These are weighting coefficients, set based on equipment importance and historical data. Points are deducted for exceeding harmonic limits, based on Calculation of the degree of exceeding the limit The overload penalty is calculated based on the load rate or the degree of power exceeding the limit.

[0066] P, , , Key quantities and derived quantities, and their relationship with environmental temperature and humidity. , Perform timestamp alignment (interpolation alignment to the same time series) ). Building the dataset .

[0067] right Implementing data governance: Missing value imputation: linear interpolation is used. .

[0068] , These are measurements taken at adjacent known times. , For the corresponding time points, linear interpolation is used to fill in the missing time points, ensuring the data integrity of the multivariate joint analysis.

[0069] Abnormal transition detection: If If true, it is marked as a jump.

[0070] De-identification: Hash fields such as the device's unique identifier.

[0071] Threshold rules for anomaly detection: If The abnormal alarm is "overload or harmonic exceedance". Physical law verification: For photovoltaic inverters, at night, when the light intensity is 0, its output power should be 0; if it is not 0, an alarm will be triggered. Correlation analysis: On the same circuit, the total power should be approximately equal to the sum of the power of each branch. If the deviation continues to exceed the limit, an alarm will be triggered indicating metering deviation or leakage.

[0072] Clean dataset Quality indicators and exception alerts are encapsulated as standardized data objects conforming to a predefined JSON Schema.

[0073] S4. Based on the three-level topology constructed in S1 and the standardized data object generated in S3, perform the following operations: When the network is functioning normally, the standardized data objects are uploaded to the cloud management platform via an edge-cloud collaborative transmission mechanism; In the event of a network outage, offline control is executed based on locally preset security policies associated with the three-tier topology and real-time data generated by S3 to maintain the operation of core energy services. After network recovery, a protocol based on three-level topology node IDs for resuming interrupted transmissions and reconciling status is used to synchronize cached data and operational status. Specifically, this includes: S41. Continuously monitor the network connection status between edge computing nodes and the cloud management platform; when the network quality index is higher than the preset normal threshold, it is determined to be in normal network mode; when the network is completely interrupted or the quality index is lower than the preset fault threshold, it is determined to be in network interruption mode. S42. Under normal network conditions, standardized data objects are uploaded via the edge-cloud collaborative transmission mechanism. S43. In network interruption mode, trigger the offline autonomous control routine; including: S431. Load the local security policies and safeguard list related to the current topology; S432. Based on the standardized data objects generated by S3, assess the operating status of each energy subsystem within the current site; S433. Based on the assessment results and strategies, generate control commands for energy storage systems, adjustable loads, and distributed power sources, and issue them for execution to maintain the continuity and security of core energy services.

[0074] S44. After the network recovers, perform breakpoint resumption and status reconciliation based on topology priority. Specifically, this includes: S441. After the network is restored, the edge computing nodes initiate a synchronization handshake with the cloud management platform. Taking the nodes in the three-level topology as the basic unit, the last successful upload timestamp / sequence number of the cached data on the edge side is compared with the last confirmed reception timestamp / sequence number recorded in the cloud. The data difference segments under each node are identified, and a data retransmission task is generated for each node. S442. Assign a topology priority weight to each data retransmission task; among which, the weight of the site-level node is the highest, followed by the loop-level node, and the weight of the end device-level node is the lowest; based on the topology priority weight, determine the execution order of each retransmission task and allocate the corresponding uplink bandwidth quota. S443. Execute the retransmission tasks of each node in the order of execution. For each task, starting from the breakpoint of the data in that node, transmit the difference data segment to the cloud using the allocated bandwidth quota. During the transmission process, verify and confirm each data packet. S444. Reconcile the sequence of control actions and key state snapshots executed on the edge side during the network outage with the cloud, and update the state images of both ends.

[0075] Continuously monitor network status . Boolean value, True indicates the link is connected, False indicates it is disconnected; Define threshold: .

[0076] like Then it will enter normal mode.

[0077] like If the value is False for more than 5 seconds, the system will enter interrupt mode.

[0078] In normal mode, standardized data objects are published to cloud topics, such as edge / data / SiteA, via the MQTT protocol. Priority tags are set for the data; high-priority anomaly alerts are published immediately, while regular data can be published in batches after a short buffer.

[0079] Once interrupt mode is entered, the offline control routine is triggered: Loading strategy: Based on the current topology, load pre-defined rules from the local policy library. For example, for a loop that includes energy storage (ESS) and photovoltaic (PV), the strategy is "prioritize critical loads and use PV and ESS to maintain off-grid operation".

[0080] State assessment: Based on real-time P, SOC and other data generated by S3, power balance is calculated. . Photovoltaic power generation capacity, To store the maximum discharge power, This represents the total active load power.

[0081] Control Decision-Making and Execution: If >0, and If the energy storage is insufficient, the system will control the discharge of energy; if still insufficient, non-critical loads will be cut off in stages via IO modules or smart circuit breakers according to the protection list. Control commands are sent to the corresponding devices via Modbus RTU or GOOSE messages.

[0082] After the network is restored, for each node in the topology (Device), retrieve the serial number of the last successfully uploaded data from the local cache database. And query the cloud for the node's serial number. The difference data segment is [ +1, ].

[0083] Priority scheduling: Assigning weights to tasks at the site level. loop level Equipment level Total bandwidth Weighted allocation: .in This represents the sum of the weights of the nodes currently awaiting synchronization.

[0084] Resumable download: For nodes From its breakpoint Starting with +1, The data is packaged and transmitted at a high speed. TCP is used to ensure reliability.

[0085] Status reconciliation: Control command logs and key status snapshots executed during the interruption are sent to the cloud. The cloud compares these with the expected command sequence simulated according to the global policy. If a conflict is found, a conflict report is generated, and the cloud adjudicates the conflict according to preset rules. Finally, both parties update their status images to a consistent version.

[0086] like Figure 2 As shown, this embodiment of the invention also provides an energy data acquisition and processing system based on edge computing, used to implement the methods described in the above embodiments. The system includes edge computing nodes deployed on the energy site side; the edge computing node includes: The device image building module includes: A unified abstraction layer and protocol conversion engine are used to automatically scan the network, identify and build a three-level topology, and automatically identify the device type and data point table when a new device is connected. The image management unit is used to generate and maintain digital images of site equipment that include the three-level topology, device attributes, and data point table. The strategic data acquisition and preprocessing module includes: The strategy configuration unit is used to determine the baseline sampling rate and priority label of various types of data based on the digital image of the site equipment and the preset service level agreement. The network sensing unit is used to monitor network status and assess network carrying capacity in real time. A dynamic sampling scheduler is used to dynamically calculate and issue actual sampling rate instructions by combining the output of the policy configuration unit and the evaluation results of the network perception unit. The data acquisition and distribution unit is used to acquire raw data streams according to the instructions and to perform priority queue management; The data preprocessing unit is used to perform aggregation or compression on data in the low-priority local aggregation queue; The edge intelligence processing module includes: A data preprocessing interface is used to read data from the strategy-based acquisition and preprocessing module and to unpack or restore compressed or aggregated data. The core computing engine performs power calculations based on preprocessed data to generate key quantities; A derivative analysis engine is used to generate derivative quantities based on the key quantities; The data fusion and governance unit is used to perform spatiotemporal alignment, quality governance, and anomaly detection of key quantities, derived quantities, and environmental data; The result encapsulation unit is used to encapsulate the processing results into standardized data objects containing key quantities, derived quantities, data quality identifiers, and anomaly identifiers. The edge-cloud collaboration and autonomous control module includes: The network status monitoring and mode decision unit is used to continuously monitor the network connection status and switch between normal network mode and network interruption mode. The edge-cloud collaborative transmission engine is used to upload the standardized data objects to the cloud management platform under normal network conditions. An offline autonomous control engine is used to perform offline control based on local security policies and the standardized data objects in network interruption mode; The intelligent synchronization unit is used to perform breakpoint resume and status reconciliation based on topology priority after the network is restored.

[0087] In some embodiments, the unified abstraction layer and protocol conversion engine in the device image building module specifically include: Protocol parsing driver library, with pre-built parsing driver modules for various industrial protocols; The protocol feature matching unit is used to match the device response features captured by scanning with a preset protocol feature library in order to automatically load the driver module; The topology inference unit is used to automatically infer and construct a three-level topology based on the read device attribute information and communication address relationship; The data point self-learning unit is used to automatically parse the device data point structure and generate a unified variable identifier mapping relationship.

[0088] In some embodiments, the offline autonomous control engine includes: The policy and constraint library stores a list of local security policies and safeguards associated with the three-tier topology. A real-time status evaluator is used to evaluate the operating status of each energy subsystem based on the standardized data objects. The local controller is used to generate and issue control commands for energy storage, loads, and distributed power sources based on evaluation results and a policy library.

[0089] In some embodiments, the intelligent synchronization unit includes: The difference analyzer is used to compare data differences between the edge and the cloud on a topology node basis. The priority scheduler is used to allocate priority weights and bandwidth quotas based on the topology hierarchy for data retransmission tasks of each node. A breakpoint resume controller is used to reliably transmit differing data segments in priority order; The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for energy data acquisition and processing based on edge computing, characterized in that, Perform the following operations at the edge computing nodes deployed on the energy site side: S1. By constructing a unified abstraction layer and an adaptive protocol conversion engine, the network is automatically scanned, and a three-level topology structure from site, loop to end device is identified and constructed. When a new device is connected, the device type and data point table are automatically identified and included in the management, and a digital image of the site device containing the three-level topology structure, device attributes and data point table is generated and maintained. S2. Based on the digital image of the site equipment and the preset service level agreement, and combined with the real-time network status, dynamically determine the sampling rate of various energy monitoring data, perform priority management on the collected raw data stream, divide it into a high-priority instant processing queue and a low-priority local aggregation queue, and perform aggregation or compression on the data in the low-priority local aggregation queue. S3. Perform on-site computation and fusion analysis on the raw data from the high-priority real-time processing queue of S2 and the aggregated or compressed data from the low-priority local aggregation queue to generate standardized data objects containing key quantities, derived quantities, data quality identifiers and anomaly identifiers. S4. Based on the three-level topology constructed in S1 and the standardized data object generated in S3, perform the following operations: When the network is functioning normally, the standardized data objects are uploaded to the cloud management platform via an edge-cloud collaborative transmission mechanism; In the event of a network outage, offline control is executed based on locally preset security policies associated with the three-tier topology and real-time data generated by S3 to maintain the operation of core energy services. After the network is restored, the cached data and running status are synchronized through a breakpoint resume and status reconciliation protocol based on the three-level topology node ID.

2. The energy data acquisition and processing method based on edge computing according to claim 1, characterized in that, S1 specifically includes the following steps: S11. Deploy a unified abstraction layer on the edge computing node. The unified abstraction layer is pre-configured with protocol parsing driver modules for multiple industrial protocols. Based on the unified abstraction layer, construct an adaptive protocol conversion engine. S12. By actively probing or listening to network packets, the network within the energy site is scanned to identify the communication addresses of online devices and capture their response packets to extract protocol-related feature information. S13. For each device identified in S12, the adaptive protocol conversion engine matches the extracted feature information with the preset protocol feature library, automatically loads the corresponding protocol parsing driver module, and then reads the device attribute information; based on the logical relationship between the device attribute information and the communication address, it automatically infers and constructs a three-level topology structure from the site, loop to the end device. S14. Based on the three-level topology, the data point structure of each device is automatically parsed through the loaded protocol parsing driver module; for each parsed raw data point, a corresponding unified variable identifier is generated, thereby forming the data point mapping relationship set of the device, which defines the correspondence between the raw data points and the unified variable identifier; the device and its data point mapping relationship set are registered to the initial site device digital image. S15. When a new device is subsequently detected to be connected, repeat steps S12 to S14 to generate a corresponding data point mapping relationship set for the new device, and dynamically update the new device and its data point mapping relationship set to the digital image of the site device.

3. The energy data acquisition and processing method based on edge computing according to claim 2, characterized in that, In S13, the steps of matching the extracted feature information with the preset protocol feature library and automatically loading the corresponding protocol parsing driver module include: S131. Calculate the similarity between the extracted feature information and each predefined protocol feature template in the protocol feature library to generate a candidate protocol list; S132. According to the priority of the candidate protocols, load the corresponding protocol parsing driver modules in sequence to conduct exploratory communication; S133. If the exploratory communication successfully receives valid response data that conforms to the protocol specification, then confirm the protocol type and bind the corresponding protocol parsing driver module; if it fails, then try the next candidate protocol in the candidate protocol list.

4. The energy data acquisition and processing method based on edge computing according to claim 3, characterized in that, In S13, the steps for automatically inferring and constructing the three-level topology include: S134. Parse the device attribute information to determine whether the device belongs to the site-level, loop-level, or terminal-level device. S135. Based on the subordination rules of identifiers or communication addresses in the device attribute information, infer the connection relationship between devices; S136. Based on the hierarchy and connection relationships of all devices, generate a three-level topology in the form of a graph structure or a hierarchical tree.

5. The energy data acquisition and processing method based on edge computing according to claim 1, characterized in that, The steps in S2 include: S21. Read the digital image of the site equipment and analyze the type, parameter category, and topological position of each equipment in the three-level topology; S22. According to the preset service level agreement, set the baseline sampling rate for different parameters and mark the parameters as high or low priority. S23. Monitor the network link status between edge computing nodes and the cloud in real time, obtain the current available bandwidth, round-trip latency and packet loss rate indicators, and evaluate the real-time network carrying capacity. S24. Combining equipment information, service level agreements, and real-time network carrying capacity, dynamically calculate and issue actual sampling rate instructions for each parameter; S25. Collect the raw data stream according to the instruction, and inject it into the corresponding queue according to the priority flag; S26. Perform time window aggregation or lossy / lossless compression on data in the low-priority local aggregation queue.

6. The energy data acquisition and processing method based on edge computing according to claim 1, characterized in that, The steps in S3 include: S31. Read raw data from the high-priority real-time processing queue and read data from the low-priority local aggregation queue. If it is a compressed data packet, perform decompression. If it is aggregated data, extract the raw data points or statistical features contained therein. S32. At the edge computing node, a pre-set primary calculation model is invoked, and based on the data obtained from S31, power calculation is performed locally to generate key quantities for characterizing the energy state; the power calculation includes at least one of three-phase voltage / current RMS value calculation, active / reactive power calculation, power factor calculation, harmonic analysis, phase-locked loop tracking, and power trend calculation. S33. Based on the key quantities, generate derived quantities that reflect equipment energy efficiency, load characteristics, or operational anomalies through predefined derived calculation rules; S34. The key quantities and derived quantities are spatiotemporally aligned with the context data from environmental sensors to construct a timestamp-synchronized multivariate joint analysis dataset. S35. Perform quality governance on the multivariate joint analysis dataset, including: detecting and labeling missing values, anomalous jumps and outliers, performing data imputation, time-scale correction and smoothing, and desensitizing sensitive fields involving privacy; S36. Based on the treated multivariate joint analysis dataset, apply threshold judgment to perform multivariate joint analysis, identify abnormal states that do not conform to the physical laws of equipment operation, and generate abnormal alarms with preliminary cause inferences. S37. Encapsulate the processing results of steps S35 and S36, including key quantities, derived quantities, data quality identifiers and anomaly identifiers, into standardized data objects.

7. The energy data acquisition and processing method based on edge computing according to claim 1, characterized in that, The steps in S4 include: S41. Continuously monitor the network connection status between edge computing nodes and the cloud management platform; when the network quality index is higher than the preset normal threshold, it is determined to be in normal network mode; when the network is completely interrupted or the quality index is lower than the preset fault threshold, it is determined to be in network interruption mode. S42. Under normal network conditions, standardized data objects are uploaded via the edge-cloud collaborative transmission mechanism. S43. In network interruption mode, trigger the offline autonomous control routine; S44. After the network is restored, perform breakpoint resume and status reconciliation based on topology priority.

8. The energy data acquisition and processing method based on edge computing according to claim 7, characterized in that, The execution logic of the offline autonomous control routine includes: S431. Load the local security policies and safeguard list related to the current topology; S432. Based on the standardized data objects generated by S3, assess the operating status of each energy subsystem within the current site; S433. Based on the assessment results and strategies, generate control commands for energy storage systems, adjustable loads, and distributed power sources, and issue them for execution to maintain the continuity and security of core energy services.

9. The energy data acquisition and processing method based on edge computing according to claim 8, characterized in that, In S44, the steps for performing breakpoint resumption and status reconciliation based on topology priority include: S441. After the network is restored, the edge computing nodes initiate a synchronization handshake with the cloud management platform. Taking the nodes in the three-level topology as the basic unit, the last successful upload timestamp / sequence number of the cached data on the edge side is compared with the last confirmed reception timestamp / sequence number recorded in the cloud. The data difference segments under each node are identified, and a data retransmission task is generated for each node. S442. Assign a topology priority weight to each data retransmission task; among which, the weight of the site-level node is the highest, followed by the loop-level node, and the weight of the end device-level node is the lowest; based on the topology priority weight, determine the execution order of each retransmission task and allocate the corresponding uplink bandwidth quota. S443. Execute the retransmission tasks of each node in the order of execution. For each task, starting from the breakpoint of the data in that node, transmit the difference data segment to the cloud using the allocated bandwidth quota. During the transmission process, verify and confirm each data packet. S444. Reconcile the sequence of control actions and key state snapshots executed on the edge side during the network outage with the cloud, and update the state images of both ends.

10. An energy data acquisition and processing system based on edge computing, characterized in that, For implementing the method of any one of claims 1 to 9, the system includes edge computing nodes deployed on the side of energy sites; The edge computing nodes include: The device image building module includes: A unified abstraction layer and protocol conversion engine are used to automatically scan the network, identify and build a three-level topology, and automatically identify the device type and data point table when a new device is connected. The image management unit is used to generate and maintain digital images of site equipment that include the three-level topology, device attributes, and data point tables. The strategic data acquisition and preprocessing module includes: The strategy configuration unit is used to determine the baseline sampling rate and priority label of various types of data based on the digital image of the site equipment and the preset service level agreement. The network sensing unit is used to monitor network status and assess network capacity in real time. A dynamic sampling scheduler is used to dynamically calculate and issue actual sampling rate instructions by combining the output of the policy configuration unit and the evaluation results of the network perception unit. The data acquisition and distribution unit is used to acquire raw data streams according to the instructions and to perform priority queue management; The data preprocessing unit is used to perform aggregation or compression on data in the low-priority local aggregation queue; The edge intelligence processing module includes: A data preprocessing interface is used to read data from the strategy-based acquisition and preprocessing module and to unpack or restore compressed or aggregated data. The core computing engine performs power calculations based on preprocessed data to generate key quantities; A derivative analysis engine is used to generate derivative quantities based on the key quantities; The data fusion and governance unit is used to perform spatiotemporal alignment, quality governance, and anomaly detection of key quantities, derived quantities, and environmental data; The result encapsulation unit is used to encapsulate the processing results into standardized data objects containing key quantities, derived quantities, data quality identifiers, and anomaly identifiers. The edge-cloud collaboration and autonomous control module includes: The network status monitoring and mode decision unit is used to continuously monitor the network connection status and switch between normal network mode and network interruption mode. The edge-cloud collaborative transmission engine is used to upload the standardized data objects to the cloud management platform under normal network conditions. An offline autonomous control engine is used to perform offline control based on local security policies and the standardized data objects in network interruption mode; The intelligent synchronization unit is used to perform breakpoint resume and status reconciliation based on topology priority after the network is restored.

Images