An efficient data processing method for cloud-edge collaborative interaction

Abstract

The application discloses a kind of cloud edge collaborative interaction efficient data processing method, belong to electric power system data processing technical field, the method includes the data processing request of cloud end receiving power grid dispatching system, analysis data type and response time delay requirement, in combination with the real-time load state of each edge node periodicity report, request is decomposed into several subtasks and is matched corresponding edge node;Each edge node is preprocessed after local measurement data and is reported to cloud end;Cloud end is combined with power grid topological model and identifies topological correlation cross-node subtask, issues collaborative computing instruction;Each edge node completes cross-node data interaction and collaborative computing and reports result;Cloud end merges distributed computing result and carries out multidimensional check, and it is output to power grid dispatching system if check passes, and the application is distributed by cloud edge collaborative architecture The distributed processing of power grid data is realized, and processing efficiency and reliability are effectively improved.

Description

Technical Field

[0001] This invention relates to the field of power system data processing technology, and in particular to a high-efficiency data processing method for cloud-edge collaborative interaction. Background Technology

[0002] Smart grid dispatching systems require real-time processing of massive amounts of measurement data. Traditional centralized cloud computing architectures result in high cloud loads, significant backbone network bandwidth pressure, and latency issues that fail to meet dispatching requirements. While edge computing can reduce transmission volume and cloud load, existing solutions treat each edge node as an independent unit, lacking cross-node collaboration mechanisms tailored to grid topology relationships. Grid dispatching operations exhibit significant topological interdependencies, making it difficult for a single edge node to independently complete cross-domain computing tasks. Furthermore, existing task allocation does not adequately consider real-time node load and topology, leading to load imbalances and compromising processing accuracy and timeliness. Summary of the Invention

[0003] To address the shortcomings of existing technologies, this invention provides an efficient data processing method for cloud-edge collaborative interaction.

[0004] This invention provides an efficient data processing method for cloud-edge collaborative interaction, applicable to systems including a cloud and multiple edge nodes. The method includes the following steps: S1. The cloud receives data processing requests from the power grid dispatching system, parses the data type and response latency requirements in the data processing requests, and obtains the real-time load status periodically reported by each edge node; based on the data type, response latency requirements, and real-time load status, the data processing requests are decomposed into several sub-tasks, and each sub-task is matched with a corresponding edge node to generate an edge task orchestration scheme. S2. Each edge node obtains the corresponding sub-task according to the edge task orchestration scheme, determines the local measurement data range to be processed according to the sub-task, preprocesses the measurement data, generates preprocessed measurement data, and reports it to the cloud. S3. Based on the preprocessed measurement data reported by each edge node, the cloud identifies cross-node subtasks with grid topology associations in conjunction with the grid topology model, and issues collaborative computing instructions to the corresponding edge nodes. The collaborative computing instructions include collaborative algorithm identifiers and data interaction rules between nodes. S4. Each edge node completes the interaction and collaborative calculation of cross-node power grid data according to the collaborative calculation instructions, generates distributed calculation results and reports them to the cloud; S5 merges the distributed computing results of each edge node in the cloud to obtain the complete power grid processing result and performs verification; if the verification passes, it is output to the power grid dispatch system; if the verification fails, the reason for the failure is output.

[0005] Preferably, step S1 specifically includes the following sub-steps: S11. The cloud receives and parses the data processing request to obtain the task type and the maximum allowed processing latency. S12. The cloud obtains the real-time load status periodically reported by each edge node, judges the timeliness of each load status, and removes load status data that exceeds the preset timeliness threshold. S13. The cloud determines the range of power grid equipment to be processed based on the task type, and divides the range into multiple sub-task data domains in combination with the power grid topology; S14. Based on the maximum processing latency and the real-time load status of each edge node, the cloud allocates an edge node to each subtask data domain and generates an edge task orchestration scheme that includes the correspondence between subtasks and edge nodes.

[0006] Preferably, the real-time load status includes: The edge node's CPU utilization, memory usage, network bandwidth usage, queue length of pending tasks, and estimated task completion time.

[0007] Preferably, step S2 specifically includes the following sub-steps: S21. Edge nodes obtain subtasks corresponding to themselves from the edge task orchestration scheme, extract the required measurement data and perform preprocessing, including data cleaning, format standardization and time scale alignment. S22. Edge nodes report the preprocessed measurement data to the cloud.

[0008] Preferably, step S3 specifically includes the following sub-steps: S31. The cloud-based system combines a power grid topology model to analyze the electrical connection relationships between power grid equipment involved in each sub-task. The electrical connection relationships include direct electrical connections and indirect electrical connections within a preset topology hop count threshold range. S32. When it is detected that subtasks belonging to different edge nodes are related in the topology and the electrical impedance of the related path is less than a preset impedance threshold, these subtasks are marked as a collaborative computing group. S33. The cloud selects the corresponding collaborative algorithm from the preset collaborative algorithm library according to the task type of the data processing request, generates a collaborative computing instruction and sends it to each edge node in the collaborative computing group. The collaborative computing instruction specifies the data to be interacted between nodes, the data interaction sequence and the collaborative algorithm parameters.

[0009] Preferably, step S4 specifically includes the following sub-steps: S41. According to the data interaction rules between nodes in the collaborative computing instructions, the edge node establishes a secure communication channel with other nodes in the group, sends the measurement data that needs to be shared locally, and receives the measurement data shared by other nodes. S42. The edge node executes the distributed collaborative algorithm specified in the collaborative computing instruction based on the local measurement data and the received shared measurement data to obtain the distributed computing result. S43. The edge node reports the distributed computing results to the cloud.

[0010] Preferably, step S5 specifically includes the following sub-steps: S51: Cloud-based concatenation of distributed computing results belonging to the same collaborative computing group yields a global processing result; S52. The cloud performs multi-dimensional verification on the global processing result, including: data integrity verification, power grid topology constraint verification, and end-to-end processing latency verification. S53. If all the multi-dimensional verifications pass, the global processing result is output to the power grid dispatching system; S54. If the verification fails, output the reason for the failure, adjust the subtask division method or matching strategy, and return to step S1 to re-execute.

[0011] Preferably, the end-to-end processing delay verification is as follows: Calculate the actual processing latency from receiving a data processing request from the cloud to generating a global processing result, and determine whether the actual processing latency is less than the maximum allowed processing latency; if the actual processing latency is greater than or equal to the maximum processing latency, the latency check fails.

[0012] This invention discloses a highly efficient data processing method for cloud-edge collaborative interaction, which has the following beneficial effects: By combining request latency requirements with the real-time load status of edge nodes to decompose tasks and match nodes, we can avoid node overload or idleness and improve system resource utilization.

[0013] Data preprocessing is performed locally at edge nodes before being reported, reducing the amount of data uploaded; computing tasks are offloaded to the edge, reducing the pressure on centralized computing in the cloud.

[0014] Based on the power grid topology model, cross-node related sub-tasks are identified, and data is shared through secure channels and distributed collaborative algorithms are executed to solve the problem that a single node cannot handle cross-domain related business.

[0015] Dimensional validation ensures correct results; if validation fails, the task partitioning strategy is automatically adjusted and re-executed to improve system reliability. Attached Figure Description

[0016] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments of the present invention are provided below.

[0017] Figure 1A flowchart illustrating an efficient data processing method for cloud-edge collaborative interaction provided by this invention; Detailed Implementation

[0018] The technical solution of the present invention will be described in detail below with reference to the accompanying drawings and specific embodiments, so that those skilled in the art can better understand the present invention and implement it. However, the embodiments are not intended to limit the present invention.

[0019] This invention provides a highly efficient data processing method for cloud-edge collaborative interaction, applicable to a distributed system comprising a cloud server and multiple edge computing nodes. The cloud server is deployed at the power grid dispatch center, responsible for global task scheduling, issuing collaborative computing instructions, and verifying results; the edge computing nodes are deployed at various substations or distribution terminals, responsible for collecting, preprocessing, and distributing collaborative computing of local measurement data.

[0020] refer to Figure 1 This invention provides an efficient data processing method for cloud-edge collaborative interaction, characterized by its applicability to a system including a cloud and multiple edge nodes, and the method comprising the following steps: S1. The cloud receives data processing requests from the power grid dispatching system, parses the data type and response latency requirements in the data processing requests, and obtains the real-time load status periodically reported by each edge node; based on the data type, response latency requirements, and real-time load status, the data processing requests are decomposed into several sub-tasks, and each sub-task is matched with a corresponding edge node to generate an edge task orchestration scheme. S2. Each edge node obtains the corresponding sub-task according to the edge task orchestration scheme, determines the range of measurement data to be processed locally according to the sub-task, preprocesses the measurement data, generates preprocessed measurement data and reports it to the cloud. S3. Based on the preprocessed measurement data reported by each edge node, the cloud identifies cross-node subtasks with grid topology associations in conjunction with the grid topology model, and issues collaborative computing instructions to the corresponding edge nodes. The collaborative computing instructions include collaborative algorithm identifiers and data interaction rules between nodes. S4. Each edge node completes the interaction and collaborative computing of cross-node power grid data according to the collaborative computing instructions, generates distributed computing results and reports them to the cloud; S5 merges the distributed computing results of each edge node in the cloud to obtain the complete power grid processing result and performs verification; if the verification passes, it is output to the power grid dispatch system; if the verification fails, the reason for the failure is output.

[0021] In the preferred embodiment, step S1 specifically includes the following sub-steps: S11. The cloud receives and parses the data processing request to obtain the task type and the maximum allowed processing latency. S12. The cloud obtains the real-time load status periodically reported by each edge node, and judges the timeliness of each load status, and removes load status data that exceeds the preset timeliness threshold; the real-time load status includes the edge node's CPU utilization, memory usage, network bandwidth usage, queue length of pending tasks, and estimated task completion time.

[0022] S13. The cloud determines the scope of power grid equipment to be processed based on the task type, and divides the scope into multiple sub-task data domains based on the power grid topology. S14. Based on the maximum processing latency and the real-time load status of each edge node, the cloud allocates an edge node to each subtask data domain and generates an edge task orchestration scheme that includes the correspondence between subtasks and edge nodes.

[0023] Specifically, the cloud receives data processing requests from the power grid dispatching system via methods such as MQTT or HTTP / 2. These requests include task identifiers, data types such as state estimation, power flow calculation, fault location or load forecasting, a list of power grid equipment to be processed, and the maximum allowed processing delay.

[0024] The cloud maintains an edge node status table. Each edge node reports its real-time load status to the cloud at a preset interval, such as every 5 seconds. This includes CPU utilization, memory usage, network bandwidth usage, queue length of pending tasks, and estimated task completion time. The cloud judges the timeliness of each load status and removes load status data that exceeds a preset timeliness threshold, such as 15 seconds, to ensure that scheduling decisions are based on the latest node status.

[0025] The cloud assigns edge nodes to each sub-task data domain based on the real-time load status and physical deployment location of each edge node. The assignment follows the proximity deployment rule and the load adaptation rule. The proximity deployment rule prioritizes assigning the sub-task data domain to the edge node that is geographically closest to the power grid equipment in that data domain. The load adaptation rule, based on proximity deployment, assigns the sub-task data domain to the edge node with a lower current load and whose expected task completion time meets the maximum processing latency requirement. After the assignment is completed, the cloud generates an edge task orchestration scheme that includes the correspondence between each sub-task data domain and the assigned edge node identifier.

[0026] In the preferred embodiment, step S2 specifically includes the following sub-steps: S21. Edge nodes obtain the subtasks corresponding to themselves from the edge task orchestration scheme, extract the required measurement data and perform preprocessing, including data cleaning, format standardization and time scale alignment. S22. Edge nodes report the preprocessed measurement data to the cloud, but do not report the subtask processing results.

[0027] Specifically, each edge node extracts subtask information matching its own identifier from the edge task orchestration scheme to determine the range of measurement data that needs to be obtained from the local database or real-time acquisition interface. The measurement data includes power grid operation data such as voltage amplitude, phase angle, active power, reactive power, and current.

[0028] Edge nodes preprocess the acquired measurement data, including removing obvious outliers and filling in missing values; they also convert measurement data from different sources into a standard data format such as JSON, unifying the units and dimensions. After preprocessing, the edge nodes report the preprocessed measurement data to the cloud.

[0029] In the preferred embodiment, step S3 specifically includes the following sub-steps: S31. The cloud loads the power grid topology model from the local topology database. The cloud combines the power grid topology model to analyze the electrical connection relationships between the power grid equipment involved in each subtask. The electrical connection relationships include direct electrical connections and indirect electrical connections within a preset topology hop threshold range. Direct electrical connection refers to two devices being directly connected via the same busbar or branch, without any other devices in between. Indirect electrical connection refers to two devices being connected via a path consisting of multiple branches, where the number of hops in the path does not exceed a preset threshold, such as 3 hops. For example, if device A is connected to busbar B via branch 1, and busbar B is connected to device C via branch 2, then the number of hops between device A and device C is 2, which is an indirect electrical connection.

[0030] S32. When it is detected that subtasks belonging to different edge nodes are topologically related, and the electrical impedance of the related path is less than a preset impedance threshold, these subtasks are marked as a collaborative computing group; wherein, the preset impedance threshold is set according to the grid voltage level and the accuracy requirements of collaborative computing. For example, it can be set to 0.1 pu for a 220kV grid and 0.05 pu for a 500kV grid; For example, subtask 1 is assigned to edge node A, involving device M; subtask 2 is assigned to edge node B, involving device N. There is a direct electrical connection between device M and device N, and the electrical impedance of the associated path is 0.03 pu, which is less than the preset impedance threshold of 0.1 pu. Therefore, subtask 1 and subtask 2 are marked as the same collaborative computing group.

[0031] S33. Based on the task type of the data processing request, the cloud selects the corresponding collaborative algorithm from the preset collaborative algorithm library, generates collaborative computing instructions, and sends them to each edge node within the collaborative computing group. The collaborative computing instructions specify the data to be exchanged between nodes, the data exchange timing, and the collaborative algorithm parameters. The preset collaborative algorithm library includes a distributed weighted least squares collaborative algorithm for state estimation tasks, a distributed Newton-Raphson collaborative algorithm for power flow calculation tasks, a distributed graph neural network collaborative algorithm for fault location tasks, and a federated average collaborative algorithm for load forecasting tasks.

[0032] In the preferred embodiment, step S4 specifically includes the following sub-steps: S41. According to the data interaction rules between nodes in the collaborative computing instructions, the edge node establishes a secure communication channel with other nodes in the group, sends the measurement data that needs to be shared locally, and receives the measurement data shared by other nodes. S42. The edge node executes the distributed collaborative algorithm specified in the collaborative computing instruction based on the local measurement data and the received shared measurement data to obtain the distributed computing result. S43. Edge nodes report the distributed computing results to the cloud.

[0033] Specifically, each edge node parses collaborative computing instructions and establishes a secure communication channel with other nodes in the group according to the data interaction rules between nodes. The secure communication channel uses TLS / SSL encryption for transmission and authentication to ensure the legitimacy of the communicating parties. Edge nodes send locally shared measurement data and receive measurement data shared by other nodes.

[0034] Edge nodes execute the distributed cooperative algorithm specified in the cooperative computing instructions based on local measurement data and received shared measurement data. Taking distributed Newton-Raphson power flow calculation as an example, each edge node independently performs local power flow iterative calculations based on the local power grid sub-region model and exchanges information with boundary nodes, and achieves global convergence through the interaction of power mismatch quantities at boundary nodes. After the calculation is completed, the edge nodes report the distributed computing results to the cloud.

[0035] In a preferred embodiment, step S5 specifically includes the following sub-steps: S51: Cloud-based concatenation of distributed computing results belonging to the same collaborative computing group yields a global processing result; S52. The cloud performs multi-dimensional verification of the global processing results, including: data integrity verification, power grid topology constraint verification, and end-to-end processing latency verification. Data integrity verification: The cloud checks whether the global processing results include the calculation results of all devices to be processed, and whether there is any missing data or format errors. Specifically, this includes: verifying whether the device identifier list in the global processing results matches the list of power grid devices to be processed in the data processing request; and checking whether the calculation result fields of each device are complete and whether the values ​​are within the valid data range. The power grid topology constraint verification includes: Kirchhoff's current law constraint verification, Kirchhoff's voltage law constraint verification, node voltage amplitude upper and lower limit constraint verification, and branch power transmission limit constraint verification.

[0036] S53. If all multi-dimensional verifications pass, the global processing result will be output to the power grid dispatching system. S54. If the verification fails, output the reason for the failure, adjust the subtask division method or matching strategy, and return to step S1 to re-execute.

[0037] Specifically, the end-to-end processing latency check is as follows: calculate the actual processing latency from receiving the data processing request from the cloud to generating the global processing result, and determine whether the actual processing latency is less than the maximum allowed processing latency; if the actual processing latency is greater than or equal to the maximum processing latency, the latency check fails.

[0038] Specifically, when the cloud receives a data processing request from the power grid dispatch system, it records the request reception time T1; after generating the global processing result and passing the verification, it records the result generation time T2. The actual processing delay ΔT = T2 - T1.

[0039] The actual processing delay ΔT includes the following components: Task orchestration latency: The time interval from T1 to the completion of the edge task orchestration scheme generation, including the time required for request parsing, load status acquisition, subtask data domain decomposition, and edge node allocation; Data preprocessing latency: The maximum value of the time intervals from each edge node acquiring the subtask to completing the measurement data preprocessing and reporting to the cloud; Collaborative computing latency: The maximum value of the time intervals between sending collaborative computing instructions from the cloud to each edge node completing distributed collaborative computing and reporting the distributed computing results; Result merging and verification latency: The time interval from receiving all distributed computing results from the cloud to completing the global processing result merging and multi-dimensional verification.

[0040] The cloud compares the actual processing latency ΔT with the maximum allowed processing latency T_max. If ΔT < T_max, the latency check passes; if ΔT ≥ T_max, the latency check fails.

Claims

1. A highly efficient data processing method for cloud-edge collaborative interaction, characterized in that, Applied to systems including cloud and multiple edge nodes, the method includes the following steps: S1. The cloud receives data processing requests from the power grid dispatching system, parses the data type and response latency requirements in the data processing requests, and obtains the real-time load status periodically reported by each edge node; based on the data type, response latency requirements, and real-time load status, the data processing requests are decomposed into several sub-tasks, and each sub-task is matched with a corresponding edge node to generate an edge task orchestration scheme. S2. Each edge node obtains the corresponding sub-task according to the edge task orchestration scheme, determines the local measurement data range to be processed according to the sub-task, preprocesses the measurement data, generates preprocessed measurement data, and reports it to the cloud. S3. Based on the preprocessed measurement data reported by each edge node, the cloud identifies cross-node subtasks with grid topology associations in conjunction with the grid topology model, and issues collaborative computing instructions to the corresponding edge nodes. S4. Each edge node completes the interaction and collaborative calculation of cross-node power grid data according to the collaborative calculation instructions, generates distributed calculation results and reports them to the cloud; S5 merges the distributed computing results of each edge node in the cloud to obtain the complete power grid processing result and performs verification; if the verification passes, it is output to the power grid dispatch system; if the verification fails, the reason for the failure is output.

2. The method according to claim 1, characterized in that, Step S1 specifically includes the following sub-steps: S11. The cloud receives and parses the data processing request to obtain the task type and the maximum allowed processing latency. S12. The cloud obtains the real-time load status periodically reported by each edge node, judges the timeliness of each load status, and removes load status data that exceeds the preset timeliness threshold. S13. The cloud determines the range of power grid equipment to be processed based on the task type, and divides the range into multiple sub-task data domains in combination with the power grid topology; S14. Based on the maximum processing latency and the real-time load status of each edge node, the cloud allocates an edge node to each subtask data domain and generates an edge task orchestration scheme that includes the correspondence between subtasks and edge nodes.

3. The method according to claim 2, characterized in that, The real-time load status includes: The edge node's CPU utilization, memory usage, network bandwidth usage, queue length of pending tasks, and estimated task completion time.

4. The method according to claim 1, characterized in that, Step S2 specifically includes the following sub-steps: S21. Edge nodes obtain subtasks corresponding to themselves from the edge task orchestration scheme, extract the required measurement data and perform preprocessing, including data cleaning, format standardization and time scale alignment. S22. Edge nodes report the preprocessed measurement data to the cloud.

5. The method according to claim 1, characterized in that, Step S3 specifically includes the following sub-steps: S31. The cloud-based system combines a power grid topology model to analyze the electrical connection relationships between power grid equipment involved in each sub-task. The electrical connection relationships include direct electrical connections and indirect electrical connections within a preset topology hop count threshold range. S32. When it is detected that subtasks belonging to different edge nodes are related in the topology and the electrical impedance of the related path is less than a preset impedance threshold, these subtasks are marked as a collaborative computing group. S33. The cloud selects the corresponding collaborative algorithm from the preset collaborative algorithm library according to the task type of the data processing request, generates a collaborative computing instruction and sends it to each edge node in the collaborative computing group. The collaborative computing instruction specifies the data to be interacted between nodes, the data interaction sequence and the collaborative algorithm parameters.

6. The method according to claim 1, characterized in that, Step S4 specifically includes the following sub-steps: S41. According to the data interaction rules between nodes in the collaborative computing instructions, the edge node establishes a secure communication channel with other nodes in the group, sends the measurement data that needs to be shared locally, and receives the measurement data shared by other nodes. S42. The edge node executes the distributed collaborative algorithm specified in the collaborative computing instruction based on the local measurement data and the received shared measurement data to obtain the distributed computing result. S43. The edge node reports the distributed computing results to the cloud.

7. The method according to claim 1, characterized in that, Step S5 specifically includes the following sub-steps: S51: Cloud-based concatenation of distributed computing results belonging to the same collaborative computing group yields a global processing result; S52. The cloud performs multi-dimensional verification on the global processing result, including: data integrity verification, power grid topology constraint verification, and end-to-end processing latency verification. S53. If all the multi-dimensional verifications pass, the global processing result is output to the power grid dispatching system; S54. If the verification fails, output the reason for the failure, adjust the subtask division method or matching strategy, and return to step S1 to re-execute.

8. The method according to claim 7, characterized in that, The end-to-end processing delay check is as follows: Calculate the actual processing latency from receiving a data processing request from the cloud to generating a global processing result, and determine whether the actual processing latency is less than the maximum allowed processing latency; If the actual processing delay is greater than or equal to the maximum processing delay, the delay check fails.

Images