A Distributed Big Data Real-Time Streaming Processing Method and System

Abstract

The present invention provides a method and system for distributed big data real-time stream processing. By generating a topological sequence diagram based on a power network, a heat network, a natural gas network, and real-time operating state parameters, and combining the real-time operating state parameters, a set of dynamic sharding rules for a multi-energy flow coupling network is generated to calculate a dynamic threshold. When any load factor exceeds the dynamic threshold, a dynamic adjustment instruction for the sharding boundary is generated. Based on the dynamic adjustment instruction and multi-energy flow coupling data, a hierarchical collaborative optimization architecture is constructed to determine the target energy interaction constraints and target timing optimization results between shards. Combining the dynamic adjustment instruction, a distributed dynamic sharding configuration instruction for the multi-energy flow coupling network is generated to perform distributed big data real-time stream processing. The technical solution provided by the present invention realizes real-time data processing and resource collaborative optimization of the multi-energy flow coupling network, and provides an efficient solution for scenarios such as smart energy and industrial Internet of Things.

Description

Technical Field

[0001] The embodiments of the present invention relate to the technical field of data processing, and in particular, to a distributed big data real-time stream processing method and system. Background Art

[0002] In the integrated energy system in the core area of a megacity (such as the central business district), the power grid, heat network, and natural gas network are deeply coupled, including multiple devices such as gas turbines, combined heat and power units, distributed photovoltaics, and energy storage. It is necessary to achieve multi-energy flow collaborative optimization on a second-level time scale.

[0003] The core defects of the current technology are as follows: The existing solutions use historical topology data to divide fixed slices, resulting in a large error rate in slice optimization and being unable to meet the second-level response requirements. Summary of the Invention

[0004] The embodiments of the present invention provide a distributed big data real-time stream processing method and system to solve the core defects in the existing technology, that is, the large error rate in slice optimization and the inability to meet the second-level response requirements.

[0005] In a first aspect, the embodiments of the present invention provide a distributed big data real-time stream processing method, including:

[0006] Based on the power grid, heat network, natural gas network, and real-time operating state parameters, generate a topology sequence diagram, and based on the topology sequence diagram and real-time operating state parameters, generate a set of dynamic slicing rules for the multi-energy flow coupling network. The set of dynamic slicing rules includes the power grid topology structure, multi-energy flow coupling data, and dynamic slicing trigger conditions;

[0007] Based on the set of dynamic slicing rules, calculate a dynamic threshold. When any load factor exceeds the dynamic threshold, generate a dynamic adjustment instruction for the slice boundary. The load factor includes the node voltage deviation rate, node pressure fluctuation gradient, and return water temperature deviation rate;

[0008] Based on the dynamic adjustment instruction and the multi-energy flow coupling data, construct a hierarchical collaborative optimization architecture to determine the target energy interaction constraint and target timing optimization result between slices;

[0009] Based on the target energy interaction constraint, the target timing optimization result, and the dynamic adjustment instruction, generate a distributed dynamic slicing configuration instruction for the multi-energy flow coupling network, and use the distributed dynamic slicing configuration instruction to drive the data stream processing unit within the slice to perform distributed big data real-time stream processing.

[0010] Optionally, based on the dynamic adjustment instruction and the multi-energy flow coupling data, construct a hierarchical collaborative optimization architecture to determine the target energy interaction constraint and target timing optimization result between slices, including:

[0011] Based on the betweenness centrality of the core layer nodes in the power grid and the multi-energy flow coupling data, combined with the weights of the multi-energy flow coupling factors and the line capacity weights, a dynamic weight matrix is generated;

[0012] Using the distributed consensus optimization algorithm, the output priorities of the photovoltaic and energy storage devices within the shard are solved in parallel to generate the timing optimization instructions for the devices within the shard, and combined with the pre-regulation instructions of the combined heat and power generation, the verified shard boundary power demand constraints are generated;

[0013] Based on the verified shard boundary power demand constraints and the dynamic weight matrix, the cross-shard energy interaction rules are collaboratively optimized to generate the inter-shard energy interaction constraints and the cross-shard timing collaboration rules;

[0014] Based on the inter-shard energy interaction constraints and the cross-shard timing collaboration rules, when the difference between the betweenness centrality of the core layer nodes and the preset betweenness value exceeds the limit threshold, the topological structure of the shard is reconstructed, and combined with the dynamic weight matrix and the pre-regulation instructions, the post-reorganization inter-shard energy interaction constraints and the timing optimization rules are generated;

[0015] When an abnormality occurs in the node voltage deviation coefficient or the thermal return water temperature within the shard, the dynamic weight matrix is frozen using the post-reorganization inter-shard energy interaction constraints and the timing optimization rules, and the transmission margin of the adjacent shards is reallocated to generate the safe mode energy interaction constraints and the abnormal timing optimization results. After the abnormality is lifted, the dynamic weight matrix is optimized based on the safe mode energy interaction constraints to generate the target energy interaction constraints and the target timing optimization results.

[0016] Optionally, using the distributed consensus optimization algorithm, the output priorities of the photovoltaic and energy storage devices within the shard are solved in parallel to generate the timing optimization instructions for the devices within the shard, and combined with the pre-regulation instructions of the combined heat and power generation, the verified shard boundary power demand constraints are generated, including:

[0017] Based on the irradiance fluctuation coefficient of the photovoltaic device, the discrete gradient of the state of charge of the energy storage device, and the device topological connectivity, a dynamic priority weight matrix for the devices within the shard is constructed;

[0018] The distributed consensus optimization of the dynamic priority weight matrix is performed using the alternating direction method of multipliers to generate the timing optimization instructions;

[0019] Using the ratio of the range of the return water temperature within the sliding time window to the set threshold as the reference sensitivity, combined with the reference sensitivity and the historical time delay data, a time delay compensation coefficient is generated, and the reference sensitivity and the time delay compensation coefficient are convolved to generate the intensity parameter of the pre-regulation instruction;

[0020] Perform cross - time - step correlation analysis on the intensity parameters of the time - series optimization instruction and the pre - regulation instruction using a bidirectional long short - term memory network to obtain the correlation analysis result. Based on the load factor, the dynamic priority weight matrix, and the correlation analysis result, calculate the preliminary boundary power demand constraint;

[0021] Construct a constraint validity verification mechanism, define a feasibility index, and use the constraint validity verification mechanism to verify the feasibility index. When the feasibility index is lower than the preset threshold, reconstruct the dynamic priority weight matrix to generate the verified boundary power demand constraint.

[0022] Optionally, generate a topological sequence graph based on the power network, the thermal network, the natural gas network, and the real - time operating state parameters. Based on the topological sequence graph and the real - time operating state parameters, generate a set of dynamic slicing rules for the multi - energy flow coupling network. The set of dynamic slicing rules includes the power network topological structure, the multi - energy flow coupling data, and the dynamic slicing trigger conditions, including:

[0023] Obtain the topological connection relationship of the power network, the hydraulic balance parameters of the thermal network, and the pressure and flow dynamic relationship of the natural gas network to construct the physical connection relationship of the multi - energy flow coupling network. Combine the real - time operating state parameters and the physical connection relationship to generate a topological sequence graph;

[0024] Based on the real - time operating state parameters, perform time - series prediction on the node voltage deviation rate of the power network, the node return water temperature of the thermal network, and the node pressure fluctuation gradient of the natural gas network to obtain multiple node state prediction values;

[0025] Use elastic net regression to train the multiple node state prediction values to determine the multi - energy flow coupling factor weights. Based on the multi - energy flow coupling factor weights, linearly combine the multiple node state prediction values to calculate the multi - energy flow coupling intensity. Based on the topological sequence graph and the multi - energy flow coupling intensity, construct a spatial distribution matrix representing the multi - energy flow coupling relationship. Combine the multi - energy flow coupling intensity and the real - time state operating parameters to generate multi - energy flow coupling data;

[0026] Use the spectral clustering algorithm and cosine similarity metric to perform dynamic slicing processing on the spatial distribution matrix to generate an initial slicing set;

[0027] Use an extreme gradient boosting classifier trained based on historical load data to generate a set of dynamic slicing trigger conditions. Combine the multi - energy flow coupling data, the initial slicing set, and the set of dynamic slicing trigger conditions to generate a set of dynamic slicing rules for the multi - energy flow coupling network.

[0028] Optionally, obtain the topological connection relationship of the power network, the hydraulic balance parameters of the thermal network, and the dynamic relationship of pressure and flow in the natural gas network to construct the physical connection relationship of the multi-energy flow coupling network. Combine the real-time operating state parameters and the physical connection relationship to generate a topological sequence diagram, including:

[0029] Collect the connection relationship data of power network equipment, analyze the physical connection path from the distribution transformer to the user node, generate the adjacency matrix of the power network, and optimize the adjacency matrix of the power network by combining the breaker state data and the node voltage deviation rate to generate a dynamic topological feature matrix containing voltage deviation attributes;

[0030] According to the supply and return water temperature data and the flow time series data, predict the return water temperature deviation rate, generate a standardized temperature deviation rate sequence, calculate the temperature deviation rate difference between adjacent nodes, and based on the temperature deviation rate difference, mark the hydraulic balance state of the thermal branches in the thermal network to generate a thermal network balance coefficient matrix;

[0031] Based on the pressure monitoring data of the natural gas network, generate node pressure distribution data, and jointly predict the pipeline pressure attenuation rate and flow fluctuation gradient by combining the Weibull distribution model and the long short-term memory network to generate a natural gas network dynamic gradient matrix;

[0032] Based on the dynamic topological feature matrix, the thermal network balance coefficient matrix, and the natural gas network dynamic gradient matrix, generate a multi-energy flow physical connection relationship matrix, and dynamically optimize the coupling weight of the multi-energy flow physical connection relationship matrix based on the real-time load fluctuation rate to generate a topological sequence diagram.

[0033] Optionally, use the spectral clustering algorithm and cosine similarity metric to perform dynamic sharding processing on the spatial distribution matrix to generate an initial shard set, including:

[0034] Based on the spatial distribution matrix, real-time state operating parameters, and time decay factor, construct a node similarity matrix representing the coupling strength between nodes, and the time decay factor is dynamically adjusted according to the real-time load fluctuation rate;

[0035] Based on the node similarity matrix and the node degree distribution characteristics, generate a normalized coupling relationship matrix;

[0036] Use the manifold learning algorithm to perform dimensionality reduction processing on the normalized coupling relationship matrix to generate a dimensionality-reduced feature matrix;

[0037] Based on the dimensionality-reduced feature matrix, combine the improved clustering algorithm with integrated balance regularization terms and the alternating direction multiplier method to optimize the standard deviation of the multi-energy flow coupling strength within the shard and the cross-shard channel occupancy ratio constraint to generate an initial shard set.

[0038] Optionally, based on the set of dynamic sharding rules, a dynamic threshold is calculated. When any load factor exceeds the dynamic threshold, a dynamic adjustment instruction for the sharding boundary is generated. The load factors include the node voltage deviation rate, the node pressure fluctuation gradient, and the return water temperature deviation rate, and it includes:

[0039] Based on the set of dynamic sharding rules, combined with the dynamic mean, dynamic standard deviation, and elasticity coefficient of the load factors within a sliding time window, a dynamic threshold is calculated;

[0040] Based on the multi-energy flow coupling strength, the standard deviation of the multi-energy flow coupling strength corresponding to each shard is calculated, and the target shards with the multi-energy flow coupling strength standard deviation higher than the preset standard value are selected;

[0041] According to the dynamic threshold and the elasticity coefficient, the load overflow ratio is calculated to generate a shard contraction coefficient or expansion coefficient, and combined with a graph neural network to update the energy interaction weight matrix of adjacent shards, obtaining an updated energy interaction weight matrix;

[0042] According to the updated energy interaction weight matrix, the transmission power gradient and the lowest occupancy ratio threshold of the cross-shard energy interaction channel during the sharding boundary adjustment process are set;

[0043] Based on the preset conflict arbitration rule, when it is detected that any load factor exceeds the dynamic threshold, combined with the shard contraction coefficient or expansion coefficient, the updated energy interaction weight matrix, the transmission power gradient, and the lowest occupancy ratio threshold, a dynamic adjustment instruction for the sharding boundary is generated.

[0044] In a second aspect, an embodiment of the present invention provides a distributed big data real-time stream processing system, including:

[0045] A generation module, configured to generate a topological sequence diagram based on a power network, a heat network, a natural gas network, and real-time operation state parameters, and generate a set of dynamic sharding rules for the multi-energy flow coupling network based on the topological sequence diagram and real-time operation state parameters. The set of dynamic sharding rules includes a power network topological structure, multi-energy flow coupling data, and dynamic sharding trigger conditions;

[0046] A calculation module, configured to calculate a dynamic threshold based on the set of dynamic sharding rules. When any load factor exceeds the dynamic threshold, a dynamic adjustment instruction for the sharding boundary is generated. The load factors include the node voltage deviation rate, the node pressure fluctuation gradient, and the return water temperature deviation rate;

[0047] A determination module, configured to construct a hierarchical collaborative optimization architecture based on the dynamic adjustment instruction and the multi-energy flow coupling data to determine the target energy interaction constraints and target timing optimization results between shards;

[0048] A processing module, configured to generate a distributed dynamic sharding configuration instruction for a multi-energy flow coupling network based on the target energy interaction constraint, the target timing optimization result, and the dynamic adjustment instruction, and drive a data flow processing unit within a shard to perform distributed big data real-time stream processing by using the distributed dynamic sharding configuration instruction.

[0049] In a third aspect, an embodiment of the present invention provides a computing device, including a processor and a memory. A computer program is stored in the memory, and the processor is configured to run the computer program to execute the distributed big data real-time stream processing method according to any one of the first aspect.

[0050] In a fourth aspect, an embodiment of the present invention provides a computer storage medium, on which computer program instructions are stored. When the computer program instructions are executed by a processor, the distributed big data real-time stream processing method according to any one of the first aspect is implemented.

[0051] In the embodiments of the present invention, based on the power network, the thermal network, the natural gas network, and the real-time operating state parameters, a topological sequence diagram is generated. Based on the topological sequence diagram and the real-time operating state parameters, a set of dynamic fragmentation rules for the multi-energy flow coupling network is generated. The set of dynamic fragmentation rules includes the power network topological structure, the multi-energy flow coupling data, and the dynamic fragmentation trigger conditions. Based on the set of dynamic fragmentation rules, a dynamic threshold is calculated. When any load factor exceeds the dynamic threshold, a dynamic adjustment instruction for the fragmentation boundary is generated. The load factors include the node voltage deviation rate, the node pressure fluctuation gradient, and the return water temperature deviation rate. Based on the dynamic adjustment instruction and the multi-energy flow coupling data, a hierarchical collaborative optimization architecture is constructed to determine the target energy interaction constraints and the target timing optimization results between the fragments. Based on the target energy interaction constraints, the target timing optimization results, and the dynamic adjustment instruction, a distributed dynamic fragmentation configuration instruction for the multi-energy flow coupling network is generated, and the distributed big data real-time stream processing of the data flow processing units within the fragments is driven by using the distributed dynamic fragmentation configuration instruction. The technical solution provided by the present invention can monitor key indicators such as the node voltage deviation rate and the pressure fluctuation gradient in real time through the calculation of the set of dynamic fragmentation rules and the dynamic threshold. When the load factor exceeds the threshold, the system automatically triggers the fragmentation boundary adjustment instruction to avoid the problem of uneven load caused by the traditional static fragmentation strategy. The hierarchical collaborative optimization architecture combines the target energy interaction constraints and the timing optimization results to optimize the data flow interaction efficiency between the fragments. The set of dynamic fragmentation rules is generated based on the topological sequence diagram and the real-time operating state parameters to ensure that the data flow processing units can respond immediately to the network state changes. The priority and order of data processing are controlled by the target timing optimization results to avoid the delay problem of the traditional batch processing system. The integration and dynamic fragmentation of the coupling data of the power, thermal, and natural gas networks support the joint scheduling of multi-energy systems. The adjustment instruction after the dynamic fragmentation rules are triggered can quickly reconstruct the fragmentation boundary in case of local node failures to ensure the overall stability of the system. The set of fragmentation rules can be dynamically expanded according to the network topology changes and real-time data to adapt to complex scenarios such as power network expansion and new energy access. By means of the hierarchical collaborative optimization architecture (such as separating the in-fragment processing from the inter-fragment coordination), the system complexity is reduced, which is applicable to large-scale distributed environments. Generally speaking, the present invention realizes the real-time data processing and resource collaborative optimization of the multi-energy flow coupling network, which is significantly superior to the traditional static fragmentation or single-energy network processing methods in terms of real-time performance, load balancing, system stability, and scalability, and provides an efficient solution for scenarios such as smart energy and industrial Internet of Things.Furthermore, the present invention realizes the efficient coordination and dynamic security control of the multi-energy flow system through a hierarchical collaborative optimization architecture. Combining the dynamic weight matrix and the distributed consensus optimization algorithm, based on the output priority of photovoltaic and energy storage devices within a slice and the pre-regulation command of combined heat and power, the power constraints at the slice boundary and the cross-slice interaction rules are generated, realizing the dynamic coordination and topological reconstruction of energy interaction between slices in the multi-energy flow coupling scenario. At the same time, the safety mode is triggered by anomaly detection (freezing the weight matrix and adjusting the transmission margin). Combining the reorganized slice constraints and optimization rules, it can quickly switch to the safe operation state when voltage or thermal parameters are abnormal, and adaptively optimize the weight matrix after the anomaly is lifted, ultimately ensuring the stability and convergence of the system, effectively improving the response speed and robustness of multi-energy flow calculation, and solving the problems of dynamic optimization and safety boundary control in complex coupling scenarios.

[0052] These aspects or other aspects of the present invention will be more clearly understood in the following description of the embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the following drawings are some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0054] Figure 1 It is a flowchart of a distributed big data real-time stream processing method provided by an embodiment of the present invention;

[0055] Figure 2 It is a schematic structural diagram of a distributed big data real-time stream processing system provided by an embodiment of the present invention;

[0056] Figure 3 It is a schematic structural diagram of a computing device provided by an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0057] In order to enable those skilled in the art to better understand the solution of the present invention, the following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the drawings in the embodiments of the present invention.

[0058] In some of the processes described in the specification, claims, and above-mentioned drawings of the present invention, there are multiple operations that appear in a specific order. However, it should be clearly understood that these operations may not be executed in the order in which they appear herein or may be executed in parallel. The serial numbers of the operations, such as 101, 102, etc., are only used to distinguish different operations, and the serial numbers themselves do not represent any execution order. Additionally, these processes may include more or fewer operations, and these operations may be executed in sequence or in parallel. It should be noted that the descriptions such as "first" and "second" in this article are used to distinguish different messages, devices, modules, etc., do not represent a sequence, and do not limit that "first" and "second" are of different types.

[0059] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without making creative efforts belong to the scope of protection of the present invention.

[0060] Figure 1 The flowchart of a distributed big data real-time stream processing method provided for an embodiment of the present invention is as Figure 1 shown, and the method includes:

[0061] The present invention focuses on solving the problems of the static nature of traditional sharding strategies in multi-energy flow coupling networks (electricity, heat, natural gas), insufficient adaptability to load fluctuations, and inability to meet the second-level response requirements. Existing technologies usually adopt fixed sharding rules, which are difficult to cope with the dynamic changes of the multi-energy network topology and real-time load differences (such as node voltage deviation, pressure fluctuation gradient, etc.), resulting in low energy interaction efficiency between shards and unbalanced resource allocation. This solution constructs a set of dynamic sharding rules based on the topological sequence diagram of the multi-energy flow coupling network and real-time operating parameters (such as voltage, pressure, temperature), associates the power network topological structure with multi-energy flow data, and realizes the strong coupling of the sharding logic and the physical network; secondly, introduces a dynamic threshold calculation mechanism, combines the mean, standard deviation, and elasticity coefficient of the load factor within a sliding time window, and triggers real-time adjustment of the sharding boundary to solve the problem of response lag caused by fixed thresholds in traditional methods; finally, through a hierarchical collaborative optimization architecture, integrates the target energy interaction constraints and the timing optimization results, and drives the processing units within the shards to execute according to the dynamic configuration instructions, overcoming the defect of insufficient collaborative optimization ability between shards in existing technologies. This solution deeply integrates the multi-energy flow coupling characteristics and dynamic sharding technology, significantly improving the resource utilization rate and stability of real-time stream processing in complex energy scenarios. Based on this, the present invention provides a distributed big data real-time stream processing method, as Figure 1 , including:

[0062] Step 101: Generate a topological sequence diagram based on the power network, the heat network, the natural gas network, and the real-time operating state parameters. Based on the topological sequence diagram and the real-time operating state parameters, generate a set of dynamic partitioning rules for the multi-energy flow coupling network. The set of dynamic partitioning rules includes the power network topological structure, the multi-energy flow coupling data, and the dynamic partitioning trigger conditions.

[0063] In this step, the topological sequence diagram refers to a dynamic visualization model that integrates the physical connection relationships and real-time operating parameters of the power, heat, and natural gas networks. By abstracting the multi-dimensional correlation relationships of nodes and edges, it characterizes the spatio-temporal distribution characteristics of the multi-energy flow coupling network. The set of dynamic partitioning rules refers to a set of logical rules for dynamically partitioning the multi-energy flow coupling network based on the topological structure and real-time parameters in the topological sequence diagram, aiming to optimize resource allocation and fault isolation capabilities. The multi-energy flow coupling data refers to a set of parameters that describe the dynamic interaction relationships and coordinated operating states between energy networks such as power, heat, and natural gas, and is used to quantify the energy conversion efficiency, transmission constraints, and economic correlations across energy networks. It is the core input of the set of dynamic partitioning rules for the multi-energy flow coupling network.

[0064] In the embodiment of the present invention, first, extract the bus-branch relationship of the power network, the pipeline-heat exchange station connection of the heat network, and the gas source-compressor topology of the natural gas network through a graph database, and combine the parameters such as node voltage, pipeline pressure, and return water temperature collected in real time by the data acquisition and monitoring system to construct a topological sequence diagram. Subsequently, use the community discovery algorithm to identify the highly coupled subnets in the network, and combine the equipment operation constraints (such as the output upper limit of the combined heat and power unit) to generate a set of dynamic partitioning rules, which includes: the power network topological structure, that is, divide the key hubs based on the node betweenness centrality. For example, set the substations with a voltage level ≥ 220 kV as the core of the subnet; the multi-energy flow coupling data, that is, calculate the cross-energy interaction weights through elastic net regression. For example, the gas-electric conversion efficiency weight of the power-to-gas device is 0.92; the dynamic partitioning trigger conditions, for example, set trigger parameters such as the node voltage deviation rate threshold (±5%) and the pressure fluctuation gradient (3 kPa / s).

[0065] Step 102: Calculate a dynamic threshold based on the set of dynamic partitioning rules. When any load factor exceeds the dynamic threshold, generate a dynamic adjustment instruction for the partitioning boundary. The load factors include the node voltage deviation rate, the node pressure fluctuation gradient, and the return water temperature deviation rate.

[0066] In this step, the dynamic threshold refers to a critical value adaptively calculated based on the real-time operating state parameters and algorithms, and is used to determine whether the system is in an abnormal state or needs to trigger an adjustment action. The dynamic adjustment instruction refers to a control command automatically generated by the system when the load factor exceeds the dynamic threshold, and is used to re-divide the partitioning boundary of the multi-energy flow coupling network, optimize resource allocation, or isolate faults.

[0067] In the embodiment of the present invention, the sliding window statistical method is adopted to calculate the dynamic threshold. For example, for load factors such as the node voltage deviation rate and the pressure fluctuation gradient, the mean value and the standard deviation are calculated with a 30-second window, and the dynamic threshold is set to the mean value ± 2 times the standard deviation. When the return water temperature deviation rate exceeds the threshold, a dynamic adjustment instruction for generating the shard boundary is triggered. The generation of this instruction is implemented through a rule engine. For example, for the adjustment of the shard boundary, if the voltage of a certain power subnetwork exceeds the limit, the breadth-first search method is used to locate the adjacent load points that can be accessed, and a subnet splitting instruction is generated. For resource reallocation, according to the real-time load rate, the energy interaction capacity of each shard is allocated through a linear programming model.

[0068] Step 103: Based on the dynamic adjustment instruction and the multi-energy flow coupling data, construct a hierarchical collaborative optimization architecture to determine the target energy interaction constraint and the target timing optimization result between shards;

[0069] In this step, the target energy interaction constraint refers to the boundary conditions for energy transmission between shards in the multi-energy flow coupling network, which is used to limit the interaction capacity, transmission efficiency, and safety threshold of different energy subnets (such as power, heat, and natural gas shards), ensuring the feasibility and stability of cross-energy coordination. The target timing optimization result refers to the optimal solution set after globally coordinating variables such as energy scheduling and device output of the shards of the multi-energy flow coupling network based on the time dimension, aiming to maximize energy efficiency and minimize operating costs across different time scales.

[0070] In the embodiment of the present invention, a two-layer optimization model is adopted. Its upper-layer architecture aims at the optimal economy of the whole network, and sets the target energy interaction constraint between shards through mixed integer programming. For example, the power transmitted from the power shard to the heat shard ≤ 50MW. Its lower-layer architecture is based on model predictive control to roll and optimize the internal scheduling plan of each shard to determine the target timing optimization result. For example, start the power-to-gas equipment for energy storage during the low electricity consumption period and limit the cross-shard transmission during the peak period. The sources of its model parameters are multi-energy flow coupling data (such as the device efficiency matrix) and dynamic adjustment instructions (such as the topological relationship after shard reorganization).

[0071] Step 104: Based on the target energy interaction constraint, the target timing optimization result, and the dynamic adjustment instruction, generate a distributed dynamic shard configuration instruction for the multi-energy flow coupling network, and use the distributed dynamic shard configuration instruction to drive the data flow processing unit within the shard to perform distributed big data real-time stream processing;

[0072] In this step, the distributed dynamic sharding configuration instruction is a set of executable control commands generated based on the real-time state, target constraints, and optimization results of the multi-energy flow coupling network. It is used to dynamically divide the network sharding boundary and drive the data flow processing units within the shard to perform parallel and decentralized real-time computing and resource scheduling. Its essence is to coordinate the dynamic adaptation rules of the physical operation and data processing logic of the multi-energy flow coupling network to ensure that the system achieves efficient and stable distributed collaborative operation under complex working conditions.

[0073] In the embodiment of the present invention, instructions are generated through a distributed task scheduling framework (such as Apache Flink). For example, based on the target energy interaction constraint and the timing optimization result, a reinforcement learning model is used to solve the optimal sharding configuration parameters. The instructions are pushed to the edge computing nodes of each shard through a message queue. The data flow processing unit adjusts parameters such as the parallelism and window calculation period according to the distributed dynamic sharding configuration instruction to process the sensor data flow within the shard in real time.

[0074] The following is a specific example. In an industrial park integrating power, heat, and natural gas networks, a sudden increase in power load causes the voltage deviation rate to exceed the limit. Through the topological sequence diagram, it is shown that the voltage deviation rate of the power sub-network A reaches 6%, triggering the dynamic sharding rule to split the sub-network A into two sub-networks A1 and A2. According to the calculated dynamic threshold, it is found that the load rate of the sub-network A1 exceeds the limit, and a dynamic adjustment instruction is generated, that is, part of the load of A1 is migrated to the adjacent sub-network B. Through the hierarchical collaborative optimization architecture, the energy interaction constraint between the sub-network A1 and the adjacent sub-network B is set to 30 MW, and it is planned to recharge energy through the power-to-gas device during the low-load period. The distributed dynamic sharding configuration instruction drives the Flink node of the sub-network A1 to increase the stream processing parallelism from 4 to 8 and monitor the load fluctuation after migration in real time.

[0075] In the embodiment of the present invention, through dynamic sharding and collaborative optimization, efficient resource allocation and real-time fault isolation of the multi-energy flow coupling network are realized; the sharding boundary and threshold are automatically adjusted based on real-time data, reducing manual intervention; the global economy and local stability are balanced through a hierarchical model, reducing the operating cost and realizing resource optimization; the distributed stream processing architecture supports millisecond-level response, and the fault recovery time can be effectively shortened.

[0076] The present invention provides a specific embodiment, step 103, based on the dynamic adjustment instruction and the multi-energy flow coupling data, construct a hierarchical collaborative optimization architecture to determine the target energy interaction constraint and the target timing optimization result between shards, specifically including the following steps:

[0077] Step 301: Based on the betweenness centrality of the core layer nodes in the power network and the multi-energy flow coupling data, combined with the multi-energy flow coupling factor weight and the line capacity weight, generate a dynamic weight matrix;

[0078] In this step, the multi-energy flow coupling factor weight is a dynamic parameter that quantifies the degree of mutual influence among energy networks such as electricity, heat, and natural gas, and characterizes the energy conversion efficiency, stability contribution, and economic correlation of different energy types during the interaction process. The line capacity weight is a parameter that reflects the transmission capacity limitations of power lines, heat pipelines, or gas pipelines, and is used to constrain the maximum allowable value of energy interaction to prevent equipment overload or network congestion. The dynamic weight matrix is a multi-dimensional matrix that integrates the multi-energy flow coupling factor weight and the line capacity weight, and is used to describe the priority relationship between nodes and paths in the multi-energy flow network, supporting the iterative calculation of distributed optimization algorithms.

[0079] In the embodiment of the present invention, the coupling factors (such as the efficiency weight of the power-to-gas device is 0.85) and line capacities (such as the current-carrying capacity of the power line is 500 A) of the electricity, heat, and natural gas networks are normalized through the analytic hierarchy process to construct a dynamic weight matrix. Among them, the coupling factor weight is calculated based on the energy conversion efficiency and economic parameters. For example, the electricity-heat coupling weight is 0.7; the line capacity weight is generated in combination with the real-time derating factor (such as the cable capacity drops to 80% of the nominal value under high temperature); the dynamic update mechanism, for example, adjusts the weight every 5 minutes through the sliding window statistical method to ensure matching with the real-time load.

[0080] Step 302: Use the distributed consensus optimization algorithm to parallelly solve the output priorities of the photovoltaic and energy storage devices within the shard, generate the timing optimization instructions for the devices within the shard, and combine with the pre-regulation instructions of the combined heat and power generation to generate the verified power demand constraints at the shard boundary;

[0081] In this step, the timing optimization instructions for the devices within the shard are the device output scheduling commands generated based on the distributed consensus algorithm, which are used to coordinate the operation timing of the photovoltaic, energy storage and other devices within the shard to achieve the balance of local economy and stability. The pre-regulation instructions of the combined heat and power generation are the control commands sent in advance to the combined heat and power generation unit, which are used to adjust its output curve to match the predicted heat and power demand and reduce the real-time scheduling pressure. The verified power demand constraints at the shard boundary are the power interaction limits between shards verified by the distributed consensus algorithm to ensure that the cross-shard energy transmission meets the safety and economic requirements.

[0082] In the embodiment of the present invention, the distributed system consensus algorithm is used to achieve the collaborative scheduling of the devices within the shard. For example, the proposal is initiated by the Leader node (such as the photovoltaic shard controller), and the charge and discharge priorities of the energy storage devices are determined by the majority of Follower nodes voting; combined with the pre-regulation instructions of the combined heat and power generation unit (such as the minimum heat-electricity ratio is 0.6), the output plan of the devices within the shard is generated; verified through the distributed system consensus algorithm to ensure that the verified power demand constraints at the shard boundary (such as the maximum transmission power is 50 MW) are accepted by the majority of nodes to avoid the risk of overload.

[0083] Step 303: Based on the verified shard boundary power demand constraints and the dynamic weight matrix, co-optimize the cross-shard energy interaction rules to generate inter-shard energy interaction constraints and cross-shard timing coordination rules;

[0084] In this step, the cross-shard timing coordination rule is a global strategy for coordinating the scheduling timings of different shards, ensuring seamless connection of cross-shard energy interaction in the time dimension and avoiding timing conflicts or resource competition.

[0085] In the embodiment of the present invention, based on the dynamic weight matrix and the verified shard boundary power demand constraints, a mixed-integer linear programming model is used to optimize cross-shard interaction, generating inter-shard energy interaction constraints and cross-shard timing coordination rules. For example, the power transmitted from the power shard to the heat shard ≤ 30MW, and the state replication mechanism of the Zookeeper atomic broadcast algorithm is used to ensure that the constraint parameters are synchronized among the leader nodes of each shard; it is defined that the priority of the power shard during peak hours is higher than that of the heat shard, and the heartbeat synchronization mechanism is used to coordinate the time window.

[0086] Step 304: Based on the inter-shard energy interaction constraints and the cross-shard timing coordination rules, when the difference between the betweenness centrality of the core layer nodes and the preset betweenness value exceeds the limit threshold, reconstruct the topological structure of the shards, and combine the dynamic weight matrix and the pre-regulation instruction to generate the energy interaction constraints and timing optimization rules of the reorganized shards;

[0087] In the embodiment of the present invention, based on the inter-shard energy interaction constraints and the cross-shard timing coordination rules, the betweenness centrality of the core layer nodes is calculated using graph theory algorithms. If the difference between the betweenness centrality of a certain core layer node and the preset betweenness value exceeds the limit threshold (such as 0.15), it is determined as a key hub; the shard splitting instruction (such as splitting the original shard A into A1 and A2) is broadcast using the Gossip protocol, and the new shard boundaries are confirmed by a majority of nodes; combined with the dynamic weight matrix and the pre-regulation instruction, the energy interaction constraints and timing optimization rules of the reorganized shards are optimized through a reinforcement learning model.

[0088] Step 305: When abnormal conditions occur in the node voltage deviation coefficient or the return water temperature of the heat in the shard, freeze the dynamic weight matrix using the energy interaction constraints of the reorganized shards and the timing optimization rules, and reallocate the transmission margin of adjacent shards to generate the energy interaction constraints in the safe mode and the abnormal timing optimization results. After the abnormal conditions are lifted, optimize the dynamic weight matrix based on the energy interaction constraints in the safe mode to generate the target energy interaction constraints and the target timing optimization results;

[0089] In this step, the safe-mode energy interaction constraint is a temporary interaction rule triggered when an anomaly is detected, which restricts high-risk transmission paths; the anomaly timing optimization result is a revised scheduling plan for the device output during the anomaly period.

[0090] In the embodiment of the present invention, when the node voltage deviation rate > 5% or the return water temperature deviation rate > 3%, dynamic weight freezing is triggered, and matrix update is suspended; the adjacent shard Leader node is selected through the Raft election algorithm to take over the abnormal load (for example, shard B takes over 50% of the transmission tasks of shard A); after the anomaly is lifted, based on the safe-mode operation data, the dynamic weight matrix is recalculated through the gradient descent algorithm, and gradually transitions to the normal mode to generate the target energy interaction constraint and the target timing optimization result.

[0091] The following is a specific example. The betweenness of the core node of the power shard in an industrial park exceeds the limit, and at the same time, the return water temperature of the heat is abnormal. The power-heat coupling weight is 0.7 and the line capacity weight is 0.8 as shown by the dynamic weight matrix; the Raft election algorithm is used to coordinate the priority power supply of the photovoltaic, and the power constraint at the shard boundary is generated (transmission ≤ 40MW); the Zookeeper atomic broadcast algorithm is used to synchronize the cross-shard rules, and the power output is restricted during peak hours; when the betweenness of the core node reaches 0.18, the Gossip protocol splits the shard and generates new constraints (transmission ≤ 25MW); when the return water temperature anomaly triggers the safe mode, the Raft election algorithm selects shard C to take over the load, and the weight is optimized by gradient descent after the anomaly is lifted.

[0092] In the embodiment of the present invention, through the dynamic weight matrix, cross-energy dynamic priority matching of the power, heat, and natural gas networks is realized; the distributed consistency optimization algorithm is used to parallelly solve the priority of the device output within the shard, reducing the impact of the photovoltaic output fluctuation on the power grid; the cross-shard timing coordination rule realizes the balance between global economy and local stability; the topological structure of the shard is reconstructed, improving the fault tolerance of the system to key node failures; when voltage or temperature anomalies are detected, the dynamic weight is frozen and the transmission margin is reallocated, optimizing the safe-mode constraint, realizing millisecond-level fault isolation and recovery; through the cross-shard timing coordination rule and the reorganized constraints, the cross-shard transmission cost is reduced, and at the same time, the risk of equipment overload is avoided.

[0093] The present invention provides a specific embodiment, step 302, using the distributed consistency optimization algorithm to parallelly solve the priority of the output of the photovoltaic and energy storage devices within the shard, generating the timing optimization instruction of the devices within the shard, and combining the pre-regulation instruction of the combined heat and power generation to generate the verified power demand constraint at the shard boundary, specifically including the following steps:

[0094] Step 311: Based on the irradiance fluctuation coefficient of the photovoltaic device, the discrete gradient of the state of charge of the energy storage device, and the device topological connectivity, construct the dynamic priority weight matrix of the devices within the shard;

[0095] In this step, the irradiance fluctuation coefficient is a parameter that quantifies the impact of random changes in light intensity on the output power of a photovoltaic device, reflecting the degree to which the irradiance deviates from the average value per unit time. It is usually the ratio of the standard deviation to the mean of the actual irradiance. The dynamic parameter that describes the differences in the state of charge (SOC) of multiple energy storage devices within a slice, reflecting the balance of the energy storage cluster, is usually the ratio of the range (the maximum SOC and the minimum SOC) of the SOC values of each energy storage device to the average SOC. The device topology connectivity is a parameter that characterizes the physical or logical connection relationship between devices within a slice, including the number of power line connections, communication link strength, etc.

[0096] In the embodiment of the present invention, the irradiance fluctuation coefficient is taken as the ratio of the standard deviation to the mean of the irradiance within a 15-minute sliding window of the photovoltaic device (for example, if the standard deviation is 50 W / m² and the mean is 800 W / m², then the coefficient is 0.0625); the discrete gradient of the state of charge is taken as the ratio of the range of the SOC of the energy storage within the slice to the mean (for example, if the range is 0.3 and the mean is 0.7, then the gradient is 0.428); for the device topology connectivity, the node betweenness centrality is used (for example, a certain power node is connected to 5 lines and the betweenness is 0.12); weights are assigned through the analytic hierarchy process (for example, the irradiance fluctuation coefficient is 0.4, the discrete gradient of the state of charge is 0.3, and the device topology connectivity is 0.3), generating a dynamic priority weight matrix, which is updated every 5 minutes.

[0097] Step 312: Use the alternating direction multiplier method to perform distributed consensus optimization on the dynamic priority weight matrix to generate a time-series optimization instruction;

[0098] In the embodiment of the present invention, the optimization of the photovoltaic and energy storage output is decomposed into multiple sub-problems, corresponding to different devices respectively. The alternating optimization process includes the update of the original variables and the update of the multipliers; for the update of the original variables, fixed multipliers are used to solve the local optimal output of each device (such as 80% of the upper limit of the photovoltaic output); for the update of the multipliers, the multiplier values are adjusted according to the constraint deviation (such as the power balance error), and after iteration to convergence, a time-series optimization instruction (such as the energy storage charge and discharge plan) is output.

[0099] Step 313: Use the ratio of the range of the return water temperature within a sliding time window to the set threshold as the reference sensitivity, combine the reference sensitivity and the historical time-delay data to generate a time-delay compensation coefficient, and perform a convolution operation on the reference sensitivity and the time-delay compensation coefficient to generate the intensity parameter of the pre-regulation instruction;

[0100] In this step, the reference sensitivity refers to the ratio of the range of the return water temperature within a sliding time window to a set threshold, which is used to reflect the sensitivity of the temperature fluctuation of the thermal network. Historical time-delay data refers to the time-series data of the delay between the recorded system input signal and the response output, which is used to model the time-delay effect. The time-delay compensation coefficient refers to the correction factor generated based on the historical time-delay data and the reference sensitivity, which is used to offset the delay effect of the control instruction. The intensity parameter of the pre-regulation instruction refers to the dynamic parameter that quantifies the intensity of the pre-regulation instruction.

[0101] In the embodiment of the present invention, the reference sensitivity takes the ratio of the range of the return water temperature within a 30-minute sliding window to the threshold (such as 5°C) (for example, the range is 8°C and the sensitivity is 1.6); convolving the historical time-delay data (such as the average delay of 0.8 seconds) with the sensitivity to generate a compensation coefficient (1.6×0.8 = 1.28); mapping the convolution result to the range of 0 to 1 (such as mapping 1.28 to the intensity parameter of 0.85), and normalizing the intensity parameter to be used for adjusting the output of the combined heat and power unit.

[0102] Step 314: Use a bidirectional long short-term memory network to perform cross-time-step correlation analysis on the timing optimization instruction and the intensity parameter of the pre-regulation instruction to obtain the correlation analysis result. Based on the load factor, the dynamic priority weight matrix, and the correlation analysis result, calculate the preliminary boundary power demand constraint;

[0103] In this step, the preliminary boundary power demand constraint refers to the initial power transmission limit generated through correlation analysis, which is used to guide the energy interaction between slices.

[0104] In the embodiment of the present invention, input the timing instruction (such as the charge and discharge curve of the energy storage) and the intensity parameter (such as 0.85) into the bidirectional long short-term memory network, capture the timing dependence through the forward and backward LSTM, and output the correlation coefficient (such as 0.72); combine the load factor (such as an 80% load rate), the dynamic weight matrix (priority 0.7), and the correlation coefficient to calculate the preliminary boundary constraint (such as the maximum transmission power of 50 MW).

[0105] Step 315: Construct a constraint validity verification mechanism, define a feasibility index, and use the constraint validity verification mechanism to verify the feasibility index. When the feasibility index is lower than the preset threshold, reconstruct the dynamic priority weight matrix to generate the verified boundary power demand constraint;

[0106] In this step, the feasibility index refers to the quantitative parameter for evaluating the effectiveness of the preliminary boundary constraint, including dimensions such as economy (such as net present value), technology (such as sliding correlation coefficient, node voltage deviation rate, and return water temperature deviation rate), etc.

[0107] In the embodiments of the present invention, a feasibility index is defined, that is, thresholds such as net present value ≥ 0 and voltage deviation rate ≤ 3% are set, and a constraint effectiveness verification mechanism is used to verify the feasibility index. If the feasibility index does not meet the standard (such as the net present value is -5%), a dynamic weight matrix reconstruction is triggered, the device priorities are reallocated (such as reducing the weight of high-fluctuation photovoltaic), and the verified boundary power demand constraint is generated (such as transmission power ≤ 40 MW).

[0108] Taking a specific example, in a certain industrial park, the power of the distributed photovoltaic is fluctuating greatly, and the temperature of the heat return water is abnormal. By calculating that the irradiance fluctuation coefficient is 0.08 and the discrete gradient of the state of charge is 0.5, a dynamic weight matrix is constructed. By using the alternating direction multiplier method to coordinate the photovoltaic output to drop to 70%, the energy storage is preferentially charged; combined with the range of the return water temperature of 10 °C, an intensity parameter of 0.9 is generated to adjust the combined heat and power output to 90%; the analysis of the bidirectional long short-term memory network shows that the midday load is strongly correlated with the temperature, and the verified boundary power demand constraint (transmission ≤ 45 MW) is generated; after verification by the constraint effectiveness verification mechanism, it is found that the voltage deviation rate is 4%, and after reconstructing the dynamic weight matrix, the verified boundary power demand constraint is tightened to 35 MW.

[0109] In the embodiments of the present invention, by constructing a dynamic priority weight matrix, the dynamic priority matching of devices within a slice can be realized according to specific scenarios; the alternating direction multiplier method realizes distributed consistency optimization and effective cooperative scheduling; the bidirectional long short-term memory network realizes cross-period correlation analysis and boundary prediction, reduces the analysis error, and improves the accuracy of boundary constraints; through constraint effectiveness verification, the success rate of fault recovery is improved.

[0110] The present invention provides a specific embodiment. Step 101: Based on the power network, the heat network, the natural gas network, and the real-time operation state parameters, a topological sequence diagram is generated. Based on the topological sequence diagram and the real-time operation state parameters, a set of dynamic slicing rules for the multi-energy flow coupling network is generated. The set of dynamic slicing rules includes the topological structure of the power network, the multi-energy flow coupling data, and the dynamic slicing trigger conditions, and specifically includes the following steps:

[0111] Step 111: Obtain the topological connection relationship of the power network, the hydraulic balance parameters of the heat network, and the pressure and flow dynamic relationship of the natural gas network to construct the physical connection relationship of the multi-energy flow coupling network. Combining the real-time operation state parameters and the physical connection relationship, a topological sequence diagram is generated;

[0112] In this step, the physical connection relationship of the multi-energy flow coupling network refers to the integrated representation of the topological connection and dynamic interaction relationship among the power, heat, and natural gas networks. The real-time operation state parameters include the voltage deviation rate of the power network, the temperature gradient of the heat network, and the pressure fluctuation amount of the natural gas network.

[0113] In the embodiments of the present invention, for the extraction of topological relationships, for the power grid: an adjacency matrix is constructed based on node betweenness centrality (such as substations, load centers) and line impedance. For example, if a substation is connected to 5 transmission lines, its node degree is 5; for the thermal network: the relationship between pipeline flow and pressure drop is quantified through a hydraulic balance equation (such as a flow deviation formula). For example, the ratio of the actual flow to the designed flow (hydraulic deviation) of a pipeline is 0.85; for the natural gas network: the dynamic pressure-flow model in the unified energy path theory is used to describe the dynamic characteristics of pipelines; the physical connection relationships of these three types of networks are encoded into a unified multi-layer adjacency matrix. For example, if power node i is coupled with thermal node j through an electric boiler, the matrix element value is 1, otherwise it is 0. Finally, a topological sequence diagram including node attributes (such as voltage level, temperature threshold) and edge attributes (such as line capacity, pressure drop coefficient) is generated.

[0114] Step 112: Based on real-time operating state parameters, perform time series prediction on the node voltage deviation rate of the power grid, the return water temperature of the nodes in the thermal network, and the node pressure fluctuation gradient of the natural gas network to obtain multiple node state prediction values;

[0115] In the embodiments of the present invention, sliding window normalization is performed on real-time operating parameters (such as voltage, temperature, pressure). For example, the mean and variance are calculated with a 30-minute window; for the power grid, a long short-term memory network is used to predict the node voltage deviation rate, inputting the historical voltage sequence and load rate, and outputting the deviation rate for the next 15 minutes (such as ±2%); for the thermal network, the historical temperature and outdoor meteorological parameters are input into a bidirectional long short-term memory network to predict the return water temperature; for the natural gas network, a dynamic state estimation algorithm is used in combination with pressure gradient fluctuation data to predict the node pressure change (such as 0.2 MPa / min).

[0116] Step 113: Use elastic net regression to train the multiple node state prediction values to determine the multi-energy flow coupling factor weights. Based on the multi-energy flow coupling factor weights, linearly combine the multiple node state prediction values to calculate the multi-energy flow coupling strength. Based on the topological sequence diagram and the multi-energy flow coupling strength, construct a spatial distribution matrix representing the multi-energy flow coupling relationship, and combine the multi-energy flow coupling strength and real-time state operating parameters to generate multi-energy flow coupling data;

[0117] In this step, the multi-energy flow coupling strength is a parameter that quantifies the degree of dynamic influence between different energy networks and reflects the strength of the electro-thermal-gas interaction. The spatial distribution matrix is a matrix that characterizes the distribution characteristics of the multi-energy flow coupling relationship in physical space.

[0118] In the embodiment of the present invention, elastic net regression is used to train the predicted values of the multiple node states to determine the weights of the multi-energy flow coupling factors. For example, taking the power voltage deviation as the dependent variable and the thermal temperature and natural gas pressure gradient as the independent variables, key coupling factors are screened through L1 / L2 regularization (such as the thermal temperature weight is 0.3 and the natural gas pressure weight is 0.1); the predicted values of each node are weighted and summed according to the weights of the multi-energy flow coupling factors; the element values of the adjacency matrix of the topological sequence graph are replaced with the coupling strengths of the corresponding nodes (such as if the coupling strength between power node i and thermal node j is 0.8, then the matrix element is 0.8) to construct a spatial distribution matrix; combining the multi-energy flow coupling strength and the real-time state operation parameters, multi-energy flow coupling data is obtained.

[0119] Step 114: Using the spectral clustering algorithm and cosine similarity metric, perform dynamic slicing processing on the spatial distribution matrix to generate an initial slicing set;

[0120] In the embodiment of the present invention, the cosine similarity metric is used to calculate the cosine similarity between nodes. For example, the similarity between node A and node B = (A·B) / (||A||·||B||). If the similarity > 0.7, it is regarded as a strong association; based on this, the spectral clustering algorithm is used for slicing, that is, the Laplacian matrix of the spatial distribution matrix is normalized, and the first k eigenvectors are extracted (such as k = 5); the K-means algorithm is used to cluster the eigenvectors, and the network is divided into several initial slices (such as a certain city is divided into 3 slices) to generate an initial slicing set.

[0121] Step 115: Using an extreme gradient boosting classifier trained based on historical load data, generate a set of dynamic slicing trigger conditions, and combine the multi-energy flow coupling data, the initial slicing set, and the set of dynamic slicing trigger conditions to generate a set of dynamic slicing rules for the multi-energy flow coupling network;

[0122] In the embodiment of the present invention, using features such as historical slice load rate and coupling strength volatility, the extreme gradient boosting classifier is trained to output slice recombination trigger conditions (such as load rate > 85% and coupling strength change > 20%) to generate a set of dynamic slicing trigger conditions, and combine the multi-energy flow coupling data and the initial slicing set to generate dynamic slicing rules (such as slice A and slice B are merged during peak hours to balance the load).

[0123] The following is a specific example. An industrial park includes power, heat, and natural gas networks and needs to achieve cross - energy collaborative optimization. By constructing a topological sequence diagram, the power nodes (substations) and heat nodes (heat exchange stations) are coupled through electric boilers, and the natural gas nodes (pressure regulating stations) and heat nodes are connected through gas boilers; it is predicted that the voltage deviation rate of the power nodes in the next hour is ±3%, the fluctuation of the return water temperature of the heat is ±2°C, and the natural gas pressure gradient is 0.3 MPa / min; the multi - energy flow coupling intensity is calculated, and it is found that the electric - heat coupling is dominant (weight 0.6). A spatial distribution matrix is constructed and the key coupling paths are marked; the spectral clustering algorithm is used to divide the network into 3 sub - networks. Sub - network 1 is centered around the substation, sub - network 2 is centered around the heat exchange station, and sub - network 3 is centered around the pressure regulating station; the extreme gradient boosting classifier is used to trigger the merger of sub - network 1 and sub - network 2 during the peak load at noon, and the transmission margin is increased by 15%.

[0124] In the embodiment of the present invention, by constructing a topological sequence diagram to integrate the physical connection relationships of the power, heat, and gas networks, and combining elastic net regression to quantify the coupling strength weights, the multi - energy flow interaction efficiency is improved, the energy transmission path is optimized, and redundant energy exchange is reduced; by screening key coupling factors through elastic net regression and combining with a set of dynamic sub - network division rules, the energy utilization efficiency is improved; by combining the spectral clustering algorithm with the spatial weight matrix, the problem that the traditional sub - network division method has poor adaptability to non - linear coupling relationships is solved.

[0125] The present invention provides a specific embodiment. Step 111: Obtain the topological connection relationship of the power network, the hydraulic balance parameters of the heat network, and the pressure and flow dynamic relationship of the natural gas network to construct the physical connection relationship of the multi - energy flow coupling network. Combining the real - time operating state parameters and the physical connection relationship, a topological sequence diagram is generated, which specifically includes the following steps:

[0126] Step 121: Collect data on the connection relationships of power network equipment, analyze the physical connection paths from distribution transformers to user nodes, generate an adjacency matrix of the power network, and optimize the adjacency matrix of the power network by combining breaker state data and node voltage deviation rate to generate a dynamic topological feature matrix containing voltage deviation attributes;

[0127] In this step, the physical connection path refers to the actual electrical connection link from the distribution transformer to the user node in the power network, including the physical topological relationships between equipment such as cables, busbars, and switches. The adjacency matrix of the power network represents a weighted square matrix of the connection relationships between nodes in the power network, and its element values reflect the physical connection status and electrical parameters. The dynamic topological feature matrix refers to a dynamic weight matrix formed by superimposing the real - time state parameter of the node voltage deviation rate on the basis of the adjacency matrix of the power network.

[0128] In the embodiments of the present invention, the cable connection relationship from the distribution transformer to the user node is obtained through the smart meter and the breaker status data acquisition device; the NetworkX library is used to construct the power network adjacency matrix, and the element values are determined by the breaker status (0 / 1) and the electrical distance; combined with the node voltage monitoring data, the power network adjacency matrix is extended with the node voltage deviation rate as the weight to generate the dynamic topology feature matrix. The optimization logic can be that when the node voltage deviation rate exceeds 5%, it is marked as an abnormal connection and triggers topology reconstruction, and a dynamic topology feature matrix is generated.

[0129] Step 122: According to the supply and return water temperature data and the flow time series data, predict the return water temperature deviation rate, generate a standardized temperature deviation rate sequence, calculate the temperature deviation rate difference between adjacent nodes, and based on the temperature deviation rate difference, mark the hydraulic balance state of the heat transfer branches in the heat network to generate a heat network balance coefficient matrix;

[0130] In this step, the hydraulic balance state marking refers to classifying and identifying the hydraulic balance degree of the heat transfer branches to reflect whether the flow distribution meets the design requirements. The heat network balance coefficient matrix represents the coefficient matrix of the hydraulic balance degree of the heat transfer branches and is used to quantify the adjustment priority.

[0131] In the embodiments of the present invention, a long short-term memory network is used to predict the supply and return water temperature time series data to generate a temperature deviation rate sequence; after standardizing the temperature deviation rate, calculate the temperature difference between adjacent nodes, and a difference exceeding 2°C is marked as an unbalanced branch; based on the temperature difference and the flow data, optimize the branch resistance coefficient through a linear programming algorithm to construct a heat network balance coefficient matrix.

[0132] Step 123: Based on the pressure monitoring data of the natural gas network, generate node pressure distribution data, and jointly predict and process the pipeline pressure decay rate and the flow fluctuation gradient by combining the Weibull distribution model and the long short-term memory network to generate a natural gas network dynamic gradient matrix;

[0133] In this step, the natural gas network dynamic gradient matrix represents the spatio-temporal distribution matrix of the pipeline pressure decay rate and the flow fluctuation gradient and is used to predict the network dynamic behavior.

[0134] In the embodiments of the present invention, based on the pressure monitoring data of the natural gas network collected by the Internet of Things sensors, node pressure data is formed, and based on this, a Gaussian kernel density distribution map is generated; the Weibull distribution model is used to fit the pipeline aging curve to predict the pipeline pressure decay rate; the long short-term memory network is used to predict the flow fluctuation gradient, and a natural gas network dynamic gradient matrix is generated by combining the pipeline pressure decay rate. The specific implementation process may be that the Weibull distribution parameters are fitted by the maximum likelihood estimation method, and the input of the long short-term memory network includes historical pressure, flow, and temperature, and the output is the gradient sequence for the next 30 minutes.

[0135] Step 124: Generate a multi-energy flow physical connection relationship matrix based on the dynamic topology feature matrix, the thermal network balance coefficient matrix, and the natural gas network dynamic gradient matrix, and dynamically optimize the coupling weights of the multi-energy flow physical connection relationship matrix based on the real-time load fluctuation rate to generate a topology sequence diagram;

[0136] In this step, the multi-energy flow physical connection relationship matrix refers to the coupling relationship matrix integrating the power, heat, and gas networks, which characterizes the physical constraints and energy transfer paths of multi-energy flow interaction.

[0137] In the embodiment of the present invention, the dynamic topology feature matrix of the power network, the thermal network balance coefficient matrix of the thermal network, and the natural gas network dynamic gradient matrix of natural gas are mapped to a three-dimensional tensor by nodes, so as to generate a physical connection relationship matrix; based on the real-time load fluctuation rate, the elastic net regression is used to dynamically adjust the matrix weights, and its objective function is set to minimize the multi-energy flow transmission loss, and the elastic net regularization parameter is set to screen key coupling factors, and finally a topology sequence diagram is generated.

[0138] Taking the multi-energy network regulation in the industrial park as an example, 3 abnormal connections are identified and reconstructed through the dynamic topology feature matrix; the thermal network balance coefficient matrix marks 2 unbalanced branches (such as the temperature deviation rate difference is 3.1 °C), and the valve opening is adjusted to 65%; the natural gas network dynamic gradient matrix predicts that the pressure decay rate is 0.02 MPa / h, triggering the compressor frequency modulation. Combining the above results, the elastic net regression is used to determine the coupling weights of the power network: thermal network: natural gas network as 0.6:0.3:0.1, so as to construct a topology sequence diagram.

[0139] The embodiment of the present invention solves the limitations of traditional single-network analysis by constructing a multi-energy flow physical connection relationship matrix, realizes the optimization of the energy transmission path, improves the electro-thermal coupling efficiency, and at the same time reduces the loss of the natural gas network; combines the elastic net regression to dynamically adjust the coupling weights, so that the topology recombination speed is improved when the load changes suddenly, and the abnormal recovery time is shortened; introduces parameters such as node voltage deviation and temperature gradient, so that the present invention has a minute-level response ability; combines the prediction results of the long short-term memory network and the hydraulic balance mark to improve the stability under abnormal conditions.

[0140] The present invention provides a specific embodiment, step 114, using the spectral clustering algorithm and the cosine similarity metric to perform dynamic slicing processing on the spatial distribution matrix to generate an initial slice set, which specifically includes the following steps:

[0141] Step 131: Construct a node similarity matrix characterizing the coupling strength between nodes based on the spatial distribution matrix, the real-time state operation parameters, and the time decay factor, and the time decay factor is dynamically adjusted according to the real-time load fluctuation rate;

[0142] In this step, the time decay factor is a dynamic parameter used to quantify the decay degree of the coupling strength between nodes with the change of time or the load fluctuation rate, reflecting the timeliness of the coupling relationship. In the embodiments of the present invention, it is dynamically adjusted according to the real-time load fluctuation rate. The node similarity matrix is a symmetric matrix characterizing the coupling strength between nodes, and its element values are jointly determined by the spatial distribution matrix, real-time state parameters, and time decay factor.

[0143] In the embodiments of the present invention, parameters such as the spatial distribution matrix, real-time voltage deviation rate, and thermal return water temperature deviation are used to calculate the node state correlation through cosine similarity, and combined with the time decay factor, a node similarity matrix characterizing the coupling strength between nodes is constructed; among them, the time decay factor is calculated by combining the load fluctuation rate and historical load data.

[0144] Step 132: Generate a normalized coupling relationship matrix based on the node similarity matrix and the node degree distribution characteristics;

[0145] In this step, the node degree distribution characteristics are used to describe the statistical distribution characteristics of the number of node connections (degrees) in the network, reflecting the global structure of the multi-energy flow network topology. The normalized coupling relationship matrix refers to a matrix obtained by standardizing the node similarity matrix to eliminate the influence of node degree differences on the multi-energy flow coupling strength.

[0146] In the embodiments of the present invention, based on the node similarity matrix, the number of connections (degrees) of each node is calculated to generate a diagonal matrix, and the diagonal elements of which are the sum of the degrees of each node; each element of the original matrix is divided by the product of the square roots of the corresponding node degrees to eliminate the excessive influence of high-connection nodes on the coupling relationship; through statistical tests, it is verified whether the node degrees conform to a preset distribution (such as scale-free characteristics), and if they deviate, the normalization coefficient is adjusted to ensure the rationality of the network structure.

[0147] Step 133: Use the manifold learning algorithm to perform dimensionality reduction processing on the normalized coupling relationship matrix to generate a dimensionality-reduced feature matrix;

[0148] In the embodiments of the present invention, the manifold learning algorithm (such as the isometric mapping algorithm) is used to calculate the low-dimensional embedding based on the node similarity matrix and retain the geodesic distance between nodes; the normalized coupling relationship matrix is eigen-decomposed, and the eigenvectors corresponding to the first k largest eigenvalues are extracted to form a low-dimensional feature matrix; it is projected into a two-dimensional space through principal component analysis to observe whether the clustering structure is separated (such as the visualization of smart city energy slices). After verifying that the clustering structure has been separated, a dimensionality-reduced feature matrix is generated.

[0149] Step 134: Based on the dimension-reduced feature matrix, optimize the standard deviation of the multi-energy flow coupling strength within the shard and the cross-shard channel ratio constraint by combining the improved clustering algorithm integrating the balanced regularization term and the alternating direction multiplier method to generate an initial shard set;

[0150] In this step, the standard deviation of the multi-energy flow coupling strength is used to measure the dispersion degree of the multi-energy flow coupling strength within the shard, and is used to optimize the stability of the energy interaction within the shard. The cross-shard channel ratio constraint is used to limit the ratio of the number of connection channels between different shards to prevent the over-concentration or dispersion of the energy transmission path.

[0151] In the embodiment of the present invention, the improved clustering algorithm integrating the balanced regularization term integrates the L2 regularization term in the objective function. By penalizing the dispersion degree of the standard deviation of the coupling strength within the shard, it suppresses the influence of abnormal shards on the overall structure, and at the same time constrains the cross-shard channel ratio not to exceed the threshold. The L2 regularization balances the coupling strength of each node within the shard by shrinking the weights rather than zeroing them, improving the stability of the model. The alternating direction multiplier method decomposes the shard optimization problem into two sub-problems: shard division and multiplier variable update. Shard division: Based on the current multiplier variable, minimize the weighted sum of the standard deviation within the shard and the cross-shard channel ratio. Multiplier update: Dynamically adjust the Lagrange multiplier according to the degree of constraint violation to force the shard division to satisfy the global constraint. Distributed optimization is achieved through alternating iteration, and the convergence speed is faster than that of traditional centralized methods.

[0152] In the embodiment of the present invention, the time decay factor is introduced to adjust the coupling weight in real time according to the load fluctuation, adapting to the time-varying characteristics of the energy network; the node similarity matrix integrates the spatial topology and real-time parameters to solve the problem that traditional static models cannot capture dynamic interactions; through the standard deviation constraint and channel ratio limit, a robust structure with high aggregation inside and low coupling outside the energy shard is realized, improving the multi-energy flow collaborative transmission efficiency.

[0153] The present invention provides a specific embodiment. Step 102: Based on the dynamic shard rule set, calculate a dynamic threshold. When any load factor exceeds the dynamic threshold, generate a dynamic adjustment instruction for the shard boundary. The load factors include the node voltage deviation rate, the node pressure fluctuation gradient, and the return water temperature deviation rate, and specifically include the following steps:

[0154] Step 201: Based on the dynamic shard rule set, combine the dynamic mean, dynamic standard deviation, and elastic coefficient of the load factors within the sliding time window to calculate the dynamic threshold;

[0155] In this step, the dynamic mean refers to the moving average of the load factor (such as CPU usage rate, energy demand, etc.) within a sliding time window, reflecting the short-term trend of the load. The dynamic standard deviation is an index of the volatility of the load factor within the sliding time window, used to quantify the degree of deviation of the load from the mean. The elasticity coefficient is a weight parameter used for dynamic threshold sensitivity adjustment, which is dynamically adjusted according to the load fluctuation characteristics (such as dynamic standard deviation, historical extreme values).

[0156] In the embodiment of the present invention, the exponential weighted moving average method is used to calculate the dynamic mean to eliminate the interference of instantaneous fluctuations; the dynamic standard deviation is used to quantify the volatility of the load deviation from the mean; the elasticity coefficient dynamically adjusts the sensitivity according to the load historical extreme value and the dynamic standard deviation (such as relaxing the threshold tolerance for fluctuations when k = 1.5). Combining the above, the dynamic threshold is calculated to achieve the adaptive change of the threshold with the system state.

[0157] Step 202: Based on the multi-energy flow coupling strength, calculate the standard deviation of the multi-energy flow coupling strength corresponding to each shard, and select the target shard whose multi-energy flow coupling strength standard deviation is higher than the preset standard value;

[0158] In the embodiment of the present invention, the standard deviation of the multi-energy flow coupling strength of each shard is calculated through the multi-energy flow coupling strength, and the target shards with a multi-energy flow coupling strength higher than the preset standard value (such as 0.1) are screened. This process combines the hash sharding algorithm (such as consistent hashing) to locate the highly discrete shards, ensuring that the coupling imbalance area is preferentially processed during shard rebalancing.

[0159] Step 203: According to the dynamic threshold and the elasticity coefficient, calculate the load overflow ratio to generate a shard contraction coefficient or expansion coefficient, and update the energy interaction weight matrix of adjacent shards in combination with the graph neural network to obtain the updated energy interaction weight matrix;

[0160] In this step, the energy interaction weight matrix is a weighted graph describing the energy transmission ability between shards.

[0161] In the embodiment of the present invention, based on the dynamic threshold and the elasticity coefficient, calculate the deviation ratio of the current load relative to the dynamic threshold to obtain the load overflow ratio. If the load overflow ratio > 0, generate an expansion coefficient and expand the shard capacity proportionally; if the load overflow ratio < 0, generate a contraction coefficient to reduce redundant resources; use the graph neural network to calculate the edge weights, and by introducing a pruning threshold, remove the weak similarity edges, and preferentially enhance the interaction weights of the low-load shards to obtain the updated energy interaction weight matrix, optimizing the energy transmission efficiency.

[0162] Step 204: According to the updated energy interaction weight matrix, set the transmission power gradient and the lowest occupancy ratio threshold of the cross-shard energy interaction channel during the shard boundary adjustment process;

[0163] In this step, the transmission power gradient refers to the maximum allowable power change rate across the shard channels, which is used to prevent network oscillations caused by sudden changes in transmission power. The minimum ratio threshold refers to the minimum ratio requirement of the shard channels in the total transmission capacity, which avoids energy islanding between shards and ensures system redundancy.

[0164] In the embodiments of the present invention, the transmission power gradient is set to ±10% / second (to prevent sudden power changes), and the minimum ratio threshold ≥ 15% (for example, the core heat source shard in an industrial park needs to reserve 20% channel redundancy).

[0165] Step 205: Based on the preset conflict arbitration rule, when it is detected that any load factor exceeds the dynamic threshold, combine the shard shrinkage coefficient or expansion coefficient, the updated energy interaction weight matrix, the transmission power gradient, and the minimum ratio threshold to generate a dynamic adjustment instruction for the shard boundary;

[0166] In this step, the preset conflict arbitration rule is a decision-making logic for resolving resource competition and conflicts during the shard adjustment process.

[0167] In the embodiments of the present invention, based on the preset conflict arbitration rule, when it is detected that any load factor exceeds the threshold, the alternating direction method of multipliers is used to decompose the optimization problem, including the shard partitioning sub-problem and the multiplier update sub-problem. Specifically, it includes minimizing the weighted sum of the standard deviation within the shard and the shard channel ratio, adjusting the Lagrange multiplier according to the degree of constraint violation, and forcing the satisfaction of the global constraint. Finally, a new shard scheme is generated by the extreme gradient boosting classifier within 10 seconds, and the weight adjustment is coordinated by combining the game theory model, and finally a dynamic adjustment instruction for the shard boundary is generated.

[0168] In the embodiments of the present invention, the shard expansion or contraction is triggered by the load overflow ratio, and the edge weights of the graph neural network are dynamically updated to achieve load balancing; the pruning threshold filters out noisy edges (such as weak connection lines in the power grid), which improves the resource allocation priority of high-load shards; by combining the shard shrinkage coefficient or expansion coefficient, the stability and efficiency of the shard adjustment are ensured. Through the synergistic effect of the dynamic threshold, the graph neural network, and the pruning strategy, the load response speed and resource utilization rate of the multi-energy flow network are improved. The preset conflict arbitration rule effectively avoids resource competition failures and improves the success rate of abnormal rollback.

[0169] Figure 2 For the embodiments of the present invention, a structural schematic diagram of a distributed big data real-time stream processing method and system is provided, as Figure 2 shown, the system includes:

[0170] A generation module 21, configured to generate a topological sequence diagram based on a power network, a heat network, a natural gas network, and real-time operation status parameters, and generate a set of dynamic sharding rules for a multi-energy flow coupling network based on the topological sequence diagram and the real-time operation status parameters, where the set of dynamic sharding rules includes a power network topological structure, multi-energy flow coupling data, and dynamic sharding trigger conditions;

[0171] A calculation module 22, configured to calculate a dynamic threshold based on the set of dynamic sharding rules, and generate a dynamic adjustment instruction for the sharding boundary when any load factor exceeds the dynamic threshold, where the load factors include a node voltage deviation rate, a node pressure fluctuation gradient, and a return water temperature deviation rate;

[0172] A determination module 23, configured to construct a hierarchical collaborative optimization architecture based on the dynamic adjustment instruction and the multi-energy flow coupling data to determine target energy interaction constraints and target timing optimization results between shards;

[0173] A processing module 24, configured to generate a distributed dynamic sharding configuration instruction for the multi-energy flow coupling network based on the target energy interaction constraints, the target timing optimization results, and the dynamic adjustment instruction, and use the distributed dynamic sharding configuration instruction to drive a data flow processing unit within a shard to perform distributed big data real-time stream processing.

[0174] Figure 2 The described distributed big data real-time stream processing system can execute Figure 1 The distributed big data real-time stream processing method described in the illustrated embodiment, and its implementation principle and technical effects will not be elaborated. For the distributed big data real-time stream processing method and system in the above embodiment, the specific manners in which each module and unit perform operations have been described in detail in the embodiment related to the method, and will not be elaborated here.

[0175] In a possible design, Figure 2 The distributed big data real-time stream processing method and system in the illustrated embodiment can be implemented as a computing device, such as Figure 3 shown, and the computing device can include a storage component 31 and a processing component 32;

[0176] The storage component 31 stores one or more computer instructions, where the one or more computer instructions are called and executed by the processing component 32.

[0177] The processing component 32 is configured to: generate a topological sequence diagram based on the power network, the thermal network, the natural gas network, and the real-time operating state parameters; generate a set of dynamic sharding rules for the multi-energy flow coupling network based on the topological sequence diagram and the real-time operating state parameters, where the set of dynamic sharding rules includes the power network topological structure, the multi-energy flow coupling data, and the dynamic sharding trigger conditions; calculate a dynamic threshold based on the set of dynamic sharding rules, and generate a dynamic adjustment instruction for the sharding boundary when any load factor exceeds the dynamic threshold, where the load factors include the node voltage deviation rate, the node pressure fluctuation gradient, and the return water temperature deviation rate; construct a hierarchical collaborative optimization architecture based on the dynamic adjustment instruction and the multi-energy flow coupling data to determine the target energy interaction constraints and the target timing optimization results between shards; generate a distributed dynamic sharding configuration instruction for the multi-energy flow coupling network based on the target energy interaction constraints, the target timing optimization results, and the dynamic adjustment instruction, and use the distributed dynamic sharding configuration instruction to drive the data flow processing units within the shards to perform distributed big data real-time stream processing.

[0178] Among them, the processing component 32 may include one or more processors to execute computer instructions to complete all or part of the steps in the above method. Of course, the processing component may also be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components for executing the above method.

[0179] The storage component 31 is configured to store various types of data to support the operation of the terminal. The storage component can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, a magnetic disk, or an optical disc.

[0180] Of course, the computing device may also necessarily include other components, such as input / output interfaces, display components, communication components, etc.

[0181] The input / output interface provides an interface between the processing component and the peripheral interface module, and the above peripheral interface module may be an output device, an input device, etc.

[0182] The communication component is configured to facilitate communication between the computing device and other devices in a wired or wireless manner, etc.

[0183] Among them, the computing device can be a physical device or an elastic computing host provided by a cloud computing platform. In this case, the computing device can refer to a cloud server. The above-mentioned processing components, storage components, etc. can be basic server resources rented or purchased from a cloud computing platform.

[0184] An embodiment of the present invention also provides a computer storage medium storing a computer program, which when executed by a computer can implement the above-mentioned Figure 1 distributed big data real-time stream processing method shown in the embodiment.

[0185] Those skilled in the art can clearly understand that for the convenience and conciseness of description, the specific working processes of the above-described systems, devices, and units can refer to the corresponding processes in the foregoing method embodiments and will not be elaborated herein.

[0186] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. Those of ordinary skill in the art can understand and implement it without creative efforts.

[0187] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the essence of the above technical solution, or the part that contributes to the prior art, can be embodied in the form of a software product. The computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., including several instructions for causing a computer device (which can be a personal computer, server, or network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0188] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit them. Although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments or equivalently replace some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A distributed big data real-time stream processing method, characterized in that: include: Based on the power network, the heat network, the natural gas network and the real-time operation status parameters, a topology sequence diagram is generated, and based on the topology sequence diagram and the real-time operation status parameters, a dynamic slicing rule set of the multi-energy flow coupling network is generated, wherein the dynamic slicing rule set includes the power network topology structure, the multi-energy flow coupling data and the dynamic slicing triggering conditions; Based on the dynamic slicing rule set, a dynamic threshold is calculated, and when any load factor exceeds the dynamic threshold, a dynamic adjustment instruction of the slicing boundary is generated, wherein the load factor includes a node voltage deviation rate, a node pressure fluctuation gradient, and a return water temperature deviation rate; Based on the dynamic adjustment instructions and the multi-energy flow coupling data, a hierarchical collaborative optimization architecture is constructed to determine target energy interaction constraints and target timing optimization results between slices; Based on the target energy interaction constraint, the target timing optimization result and the dynamic adjustment instruction, a distributed dynamic sharding configuration instruction of the multi-energy flow coupling network is generated, and the distributed dynamic sharding configuration instruction is used to drive the data flow processing unit in the shard to perform distributed big data real-time flow processing.

2. The method according to claim 1, characterized in that Based on the dynamic adjustment instruction and the multi-energy flow coupling data, a hierarchical collaborative optimization architecture is constructed to determine the target energy interaction constraints and target timing optimization results between the slices, including: Based on the core layer node betweenness in the power network and the multi-energy flow coupling data, a dynamic weight matrix is ​​generated by combining the multi-energy flow coupling factor weight and the line capacity weight; A distributed consistency optimization algorithm is used to solve the output priorities of photovoltaic and energy storage devices in the shard in parallel, generate timing optimization instructions for the devices in the shard, and combine the pre-regulation instructions of cogeneration to generate the verified shard boundary power demand constraints; Based on the verified shard boundary power demand constraint and the dynamic weight matrix, collaboratively optimize the cross-shard energy interaction rules to generate inter-shard energy interaction constraints and cross-shard timing collaboration rules; Based on the inter-shard energy interaction constraints and the cross-shard timing coordination rules, when the difference between the core layer node betweenness and the preset betweenness value exceeds the limit threshold, the topological structure of the shard is reconstructed, and the reorganized shard energy interaction constraints and timing optimization rules are generated in combination with the dynamic weight matrix and the pre-adjustment instructions; When an abnormality is detected in the node voltage deviation coefficient or the thermal return water temperature within the slice, the dynamic weight matrix is ​​frozen using the reorganized slice energy interaction constraints and the timing optimization rules, and the adjacent slice transmission margins are redistributed to generate safe mode energy interaction constraints and abnormal timing optimization results. When the abnormality is resolved, the dynamic weight matrix is ​​optimized based on the safe mode energy interaction constraints to generate target energy interaction constraints and target timing optimization results.

3. The method according to claim 2, characterized in that A distributed consistency optimization algorithm is used to solve the output priorities of photovoltaic and energy storage devices in the shard in parallel, generate timing optimization instructions for the devices in the shard, and combine the pre-regulation instructions of cogeneration to generate the verified shard boundary power demand constraints, including: Based on the irradiance fluctuation coefficient of photovoltaic equipment, the discrete gradient of the state of charge of energy storage equipment and the topological connectivity of equipment, a dynamic priority weight matrix of equipment in the shard is constructed; Performing distributed consistency optimization on the dynamic priority weight matrix using an alternating direction multiplier method to generate timing optimization instructions; The ratio of the return water temperature extreme difference within the sliding time window to the set threshold is used as the reference sensitivity, and the reference sensitivity and the historical time lag data are combined to generate a time lag compensation coefficient, and the reference sensitivity and the time lag compensation coefficient are convolved to generate the intensity parameter of the pre-regulation instruction; Using a bidirectional long short-term memory network, performing cross-time step correlation analysis on the intensity parameters of the timing optimization instruction and the pre-adjustment instruction to obtain a correlation analysis result, and calculating a preliminary boundary power demand constraint based on a load factor, the dynamic priority weight matrix and the correlation analysis result; A constraint validity verification mechanism is constructed, a feasibility index is defined, and the feasibility index is verified using the constraint validity verification mechanism. When the feasibility index is lower than a preset threshold, the dynamic priority weight matrix is ​​reconstructed to generate a verified boundary power demand constraint.

4. The method according to claim 1, characterized in that: Based on the power network, the heat network, the natural gas network and the real-time operation status parameters, a topology sequence diagram is generated. Based on the topology sequence diagram and the real-time operation status parameters, a dynamic slicing rule set of the multi-energy flow coupling network is generated. The dynamic slicing rule set includes the power network topology structure, the multi-energy flow coupling data and the dynamic slicing triggering conditions, including: Obtain the topological connection relationship of the power network, the hydraulic balance parameters of the thermal network, and the pressure and flow dynamic relationship of the natural gas network to construct the physical connection relationship of the multi-energy flow coupling network, and generate a topological sequence diagram by combining the real-time operating state parameters and the physical connection relationship; Based on the real-time operating status parameters, the node voltage deviation rate of the power network, the node return water temperature of the thermal network, and the node pressure fluctuation gradient of the natural gas network are predicted in time series to obtain multiple node status prediction values; The plurality of node state prediction values ​​are trained by using elastic network regression to determine the weight of the multi-energy flow coupling factor, and based on the weight of the multi-energy flow coupling factor, the plurality of node state prediction values ​​are linearly combined to calculate the multi-energy flow coupling strength, and a spatial distribution matrix characterizing the multi-energy flow coupling relationship is constructed based on the topological sequence diagram and the multi-energy flow coupling strength, and the multi-energy flow coupling data is generated by combining the multi-energy flow coupling strength and the real-time state operation parameters; Using a spectral clustering algorithm and a cosine similarity metric, the spatial distribution matrix is ​​dynamically sliced ​​to generate an initial slice set; A dynamic sharding trigger condition set is generated by using an extreme gradient boosting classifier trained based on historical load data, and a dynamic sharding rule set of a multi-energy flow coupling network is generated by combining the multi-energy flow coupling data, the initial sharding set and the dynamic sharding trigger condition set.

5. The method according to claim 4, characterized in that The topological connection relationship of the power network, the hydraulic balance parameters of the thermal network, and the pressure and flow dynamic relationship of the natural gas network are obtained to construct the physical connection relationship of the multi-energy flow coupling network. Combined with the real-time operating state parameters and the physical connection relationship, a topological sequence diagram is generated, including: Collecting power network equipment connection relationship data, parsing the physical connection path from the distribution transformer to the user node, generating a power network adjacency matrix, combining circuit breaker status data and node voltage deviation rate, optimizing the power network adjacency matrix, and generating a dynamic topology feature matrix containing voltage deviation attributes; According to the supply and return water temperature data and flow time series data, the return water temperature deviation rate is predicted, and a standardized temperature deviation rate sequence is generated to calculate the temperature deviation rate difference of adjacent nodes. Based on the temperature deviation rate difference, the hydraulic balance state of the thermal branches in the thermal network is marked to generate a thermal network balance coefficient matrix; Based on the pressure monitoring data of the natural gas network, the node pressure distribution data is generated, and the pipeline pressure decay rate and flow fluctuation gradient are jointly predicted and processed by combining the Weibull distribution model and the long short-term memory network to generate the dynamic gradient matrix of the natural gas network; Based on the dynamic topological characteristic matrix, the thermal network balance coefficient matrix and the natural gas network dynamic gradient matrix, a multi-energy flow physical connection relationship matrix is ​​generated. Based on the real-time load fluctuation rate, the coupling weight of the multi-energy flow physical connection relationship matrix is ​​dynamically optimized to generate a topological sequence diagram.

6. The method according to claim 4, characterized in that The spatial distribution matrix is ​​dynamically sliced ​​using a spectral clustering algorithm and cosine similarity measurement to generate an initial slice set, including: Based on the spatial distribution matrix, the real-time state operating parameters and the time decay factor, a node similarity matrix characterizing the coupling strength between nodes is constructed, and the time decay factor is dynamically adjusted according to the real-time load fluctuation rate; Based on the node similarity matrix and node degree distribution characteristics, a normalized coupling relationship matrix is ​​generated; Using a manifold learning algorithm to perform dimensionality reduction processing on the normalized coupling relationship matrix to generate a dimensionality-reduced feature matrix; Based on the reduced-dimensional feature matrix, the improved clustering algorithm with integrated balanced regularization term and the alternating direction multiplier method are combined to optimize the standard deviation of the multi-energy flow coupling intensity within the shard and the cross-shard channel ratio constraints to generate an initial shard set.

7. The method according to claim 1, characterized in that Based on the dynamic sharding rule set, a dynamic threshold is calculated. When any load factor exceeds the dynamic threshold, a dynamic adjustment instruction of the sharding boundary is generated. The load factor includes a node voltage deviation rate, a node pressure fluctuation gradient, and a return water temperature deviation rate, including: Based on the dynamic sharding rule set, combined with the dynamic mean, dynamic standard deviation and elasticity coefficient of the load factor in the sliding time window, a dynamic threshold is calculated; Based on the multi-energy flow coupling strength, the standard deviation of the multi-energy flow coupling strength corresponding to each slice is calculated, and the target slice whose standard deviation of the multi-energy flow coupling strength is higher than a preset standard value is selected; According to the dynamic threshold and the elastic coefficient, the load overflow ratio is calculated to generate a shard contraction coefficient or an expansion coefficient, and the energy interaction weight matrix of adjacent shards is updated in combination with the graph neural network to obtain an updated energy interaction weight matrix; According to the updated energy interaction weight matrix, setting the transmission power gradient and minimum proportion threshold of the cross-shard energy interaction channel during the shard boundary adjustment process; Based on the preset conflict arbitration rules, when any load factor is detected to exceed the dynamic threshold, a dynamic adjustment instruction for the slice boundary is generated in combination with the slice contraction coefficient or expansion coefficient, the updated energy interaction weight matrix, the transmission power gradient and the minimum share threshold.

8. A distributed big data real-time stream processing system, characterized in that: include: A generation module, for generating a topology sequence diagram based on the power network, the heat network, the natural gas network and the real-time operation status parameters, and generating a dynamic slicing rule set of the multi-energy flow coupling network based on the topology sequence diagram and the real-time operation status parameters, wherein the dynamic slicing rule set includes the power network topology structure, the multi-energy flow coupling data and the dynamic slicing triggering condition; A calculation module, configured to calculate a dynamic threshold based on the dynamic sharding rule set, and generate a dynamic adjustment instruction for a sharding boundary when any load factor exceeds the dynamic threshold, wherein the load factors include a node voltage deviation rate, a node pressure fluctuation gradient, and a return water temperature deviation rate; A determination module, configured to construct a hierarchical collaborative optimization framework based on the dynamic adjustment instruction and the multi-energy flow coupling data to determine target energy interaction constraints and target timing optimization results between slices; A processing module is used to generate a distributed dynamic sharding configuration instruction for a multi-energy flow coupling network based on the target energy interaction constraint, the target timing optimization result and the dynamic adjustment instruction, and use the distributed dynamic sharding configuration instruction to drive the data flow processing unit within the shard to perform distributed big data real-time flow processing.

9. A computing device, characterized in that It comprises a processing component and a storage component; the storage component stores one or more computer instructions; the one or more computer instructions are used to be called and executed by the processing component to implement a distributed big data real-time stream processing method as described in any one of claims 1 to 7.

10. A computer storage medium, characterized in that: A computer program is stored, and when the computer program is executed by a computer, a distributed big data real-time stream processing method as described in any one of claims 1 to 7 is implemented.

Images