Distributed big data real-time stream processing method and system

Abstract

The invention provides a distributed big data real-time stream processing method and system, and the method comprises the steps: generating a topological sequence diagram based on a power network, a thermal network, a natural gas network and real-time operation state parameters, generating a dynamic fragmentation rule set of a multi-energy flow coupling network in combination with the real-time operation state parameters, and calculating a dynamic threshold value, when any load factor exceeds a dynamic threshold value, generating a dynamic adjustment instruction of the fragment boundary; based on the dynamic adjustment instruction and the multi-energy flow coupling data, constructing a hierarchical collaborative optimization architecture to determine a target energy interaction constraint and a target time sequence optimization result between fragments, and generating a distributed dynamic fragment configuration instruction of the multi-energy flow coupling network in combination with the dynamic adjustment instruction to perform distributed big data real-time flow processing; according to the technical scheme provided by the invention, real-time data processing and resource collaborative optimization of the multi-energy-flow coupling network are realized, and an efficient solution is provided for scenes such as smart energy and industrial Internet of Things.

Description

Technical Field

[0001] The embodiments of the present invention relate to the technical field of data processing, and in particular, to a distributed big data real-time stream processing method and system. Background Art

[0002] In the integrated energy system in the core area of a megacity (such as a central business district), the power network, heat network, and natural gas network are deeply coupled, including multiple devices such as gas turbines, combined heat and power units, distributed photovoltaics, and energy storage. It is necessary to achieve multi-energy flow collaborative optimization on a second-level time scale.

[0003] The core defects of the current technology are as follows: The existing solutions use historical topology data to divide fixed slices, resulting in a large error rate in slice optimization and an inability to meet the second-level response requirements. Summary of the Invention

[0004] The embodiments of the present invention provide a distributed big data real-time stream processing method and system to solve the core defects in the existing technology, namely, the large error rate in slice optimization and the inability to meet the second-level response requirements.

[0005] In a first aspect, the embodiments of the present invention provide a distributed big data real-time stream processing method, including: Generating a topology sequence diagram based on the power network, heat network, natural gas network, and real-time operating state parameters, and generating a set of dynamic slicing rules for the multi-energy flow coupling network based on the topology sequence diagram and real-time operating state parameters. The set of dynamic slicing rules includes the power network topology structure, multi-energy flow coupling data, and dynamic slicing trigger conditions; Calculating a dynamic threshold based on the set of dynamic slicing rules. When any load factor exceeds the dynamic threshold, generating a dynamic adjustment instruction for the slice boundary. The load factors include the node voltage deviation rate, node pressure fluctuation gradient, and return water temperature deviation rate; Constructing a hierarchical collaborative optimization architecture based on the dynamic adjustment instruction and the multi-energy flow coupling data to determine the target energy interaction constraints and target timing optimization results between slices; Generating a distributed dynamic slicing configuration instruction for the multi-energy flow coupling network based on the target energy interaction constraints, the target timing optimization results, and the dynamic adjustment instruction, and using the distributed dynamic slicing configuration instruction to drive the data flow processing units within the slices to perform distributed big data real-time stream processing.

[0006] Optionally, constructing a hierarchical collaborative optimization architecture based on the dynamic adjustment instruction and the multi-energy flow coupling data to determine the target energy interaction constraints and target timing optimization results between slices includes: Generate a dynamic weight matrix based on the betweenness centrality of the core layer nodes in the power grid and the multi-energy flow coupling data, in combination with the weights of the multi-energy flow coupling factors and the line capacity weights. Adopt a distributed consensus optimization algorithm to parallelly solve the output priorities of the photovoltaic and energy storage devices within the shard, generate the timing optimization instructions for the devices within the shard, and combine with the pre-regulation instructions of the combined heat and power generation to generate the verified power demand constraints at the shard boundary. Based on the verified power demand constraints at the shard boundary and the dynamic weight matrix, co-optimize the cross-shard energy interaction rules to generate the cross-shard energy interaction constraints and the cross-shard timing coordination rules. Based on the cross-shard energy interaction constraints and the cross-shard timing coordination rules, when the difference between the betweenness centrality of the core layer nodes and the preset betweenness value exceeds the limit threshold, reconstruct the topological structure of the shard, and combine with the dynamic weight matrix and the pre-regulation instructions to generate the reconstructed cross-shard energy interaction constraints and the timing optimization rules. When it is detected that the node voltage deviation coefficient or the thermal return water temperature within the shard is abnormal, freeze the dynamic weight matrix by using the reconstructed cross-shard energy interaction constraints and the timing optimization rules, and reallocate the transmission margin of the adjacent shards to generate the energy interaction constraints in the safe mode and the abnormal timing optimization results. When the abnormality is lifted, optimize the dynamic weight matrix based on the energy interaction constraints in the safe mode to generate the target energy interaction constraints and the target timing optimization results.

[0007] Optionally, adopt a distributed consensus optimization algorithm to parallelly solve the output priorities of the photovoltaic and energy storage devices within the shard, generate the timing optimization instructions for the devices within the shard, and combine with the pre-regulation instructions of the combined heat and power generation to generate the verified power demand constraints at the shard boundary, including: Based on the irradiance fluctuation coefficient of the photovoltaic device, the discrete gradient of the state of charge of the energy storage device, and the device topological connectivity, construct the dynamic priority weight matrix of the devices within the shard. Adopt the alternating direction multiplier method to perform distributed consensus optimization on the dynamic priority weight matrix to generate the timing optimization instructions. Use the ratio of the extreme difference of the return water temperature within the sliding time window to the set threshold as the reference sensitivity, combine the reference sensitivity and the historical time delay data to generate the time delay compensation coefficient, and perform convolution operation on the reference sensitivity and the time delay compensation coefficient to generate the intensity parameter of the pre-regulation instruction. Use the bidirectional long short-term memory network to perform cross-time-step correlation analysis on the timing optimization instructions and the intensity parameter of the pre-regulation instruction to obtain the correlation analysis result, and calculate the preliminary power demand constraints at the boundary based on the load factor, the dynamic priority weight matrix, and the correlation analysis result. Construct a constraint validity verification mechanism, define feasibility indicators, and use the constraint validity verification mechanism to verify the feasibility indicators. When the feasibility indicators are lower than a preset threshold, reconstruct the dynamic priority weight matrix to generate a verified boundary power demand constraint.

[0008] Optionally, based on the power grid, heat network, natural gas network, and real-time operating state parameters, generate a topological sequence diagram. Based on the topological sequence diagram and real-time operating state parameters, generate a set of dynamic slicing rules for the multi-energy flow coupling network. The set of dynamic slicing rules includes the power grid topological structure, multi-energy flow coupling data, and dynamic slicing trigger conditions, including: Obtain the topological connection relationship of the power grid, the hydraulic balance parameters of the heat network, and the pressure and flow dynamic relationship of the natural gas network to construct the physical connection relationship of the multi-energy flow coupling network. Combine the real-time operating state parameters and the physical connection relationship to generate a topological sequence diagram; Based on the real-time operating state parameters, perform time series prediction on the node voltage deviation rate of the power grid, the node return water temperature of the heat network, and the node pressure fluctuation gradient of the natural gas network to obtain multiple node state prediction values; Use elastic net regression to train the multiple node state prediction values to determine the multi-energy flow coupling factor weights. Based on the multi-energy flow coupling factor weights, linearly combine the multiple node state prediction values to calculate the multi-energy flow coupling strength. Based on the topological sequence diagram and the multi-energy flow coupling strength, construct a spatial distribution matrix representing the multi-energy flow coupling relationship. Combine the multi-energy flow coupling strength and real-time state operating parameters to generate multi-energy flow coupling data; Use the spectral clustering algorithm and cosine similarity metric to perform dynamic slicing processing on the spatial distribution matrix to generate an initial slicing set; Use an extreme gradient boosting classifier trained based on historical load data to generate a set of dynamic slicing trigger conditions. Combine the multi-energy flow coupling data, the initial slicing set, and the set of dynamic slicing trigger conditions to generate a set of dynamic slicing rules for the multi-energy flow coupling network.

[0009] Optionally, obtain the topological connection relationship of the power grid, the hydraulic balance parameters of the heat network, and the pressure and flow dynamic relationship of the natural gas network to construct the physical connection relationship of the multi-energy flow coupling network. Combine the real-time operating state parameters and the physical connection relationship to generate a topological sequence diagram, including: Collect power grid equipment connection relationship data, analyze the physical connection path from the distribution transformer to the user node to generate a power grid adjacency matrix. Combine the breaker state data and the node voltage deviation rate to optimize the power grid adjacency matrix to generate a dynamic topological feature matrix containing voltage deviation attributes; Predict the return water temperature deviation rate based on the supply and return water temperature data and the flow time series data, generate a standardized temperature deviation rate sequence, calculate the temperature deviation rate difference between adjacent nodes, and based on the temperature deviation rate difference, mark the hydraulic balance state of the heating branches in the heating network to generate a heating network balance coefficient matrix; Generate node pressure distribution data based on the pressure monitoring data of the natural gas network, and perform joint prediction processing on the pipeline pressure attenuation rate and the flow fluctuation gradient by combining the Weibull distribution model and the long short-term memory network to generate a natural gas network dynamic gradient matrix; Generate a multi-energy flow physical connection relationship matrix based on the dynamic topology feature matrix, the heating network balance coefficient matrix, and the natural gas network dynamic gradient matrix, and dynamically optimize the coupling weights of the multi-energy flow physical connection relationship matrix based on the real-time load fluctuation rate to generate a topology sequence diagram.

[0010] Optionally, use the spectral clustering algorithm and the cosine similarity metric to perform dynamic slicing processing on the spatial distribution matrix to generate an initial slice set, including: Construct a node similarity matrix representing the coupling strength between nodes based on the spatial distribution matrix, the real-time state operation parameters, and the time decay factor, and the time decay factor is dynamically adjusted according to the real-time load fluctuation rate; Generate a normalized coupling relationship matrix based on the node similarity matrix and the node degree distribution characteristics; Use the manifold learning algorithm to perform dimensionality reduction processing on the normalized coupling relationship matrix to generate a dimensionality-reduced feature matrix; Based on the dimensionality-reduced feature matrix, combine the improved clustering algorithm with the integrated balance regularization term and the alternating direction multiplier method to optimize the standard deviation of the multi-energy flow coupling strength within the slice and the cross-slice channel occupancy ratio constraint to generate an initial slice set.

[0011] Optionally, based on the dynamic slicing rule set, calculate a dynamic threshold, and when any load factor exceeds the dynamic threshold, generate a dynamic adjustment instruction for the slice boundary. The load factors include the node voltage deviation rate, the node pressure fluctuation gradient, and the return water temperature deviation rate, including: Calculate the dynamic threshold based on the dynamic slicing rule set, combined with the dynamic mean, dynamic standard deviation, and elastic coefficient of the load factors within the sliding time window; Calculate the standard deviation of the multi-energy flow coupling strength corresponding to each slice based on the multi-energy flow coupling strength, and select the target slice with the multi-energy flow coupling strength standard deviation higher than the preset standard value; Calculate the load overflow ratio according to the dynamic threshold and the elastic coefficient to generate a slice contraction coefficient or expansion coefficient, and update the energy interaction weight matrix of adjacent slices by combining the graph neural network to obtain the updated energy interaction weight matrix; According to the updated energy interaction weight matrix, set the transmission power gradient and the lowest occupancy ratio threshold of the cross-shard energy interaction channel during the shard boundary adjustment process; Based on the preset conflict arbitration rule, when it is detected that any load factor exceeds the dynamic threshold, combine the shard contraction coefficient or expansion coefficient, the updated energy interaction weight matrix, the transmission power gradient, and the lowest occupancy ratio threshold to generate a dynamic adjustment instruction for the shard boundary.

[0012] In a second aspect, an embodiment of the present invention provides a distributed big data real-time stream processing system, including: A generation module, configured to generate a topological sequence diagram based on a power network, a heat network, a natural gas network, and real-time operating state parameters, and generate a set of dynamic sharding rules for a multi-energy flow coupling network based on the topological sequence diagram and the real-time operating state parameters, where the set of dynamic sharding rules includes a power network topological structure, multi-energy flow coupling data, and a dynamic sharding trigger condition; A calculation module, configured to calculate a dynamic threshold based on the set of dynamic sharding rules, and generate a dynamic adjustment instruction for the shard boundary when any load factor exceeds the dynamic threshold, where the load factor includes a node voltage deviation rate, a node pressure fluctuation gradient, and a return water temperature deviation rate; A determination module, configured to construct a hierarchical collaborative optimization architecture based on the dynamic adjustment instruction and the multi-energy flow coupling data to determine the target energy interaction constraint and the target timing optimization result between shards; A processing module, configured to generate a distributed dynamic sharding configuration instruction for the multi-energy flow coupling network based on the target energy interaction constraint, the target timing optimization result, and the dynamic adjustment instruction, and use the distributed dynamic sharding configuration instruction to drive the data stream processing unit within the shard to perform distributed big data real-time stream processing.

[0013] In a third aspect, an embodiment of the present invention provides a computing device, including a processor and a memory, where a computer program is stored in the memory, and the processor is configured to run the computer program to execute the distributed big data real-time stream processing method according to any one of the first aspects.

[0014] In a fourth aspect, an embodiment of the present invention provides a computer storage medium, on which computer program instructions are stored, and when the computer program instructions are executed by a processor, the distributed big data real-time stream processing method according to any one of the first aspects is implemented.

[0015] In the embodiments of the present invention, based on the power network, the heat network, the natural gas network, and the real-time operating state parameters, a topological sequence diagram is generated. Based on the topological sequence diagram and the real-time operating state parameters, a set of dynamic slicing rules for the multi-energy flow coupling network is generated. The set of dynamic slicing rules includes the power network topological structure, the multi-energy flow coupling data, and the dynamic slicing trigger conditions. Based on the set of dynamic slicing rules, a dynamic threshold is calculated. When any load factor exceeds the dynamic threshold, a dynamic adjustment instruction for the slicing boundary is generated. The load factors include the node voltage deviation rate, the node pressure fluctuation gradient, and the return water temperature deviation rate. Based on the dynamic adjustment instruction and the multi-energy flow coupling data, a hierarchical collaborative optimization architecture is constructed to determine the target energy interaction constraints and the target timing optimization results between slices. Based on the target energy interaction constraints, the target timing optimization results, and the dynamic adjustment instruction, a distributed dynamic slicing configuration instruction for the multi-energy flow coupling network is generated, and the distributed big data real-time stream processing of the data flow processing unit within the slice is driven by using the distributed dynamic slicing configuration instruction. The technical solution provided by the present invention can monitor key indicators such as the node voltage deviation rate and the pressure fluctuation gradient in real time through the calculation of the set of dynamic slicing rules and the dynamic threshold. When the load factor exceeds the threshold, the system automatically triggers the slicing boundary adjustment instruction to avoid the problem of uneven load caused by the traditional static slicing strategy. The hierarchical collaborative optimization architecture combines the target energy interaction constraints and the timing optimization results to optimize the data flow interaction efficiency between slices. The set of dynamic slicing rules is generated based on the topological sequence diagram and the real-time operating state parameters to ensure that the data flow processing unit can respond immediately to the network state changes. The priority and order of data processing are controlled by the target timing optimization results to avoid the delay problem of the traditional batch processing system. The integration and dynamic slicing of the coupling data of the power, heat, and natural gas networks support the joint dispatching of multi-energy systems. The adjustment instruction after the dynamic slicing rule is triggered can quickly reconstruct the slicing boundary in case of local node failures to ensure the overall stability of the system. The set of slicing rules can be dynamically expanded according to the network topology changes and real-time data to adapt to complex scenarios such as power network expansion and new energy access. By the hierarchical collaborative optimization architecture (such as separating the intra-slice processing and the inter-slice coordination), the system complexity is reduced, and it is applicable to large-scale distributed environments. Generally speaking, the present invention realizes the real-time data processing and resource collaborative optimization of the multi-energy flow coupling network, and is significantly superior to the traditional static slicing or single-energy network processing methods in terms of real-time performance, load balancing, system stability, and scalability, providing an efficient solution for scenarios such as smart energy and industrial Internet of Things.Furthermore, the present invention realizes the efficient coordination and dynamic security control of the multi-energy flow system through a hierarchical collaborative optimization architecture. By combining a dynamic weight matrix and a distributed consensus optimization algorithm, based on the output priority of photovoltaic and energy storage devices within a slice and the pre-regulation instructions of combined heat and power generation, power constraints at the slice boundary and cross-slice interaction rules are generated, realizing the dynamic coordination and topological reconstruction of energy interaction between slices in a multi-energy flow coupling scenario. At the same time, through anomaly detection, the safety mode is triggered (freezing the weight matrix and adjusting the transmission margin). By combining the reorganized slice constraints and optimization rules, it can quickly switch to the safe operation state when voltage or thermal parameters are abnormal, and adaptively optimize the weight matrix after the anomaly is lifted, ultimately ensuring the stability and convergence of the system; effectively improving the response speed and robustness of multi-energy flow calculation, and solving the problems of dynamic optimization and safety boundary control in complex coupling scenarios.

[0016] These aspects or other aspects of the present invention will be more clearly understood in the following description of the embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the drawings in the following description are some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0018] Figure 1 It is a flowchart of a distributed big data real-time stream processing method provided by an embodiment of the present invention; Figure 2 It is a schematic structural diagram of a distributed big data real-time stream processing system provided by an embodiment of the present invention; Figure 3 It is a schematic structural diagram of a computing device provided by an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0019] In order to enable those skilled in the art to better understand the solutions of the present invention, the following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the drawings in the embodiments of the present invention.

[0020] In some of the processes described in the specification, claims, and above-mentioned drawings of the present invention, a plurality of operations appear in a specific order. However, it should be clearly understood that these operations may not be executed in the order in which they appear herein or may be executed in parallel. The serial numbers of the operations, such as 101, 102, etc., are only used to distinguish different operations, and the serial numbers themselves do not represent any execution order. In addition, these processes may include more or fewer operations, and these operations may be executed in sequence or in parallel. It should be noted that the descriptions such as "first" and "second" herein are used to distinguish different messages, devices, modules, etc., and do not represent a sequence, nor do they limit that "first" and "second" are of different types.

[0021] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative efforts shall fall within the protection scope of the present invention.

[0022] Figure 1 The flowchart of a distributed big data real-time stream processing method provided for an embodiment of the present invention is as Figure 1 shown, and the method includes: The present invention focuses on solving the problems of the static nature of traditional sharding strategies in multi-energy flow coupling networks (electricity, heat, natural gas), insufficient adaptability to load fluctuations, and inability to meet the second-level response requirements. Existing technologies usually adopt fixed sharding rules, which are difficult to cope with the dynamic changes of multi-energy network topologies and real-time load differences (such as node voltage deviations, pressure fluctuation gradients, etc.), resulting in low energy interaction efficiency between shards and unbalanced resource allocation. This solution constructs a set of dynamic sharding rules based on the topological sequence diagram of the multi-energy flow coupling network and real-time operating parameters (such as voltage, pressure, temperature), associates the power network topology structure with multi-energy flow data, and realizes the strong coupling of sharding logic and physical network; secondly, a dynamic threshold calculation mechanism is introduced, and combined with the mean, standard deviation, and elasticity coefficient of the load factor within a sliding time window, the sharding boundary is adjusted in real time to solve the problem of response lag caused by fixed thresholds in traditional methods; finally, through a hierarchical collaborative optimization architecture, the target energy interaction constraints and timing optimization results are integrated to drive the processing units within the shards to execute according to dynamic configuration instructions, overcoming the defect of insufficient collaborative optimization ability between shards in existing technologies. This solution deeply integrates the multi-energy flow coupling characteristics with dynamic sharding technology, significantly improving the resource utilization rate and stability of real-time stream processing in complex energy scenarios. Based on this, the present invention provides a distributed big data real-time stream processing method, as Figure 1 , including: Step 101: Generate a topological sequence diagram based on the power network, thermal network, natural gas network, and real-time operating state parameters. Based on the topological sequence diagram and real-time operating state parameters, generate a set of dynamic sharding rules for the multi-energy flow coupling network. The set of dynamic sharding rules includes the power network topological structure, multi-energy flow coupling data, and dynamic sharding trigger conditions. In this step, the topological sequence diagram refers to a dynamic visualization model that integrates the physical connection relationships and real-time operating parameters of the power, thermal, and natural gas networks. By abstracting the multi-dimensional correlation relationships of nodes and edges, it characterizes the spatio-temporal distribution characteristics of the multi-energy flow coupling network. The set of dynamic sharding rules refers to a set of logical rules for dynamically partitioning the multi-energy flow coupling network based on the topological structure and real-time parameters in the topological sequence diagram, aiming to optimize resource allocation and fault isolation capabilities. The multi-energy flow coupling data refers to a set of parameters that describe the dynamic interaction relationships and coordinated operating states between energy networks such as power, thermal, and natural gas, and is used to quantify the energy conversion efficiency, transmission constraints, and economic correlations across energy networks. It is the core input of the set of dynamic sharding rules for the multi-energy flow coupling network.

[0023] In the embodiment of the present invention, first, extract the bus-branch relationship of the power network, the pipeline-heat exchange station connection of the thermal network, and the gas source-compressor topology of the natural gas network through a graph database. Combine the parameters such as node voltage, pipeline pressure, and return water temperature collected in real time by the data acquisition and monitoring system to construct a topological sequence diagram. Subsequently, use the community discovery algorithm to identify the highly coupled subnets in the network, and combine the equipment operation constraints (such as the output upper limit of the combined heat and power unit) to generate a set of dynamic sharding rules, which includes: the power network topological structure, that is, divide the key hubs based on the node betweenness centrality. For example, set the substations with a voltage level ≥ 220 kV as the core of the subnet; the multi-energy flow coupling data, that is, calculate the cross-energy interaction weights through elastic net regression. For example, the gas-electric conversion efficiency weight of the power-to-gas equipment is 0.92; the dynamic sharding trigger conditions, for example, set trigger parameters such as the node voltage deviation rate threshold (±5%) and pressure fluctuation gradient (3 kPa / s).

[0024] Step 102: Calculate a dynamic threshold based on the set of dynamic sharding rules. When any load factor exceeds the dynamic threshold, generate a dynamic adjustment instruction for the sharding boundary. The load factors include the node voltage deviation rate, node pressure fluctuation gradient, and return water temperature deviation rate. In this step, the dynamic threshold refers to a critical value calculated adaptively based on real-time operating state parameters and algorithms, and is used to determine whether the system is in an abnormal state or needs to trigger an adjustment action. The dynamic adjustment instruction refers to a control command automatically generated by the system when the load factor exceeds the dynamic threshold, and is used to re-divide the sharding boundary of the multi-energy flow coupling network, optimize resource allocation, or isolate faults.

[0025] In the embodiments of the present invention, a sliding window statistical method is used to calculate the dynamic threshold. For example, for load factors such as the node voltage deviation rate and the pressure fluctuation gradient, the mean value and the standard deviation are calculated with a 30 - second window, and the dynamic threshold is set to the mean value ± 2 times the standard deviation. When the return water temperature deviation rate exceeds the threshold, a dynamic adjustment instruction for generating the shard boundary is triggered. This instruction is generated through a rule engine. For example, for the adjustment of the shard boundary, if the voltage of a certain power sub - network exceeds the limit, a breadth - first search method is used to locate adjacent load points that can be accessed, and a sub - network splitting instruction is generated. For resource re - allocation, according to the real - time load rate, the energy interaction capacity of each shard is allocated through a linear programming model.

[0026] Step 103: Based on the dynamic adjustment instruction and the multi - energy flow coupling data, construct a hierarchical collaborative optimization architecture to determine the target energy interaction constraints and the target timing optimization results between shards; In this step, the target energy interaction constraints refer to the boundary conditions for energy transmission between shards in the multi - energy flow coupling network, which are used to limit the interaction capacity, transmission efficiency, and safety thresholds of different energy sub - networks (such as power, heat, and natural gas shards), ensuring the feasibility and stability of cross - energy coordination. The target timing optimization results refer to the optimal solution set after globally coordinating variables such as energy scheduling and equipment output of the shards of the multi - energy flow coupling network based on the time dimension, aiming to maximize energy efficiency and minimize operating costs across different time scales.

[0027] In the embodiments of the present invention, a two - layer optimization model is adopted. Its upper - layer architecture aims at the optimal economy of the whole network, and sets the target energy interaction constraints between shards through mixed - integer programming. For example, the power transmitted from the power shard to the heat shard ≤ 50MW. Its lower - layer architecture is based on model predictive control to roll - optimize the internal scheduling plan of each shard to determine the target timing optimization results. For example, start the power - to - gas equipment for energy storage during the low - electricity - consumption period and limit cross - shard transmission during the peak period. The sources of its model parameters are multi - energy flow coupling data (such as the equipment efficiency matrix) and dynamic adjustment instructions (such as the topological relationship after shard reorganization).

[0028] Step 104: Based on the target energy interaction constraints, the target timing optimization results, and the dynamic adjustment instruction, generate a distributed dynamic shard configuration instruction for the multi - energy flow coupling network, and use the distributed dynamic shard configuration instruction to drive the data flow processing unit within the shard to perform distributed big - data real - time stream processing; In this step, the distributed dynamic shard configuration instruction is a set of executable control commands generated based on the real - time state, target constraints, and optimization results of the multi - energy flow coupling network, which is used to dynamically divide the network shard boundary and drive the data flow processing unit within the shard to perform parallel and decentralized real - time computing and resource scheduling. Its essence is to coordinate the dynamic adaptation rules between the physical operation and data - processing logic of the multi - energy flow coupling network, ensuring the efficient and stable distributed collaborative operation of the system under complex working conditions.

[0029] In the embodiment of the present invention, instructions are generated through a distributed task scheduling framework (such as Apache Flink). For example, based on the target energy interaction constraint and the timing optimization result, a reinforcement learning model is used to solve the optimal sharding configuration parameters; the instructions are pushed to the edge computing nodes of each shard through a message queue; the data stream processing unit adjusts parameters such as the parallelism and window calculation period according to the distributed dynamic sharding configuration instructions, and processes the sensor data stream within the shard in real time.

[0030] The following is a specific example. In an industrial park, the integrated power, heat, and natural gas networks suddenly encounter a situation where the power load surges, resulting in the voltage deviation rate exceeding the limit. Through the topological sequence diagram, it is shown that the voltage deviation rate of the power subnet A reaches 6%, triggering the dynamic sharding rule, and subnet A is split into two subnets, A1 and A2; according to the calculated dynamic threshold, it is found that the load rate of subnet A1 exceeds the limit, and a dynamic adjustment instruction is generated, that is, part of the load of A1 is migrated to the adjacent subnet B; through the hierarchical collaborative optimization architecture, the energy interaction constraint between subnet A1 and the adjacent subnet B is set to 30 MW, and it is planned to recharge energy through the power-to-gas equipment during the low valley period; the distributed dynamic sharding configuration instruction drives the Flink node of subnet A1 to increase the stream processing parallelism from 4 to 8, and monitors the load fluctuation after migration in real time.

[0031] In the embodiment of the present invention, through dynamic sharding and collaborative optimization, efficient resource allocation and real-time fault isolation of the multi-energy flow coupling network are realized; the sharding boundary and threshold are automatically adjusted based on real-time data, reducing manual intervention; the global economy and local stability are balanced through a hierarchical model, reducing the operation cost and realizing resource optimization; the distributed stream processing architecture supports millisecond-level response, and the fault recovery time can be effectively shortened.

[0032] The present invention provides a specific embodiment. Step 103: Based on the dynamic adjustment instruction and the multi-energy flow coupling data, a hierarchical collaborative optimization architecture is constructed to determine the target energy interaction constraint and the target timing optimization result between shards, which specifically includes the following steps: Step 301: Based on the betweenness centrality of the core layer nodes in the power network and the multi-energy flow coupling data, combined with the multi-energy flow coupling factor weight and the line capacity weight, a dynamic weight matrix is generated; In this step, the multi-energy flow coupling factor weight is a dynamic parameter that quantifies the degree of mutual influence among energy networks such as electricity, heat, and natural gas, and characterizes the energy conversion efficiency, stability contribution, and economic correlation of different energy types during the interaction process. The line capacity weight is a parameter that reflects the transmission capacity limitations of power lines, heat pipelines, or gas pipelines, and is used to constrain the maximum allowable value of energy interaction to prevent equipment overload or network congestion. The dynamic weight matrix is a multi-dimensional matrix that integrates the multi-energy flow coupling factor weight and the line capacity weight, and is used to describe the priority relationship between nodes and paths in the multi-energy flow network, supporting the iterative calculation of distributed optimization algorithms.

[0033] In the embodiment of the present invention, the coupling factors (such as the efficiency weight of the power-to-gas equipment is 0.85) and line capacities (such as the current-carrying capacity of the power line is 500 A) of the electricity, heat, and natural gas networks are normalized through the analytic hierarchy process to construct a dynamic weight matrix. Among them, the coupling factor weight is calculated based on the energy conversion efficiency and economic parameters. For example, the electricity-heat coupling weight is 0.7; the line capacity weight is generated in combination with the real-time derating factor (such as the cable capacity drops to 80% of the nominal value under high temperature); the dynamic update mechanism, for example, adjusts the weight every 5 minutes through the sliding window statistical method to ensure matching with the real-time load.

[0034] Step 302: Use the distributed consensus optimization algorithm to parallelly solve the output priorities of the photovoltaic and energy storage devices within the shard, generate the timing optimization instructions for the devices within the shard, and combine with the pre-regulation instructions of the combined heat and power generation to generate the verified power demand constraints at the shard boundary; In this step, the timing optimization instructions for the devices within the shard are the device output scheduling commands generated based on the distributed consensus algorithm, which are used to coordinate the operation timing of devices such as photovoltaic and energy storage within the shard to achieve the balance of local economy and stability. The pre-regulation instructions of the combined heat and power generation are the control commands sent to the combined heat and power generation units in advance, which are used to adjust their output curves to match the predicted heat and power demands and reduce the real-time scheduling pressure. The verified power demand constraints at the shard boundary are the power interaction limits between shards verified by the distributed consensus algorithm, ensuring that the cross-shard energy transmission meets the safety and economic requirements.

[0035] In the embodiment of the present invention, the distributed system consensus algorithm is used to achieve the collaborative scheduling of the devices within the shard. For example, a proposal is initiated by the Leader node (such as the photovoltaic shard controller), and the charge and discharge priorities of the energy storage devices are determined by the voting of the majority of Follower nodes; combined with the pre-regulation instructions of the combined heat and power generation units (such as the minimum heat-electricity ratio of 0.6), the output plan of the devices within the shard is generated; verified through the distributed system consensus algorithm to ensure that the verified power demand constraints at the shard boundary (such as the maximum transmission power of 50 MW) are accepted by the majority of nodes to avoid the risk of overload.

[0036] Step 303: Based on the verified shard boundary power demand constraints and the dynamic weight matrix, co-optimize the cross-shard energy interaction rules to generate inter-shard energy interaction constraints and cross-shard timing coordination rules; In this step, the cross-shard timing coordination rule is a global strategy for coordinating the scheduling timings of different shards, ensuring seamless connection of cross-shard energy interaction in the time dimension and avoiding timing conflicts or resource competition.

[0037] In the embodiment of the present invention, based on the dynamic weight matrix and the verified shard boundary power demand constraints, a mixed-integer linear programming model is used to optimize cross-shard interaction to generate inter-shard energy interaction constraints and cross-shard timing coordination rules. For example, the power transmitted from the power shard to the heat shard ≤ 30MW, and the state replication mechanism of the Zookeeper atomic broadcast algorithm is used to ensure that the constraint parameters are synchronized by the leader nodes of each shard; it is defined that the priority of the power shard during peak hours is higher than that of the heat shard, and the time window is coordinated through the heartbeat synchronization mechanism.

[0038] Step 304: Based on the inter-shard energy interaction constraints and the cross-shard timing coordination rules, when the difference between the betweenness centrality of the core layer nodes and the preset betweenness value exceeds the limit threshold, reconstruct the topological structure of the shards, and combine the dynamic weight matrix and the pre-regulation instruction to generate the energy interaction constraints and timing optimization rules of the reorganized shards; In the embodiment of the present invention, based on the inter-shard energy interaction constraints and the cross-shard timing coordination rules, the betweenness centrality of the core layer nodes is calculated using the graph theory algorithm. If the difference between the betweenness centrality of a certain core layer node and the preset betweenness value exceeds the limit threshold (such as 0.15), it is determined as a key hub; the Gossip protocol is used to broadcast the shard splitting instruction (such as splitting the original shard A into A1 and A2), and the new shard boundary is confirmed by a majority of nodes; combined with the dynamic weight matrix and the pre-regulation instruction, the energy interaction constraints and timing optimization rules of the reorganized shards are optimized and generated through a reinforcement learning model.

[0039] Step 305: When abnormal conditions occur in the node voltage deviation coefficient or the heat return water temperature within the shard, freeze the dynamic weight matrix using the energy interaction constraints of the reorganized shards and the timing optimization rules, and reallocate the transmission margin of adjacent shards to generate the energy interaction constraints in the safe mode and the abnormal timing optimization results. When the abnormality is lifted, optimize the dynamic weight matrix based on the energy interaction constraints in the safe mode to generate the target energy interaction constraints and the target timing optimization results; In this step, the energy interaction constraints in the safe mode are temporary interaction rules triggered when an abnormality is detected, restricting high-risk transmission paths; the abnormal timing optimization result is the corrected scheduling plan for the equipment output during the abnormal period.

[0040] In the embodiments of the present invention, when the node voltage deviation rate > 5% or the return water temperature deviation rate > 3%, dynamic weight freezing is triggered and matrix update is suspended; the adjacent shard Leader node is selected through the Raft election algorithm to take over the abnormal load (for example, shard B takes over 50% of the transmission tasks of shard A); after the abnormality is lifted, based on the data running in the safe mode, the dynamic weight matrix is recalculated through the gradient descent algorithm, and gradually transitions to the normal mode to generate the target energy interaction constraint and the target timing optimization result.

[0041] The following is a specific example. The betweenness of the core node of the power shard in an industrial park exceeds the limit, and at the same time, the return water temperature of the heat supply is abnormal. The power-heat coupling weight is shown as 0.7 and the line capacity weight is 0.8 through the dynamic weight matrix; the Raft election algorithm is used to coordinate the priority power supply of the photovoltaic, and the power constraint at the shard boundary is generated (transmission ≤ 40 MW); the Zookeeper atomic broadcast algorithm is used to synchronize the cross-shard rules, and the power output is restricted during peak hours; when the betweenness of the core node reaches 0.18, the Gossip protocol splits the shard and generates new constraints (transmission ≤ 25 MW); when the abnormal return water temperature triggers the safe mode, the Raft election algorithm selects shard C to take over the load, and the weight is optimized by the gradient descent after the abnormality is lifted.

[0042] In the embodiments of the present invention, through the dynamic weight matrix, cross-energy dynamic priority matching of the power, heat, and natural gas networks is realized; the distributed consistency optimization algorithm is adopted to parallelly solve the output priorities of the devices within the shard, reducing the impact of the photovoltaic output fluctuation on the power grid; the cross-shard timing coordination rule realizes the balance between global economy and local stability; the topological structure of the shard is reconstructed, improving the fault tolerance of the system to key node failures; when voltage or temperature abnormalities are detected, the dynamic weight is frozen and the transmission margin is reallocated, optimizing the safe mode constraints, realizing millisecond-level fault isolation and recovery; through the cross-shard timing coordination rule and the constraints after reorganization, the cross-shard transmission cost is reduced, and at the same time, the risk of equipment overload is avoided.

[0043] The present invention provides a specific embodiment. In step 302, the distributed consistency optimization algorithm is adopted to parallelly solve the output priorities of the photovoltaic and energy storage devices within the shard, generate the timing optimization instructions for the devices within the shard, and combine the pre-regulation instructions of the combined heat and power to generate the verified power demand constraint at the shard boundary, which specifically includes the following steps: Step 311: Based on the irradiance fluctuation coefficient of the photovoltaic device, the discrete gradient of the state of charge of the energy storage device, and the device topology connectivity, construct the dynamic priority weight matrix of the devices within the shard; In this step, the irradiance fluctuation coefficient is a parameter that quantifies the impact of random changes in light intensity on the output power of a photovoltaic device, reflecting the degree to which the irradiance deviates from the average value per unit time. It is usually the ratio of the standard deviation to the mean of the actual irradiance. The dynamic parameter that describes the differences in the state of charge (SOC) of multiple energy storage devices within a slice, reflecting the balance of the energy storage cluster, is usually the ratio of the range (the maximum SOC and the minimum SOC) of the SOC values of each energy storage device to the average SOC. The device topology connectivity is a parameter that characterizes the physical or logical connection relationship between devices within a slice, including the number of power line connections, communication link strength, etc.

[0044] In the embodiment of the present invention, the irradiance fluctuation coefficient is taken as the ratio of the standard deviation to the mean of the irradiance within a 15-minute sliding window of the photovoltaic device (for example, if the standard deviation is 50 W / m² and the mean is 800 W / m², then the coefficient is 0.0625); the discrete gradient of the state of charge is taken as the ratio of the range of the SOC of the energy storage within the slice to the mean (for example, if the range is 0.3 and the mean is 0.7, then the gradient is 0.428); for the device topology connectivity, the node betweenness centrality is used (for example, a certain power node is connected to 5 lines and the betweenness is 0.12); weights are assigned through the analytic hierarchy process (for example, the irradiance fluctuation coefficient is 0.4, the discrete gradient of the state of charge is 0.3, and the device topology connectivity is 0.3), generating a dynamic priority weight matrix, which is updated every 5 minutes.

[0045] Step 312: Use the alternating direction multiplier method to perform distributed consensus optimization on the dynamic priority weight matrix to generate a timing optimization instruction; In the embodiment of the present invention, the optimization of the photovoltaic and energy storage output is decomposed into multiple sub-problems, corresponding to different devices respectively. The alternating optimization process includes the update of the original variables and the update of the multipliers; for the update of the original variables, fixed multipliers are used to solve the local optimal output of each device (such as 80% of the upper limit of the photovoltaic output); for the update of the multipliers, the multiplier values are adjusted according to the constraint deviation (such as the power balance error), and after iteration to convergence, a timing optimization instruction (such as the energy storage charge and discharge plan) is output.

[0046] Step 313: Use the ratio of the range of the return water temperature within a sliding time window to a set threshold as the reference sensitivity. Combine the reference sensitivity and historical time delay data to generate a time delay compensation coefficient, and perform a convolution operation on the reference sensitivity and the time delay compensation coefficient to generate the intensity parameter of the pre-regulation instruction; In this step, the reference sensitivity refers to the ratio of the range of the return water temperature within a sliding time window to a set threshold, which is used to reflect the sensitivity of the temperature fluctuation of the thermal network. The historical time delay data refers to the time series data recording the delay between the system input signal and the response output, which is used to model the time delay effect. The time delay compensation coefficient refers to the correction factor generated based on the historical time delay data and the reference sensitivity, which is used to offset the delay effect of the control instruction. The intensity parameter of the pre-regulation instruction refers to the dynamic parameter that quantifies the action intensity of the pre-regulation instruction.

[0047] In the embodiment of the present invention, the reference sensitivity is taken as the ratio of the range of the return water temperature within a 30 - minute sliding window to a threshold value (such as 5°C) (for example, when the range is 8°C, the sensitivity is 1.6); based on the historical time - lag data (such as an average delay of 0.8 seconds) and the sensitivity for convolution, a compensation coefficient is generated (1.6×0.8 = 1.28); the convolution result is mapped to the range of 0 to 1 (for example, 1.28 is mapped to an intensity parameter of 0.85), and intensity parameter normalization is performed to be used for adjusting the output of the combined heat and power unit.

[0048] Step 314: Use a bidirectional long - short - term memory network to perform cross - time - step correlation analysis on the intensity parameters of the timing optimization instruction and the pre - adjustment instruction to obtain a correlation analysis result. Based on the load factor, the dynamic priority weight matrix, and the correlation analysis result, calculate the preliminary boundary power demand constraint; In this step, the preliminary boundary power demand constraint refers to the initial power transmission limit generated through correlation analysis, which is used to guide the energy interaction between slices.

[0049] In the embodiment of the present invention, input a timing instruction (such as the energy storage charge - discharge curve) and an intensity parameter (such as 0.85) into the bidirectional long - short - term memory network. Capture the timing dependence through forward and backward LSTM, and output a correlation coefficient (such as 0.72); combine the load factor (such as an 80% load rate), the dynamic weight matrix (priority 0.7), and the correlation coefficient to calculate the preliminary boundary constraint (such as a maximum transmission power of 50 MW).

[0050] Step 315: Construct a constraint validity verification mechanism, define a feasibility index, and use the constraint validity verification mechanism to verify the feasibility index. When the feasibility index is lower than a preset threshold, reconstruct the dynamic priority weight matrix to generate a verified boundary power demand constraint; In this step, the feasibility index refers to a quantitative parameter for evaluating the effectiveness of the preliminary boundary constraint, including dimensions such as economy (such as net present value), technology (such as sliding correlation coefficient, node voltage deviation rate, and return water temperature deviation rate), etc.

[0051] In the embodiment of the present invention, define a feasibility index, that is, set thresholds such as net present value ≥ 0 and voltage deviation rate ≤ 3%. Use the constraint validity verification mechanism to verify the feasibility index. If the feasibility index does not meet the standard (such as a net present value of - 5%), trigger the reconstruction of the dynamic weight matrix, re - allocate the device priorities (such as reducing the weight of highly volatile photovoltaic), and generate a verified boundary power demand constraint (such as transmission power ≤ 40 MW).

[0052] Taking a specific example, in a certain industrial park, the power of the distributed photovoltaic system fluctuates greatly, and the temperature of the return water of the heat supply is abnormal. By calculating that the irradiance fluctuation coefficient is 0.08 and the discrete gradient of the state of charge is 0.5, a dynamic weight matrix is constructed. Through the alternating direction multiplier method, the photovoltaic output is coordinated to drop to 70%, and the energy storage is preferentially charged; combined with the range of the return water temperature of 10°C, an intensity parameter of 0.9 is generated to adjust the combined heat and power output to 90%; the analysis of the bidirectional long short-term memory network shows that the midday load is strongly correlated with the temperature, and the verified boundary power demand constraint (transmission ≤ 45 MW) is generated; after verification by the constraint effectiveness verification mechanism, it is found that the voltage deviation rate is 4%. After reconstructing the dynamic weight matrix, the verified boundary power demand constraint is tightened to 35 MW.

[0053] In the embodiment of the present invention, by constructing a dynamic priority weight matrix, the dynamic priority matching of devices within a slice can be realized according to specific scenarios; the alternating direction multiplier method realizes distributed consensus optimization and effective cooperative scheduling; the bidirectional long short-term memory network realizes cross-period correlation analysis and boundary prediction, reduces the analysis error, and improves the accuracy of boundary constraints; through the verification of constraint effectiveness, the success rate of fault recovery is improved.

[0054] The present invention provides a specific embodiment. Step 101: Based on the power network, the heat network, the natural gas network, and the real-time operation state parameters, generate a topological sequence diagram. Based on the topological sequence diagram and the real-time operation state parameters, generate a set of dynamic slicing rules for the multi-energy flow coupling network. The set of dynamic slicing rules includes the topological structure of the power network, the multi-energy flow coupling data, and the dynamic slicing trigger conditions, and specifically includes the following steps: Step 111: Obtain the topological connection relationship of the power network, the hydraulic balance parameters of the heat network, and the dynamic relationship between the pressure and flow of the natural gas network to construct the physical connection relationship of the multi-energy flow coupling network. Combine the real-time operation state parameters and the physical connection relationship to generate a topological sequence diagram; In this step, the physical connection relationship of the multi-energy flow coupling network refers to the integrated representation of the topological connection and dynamic interaction relationship between the power, heat, and natural gas networks. The real-time operation state parameters include the voltage deviation rate of the power network, the temperature gradient of the heat network, and the pressure fluctuation of the natural gas network.

[0055] In the embodiments of the present invention, for the extraction of topological relationships, for the power grid: an adjacency matrix is constructed based on node betweenness centrality (such as substations, load centers) and line impedance. For example, if a substation is connected to 5 transmission lines, its node degree is 5; for the thermal network: the relationship between pipeline flow rate and pressure drop is quantified through a hydraulic balance equation (such as the flow rate mismatch formula). For example, the ratio of the actual flow rate to the designed flow rate (hydraulic mismatch) of a pipeline is 0.85; for the natural gas network: the dynamic pressure-flow model in the unified energy path theory is used to describe the dynamic characteristics of pipelines; the physical connection relationships of these three types of networks are encoded into a unified multi-layer adjacency matrix. For example, if power node i is coupled with thermal node j through an electric boiler, the matrix element value is 1, otherwise it is 0. Finally, a topological sequence diagram including node attributes (such as voltage level, temperature threshold) and edge attributes (such as line capacity, pressure drop coefficient) is generated.

[0056] Step 112: Based on real-time operating state parameters, perform time series prediction on the node voltage deviation rate of the power grid, the return water temperature of the nodes in the thermal network, and the node pressure fluctuation gradient of the natural gas network to obtain multiple node state prediction values; In the embodiments of the present invention, sliding window normalization is performed on real-time operating parameters (such as voltage, temperature, pressure). For example, the mean and variance are calculated with a 30-minute window; for the power grid, a long short-term memory network is used to predict the node voltage deviation rate, inputting the historical voltage sequence and load rate, and outputting the deviation rate for the next 15 minutes (such as ±2%); for the thermal network, the historical temperature and outdoor meteorological parameters are input into a bidirectional long short-term memory network for predicting the return water temperature; for the natural gas network, a dynamic state estimation algorithm is used in combination with pressure gradient fluctuation data to predict the node pressure change (such as 0.2 MPa / min).

[0057] Step 113: Use elastic net regression to train the multiple node state prediction values, determine the weights of the multi-energy flow coupling factors, and based on the weights of the multi-energy flow coupling factors, perform a linear combination of the multiple node state prediction values to calculate the multi-energy flow coupling strength. Based on the topological sequence diagram and the multi-energy flow coupling strength, construct a spatial distribution matrix representing the multi-energy flow coupling relationship, and combine the multi-energy flow coupling strength and real-time state operating parameters to generate multi-energy flow coupling data; In this step, the multi-energy flow coupling strength is a parameter that quantifies the degree of dynamic influence between different energy networks and reflects the strength of the electro-thermal-gas interaction. The spatial distribution matrix is a matrix that characterizes the distribution characteristics of the multi-energy flow coupling relationship in physical space.

[0058] In the embodiments of the present invention, elastic net regression is used to train the predicted values of the multiple node states to determine the weights of the multi-energy flow coupling factors. For example, taking the power voltage deviation as the dependent variable and the thermal temperature and natural gas pressure gradient as the independent variables, key coupling factors are screened through L1 / L2 regularization (such as the thermal temperature weight is 0.3 and the natural gas pressure weight is 0.1); the predicted values of each node are weighted and summed according to the weights of the multi-energy flow coupling factors; the element values of the adjacency matrix of the topological sequence graph are replaced with the coupling strengths of the corresponding nodes (such as if the coupling strength between power node i and thermal node j is 0.8, then the matrix element is 0.8) to construct a spatial distribution matrix; combining the multi-energy flow coupling strength and the real-time state operation parameters to obtain multi-energy flow coupling data.

[0059] Step 114: Using the spectral clustering algorithm and cosine similarity metric, perform dynamic slicing processing on the spatial distribution matrix to generate an initial slice set; In the embodiments of the present invention, the cosine similarity metric is used to calculate the cosine similarity between nodes. For example, the similarity between node A and node B = (A·B) / (||A||·||B||). If the similarity > 0.7, it is regarded as a strong association; based on this, the spectral clustering algorithm is used for slicing, that is, the Laplacian matrix of the spatial distribution matrix is normalized, and the first k eigenvectors are extracted (such as k = 5); the K-means algorithm is used to cluster the eigenvectors, and the network is divided into several initial slices (such as a certain city is divided into 3 slices) to generate an initial slice set.

[0060] Step 115: Using an extreme gradient boosting classifier trained based on historical load data, generate a set of dynamic slicing trigger conditions, and combine the multi-energy flow coupling data, the initial slice set, and the set of dynamic slicing trigger conditions to generate a set of dynamic slicing rules for the multi-energy flow coupling network; In the embodiments of the present invention, using features such as historical slice load rate and coupling strength volatility, the extreme gradient boosting classifier is trained to output slice recombination trigger conditions (such as load rate > 85% and coupling strength change > 20%) to generate a set of dynamic slicing trigger conditions, and combining the multi-energy flow coupling data and the initial slice set to generate dynamic slicing rules (such as slice A and slice B are merged during peak hours to balance the load).

[0061] The following is a specific example. An industrial park includes power, heat, and natural gas networks and needs to achieve cross - energy collaborative optimization. By constructing a topological sequence diagram, the power nodes (substations) and heat nodes (heat exchange stations) are coupled through electric boilers, and the natural gas nodes (pressure regulating stations) and heat nodes are connected through gas boilers; predict the voltage deviation rate of power nodes within ±3% and the fluctuation of heat return water temperature within ±2°C in the next hour, and the natural gas pressure gradient of 0.3 MPa / min; calculate the multi - energy flow coupling intensity, find that the electric - heat coupling is dominant (weight 0.6), construct a spatial distribution matrix and mark the key coupling paths; use the spectral clustering algorithm to divide the network into 3 sub - networks. Sub - network 1 is centered around the substation, sub - network 2 is centered around the heat exchange station, and sub - network 3 is centered around the pressure regulating station; use the extreme gradient boosting classifier to trigger the merger of sub - network 1 and sub - network 2 during the peak load at noon, and the transmission margin is increased by 15%.

[0062] In the embodiment of the present invention, by constructing a topological sequence diagram to integrate the physical connection relationships of the power, heat, and gas networks, and combining elastic net regression to quantify the coupling strength weights, the multi - energy flow interaction efficiency is improved, the energy transmission path is optimized, and redundant energy exchange is reduced; by screening key coupling factors through elastic net regression and combining with a set of dynamic sub - network division rules, the energy utilization efficiency is improved; by combining the spectral clustering algorithm with the spatial weight matrix, the problem of poor adaptability of traditional sub - network division methods to non - linear coupling relationships is solved.

[0063] The present invention provides a specific embodiment. Step 111: Obtain the topological connection relationship of the power network, the hydraulic balance parameters of the heat network, and the dynamic relationships of pressure and flow of the natural gas network to construct the physical connection relationship of the multi - energy flow coupling network. Combine the real - time operating state parameters and the physical connection relationship to generate a topological sequence diagram, which specifically includes the following steps: Step 121: Collect the data of the connection relationships of power network equipment, analyze the physical connection paths from distribution transformers to user nodes to generate an adjacency matrix of the power network. Combine the breaker state data and the voltage deviation rate of nodes to optimize the adjacency matrix of the power network and generate a dynamic topological feature matrix containing voltage deviation attributes; In this step, the physical connection path refers to the actual electrical connection link from the distribution transformer to the user node in the power network, including the physical topological relationships between devices such as cables, busbars, and switches. The adjacency matrix of the power network represents a weighted square matrix of the connection relationships between nodes in the power network, and its element values reflect the physical connection status and electrical parameters. The dynamic topological feature matrix refers to a dynamic weight matrix formed by superimposing the real - time state parameter of the voltage deviation rate of nodes on the basis of the adjacency matrix of the power network.

[0064] In the embodiments of the present invention, the cable connection relationship from the distribution transformer to the user node is obtained through the smart meter and the breaker status data acquisition device; the NetworkX library is used to construct the power network adjacency matrix, and the element values are determined by the breaker status (0 / 1) and the electrical distance; combined with the node voltage monitoring data, the power network adjacency matrix is extended with the node voltage deviation rate as the weight to generate the dynamic topology feature matrix. The optimization logic can be that when the node voltage deviation rate exceeds 5%, it is marked as an abnormal connection and the topology reconstruction is triggered, and the dynamic topology feature matrix is generated.

[0065] Step 122: According to the supply and return water temperature data and the flow time series data, predict the return water temperature deviation rate, generate the standardized temperature deviation rate sequence, calculate the temperature deviation rate difference between adjacent nodes, and based on the temperature deviation rate difference, mark the hydraulic balance state of the heat transfer branches in the heat network to generate the heat network balance coefficient matrix; In this step, the hydraulic balance state marking refers to classifying and identifying the hydraulic balance degree of the heat transfer branches to reflect whether the flow distribution meets the design requirements. The heat network balance coefficient matrix represents the coefficient matrix of the hydraulic balance degree of the heat transfer branches and is used to quantify the regulation priority.

[0066] In the embodiments of the present invention, the long short-term memory network is used to predict the supply and return water temperature time series data to generate the temperature deviation rate sequence; after standardizing the temperature deviation rate, calculate the temperature difference between adjacent nodes, and a temperature difference exceeding 2°C is marked as an unbalanced branch; based on the temperature difference and the flow data, optimize the branch resistance coefficient through the linear programming algorithm to construct the heat network balance coefficient matrix.

[0067] Step 123: Based on the pressure monitoring data of the natural gas network, generate the node pressure distribution data, and perform joint prediction processing on the pipeline pressure decay rate and the flow fluctuation gradient by combining the Weibull distribution model and the long short-term memory network to generate the natural gas network dynamic gradient matrix; In this step, the natural gas network dynamic gradient matrix represents the spatio-temporal distribution matrix of the pipeline pressure decay rate and the flow fluctuation gradient and is used to predict the network dynamic behavior.

[0068] In the embodiments of the present invention, based on the pressure monitoring data of the natural gas network collected by the Internet of Things sensors, the node pressure data is formed, and based on this, the Gaussian kernel density distribution map is generated; the Weibull distribution model is used to fit the pipeline aging curve to predict the pipeline pressure decay rate; the long short-term memory network is used to predict the flow fluctuation gradient, and the natural gas network dynamic gradient matrix is generated by combining the pipeline pressure decay rate. The specific implementation process may be that the Weibull distribution parameters are fitted by the maximum likelihood estimation method, and the input of the long short-term memory network includes historical pressure, flow, and temperature, and the output is the gradient sequence for the next 30 minutes.

[0069] Step 124: Generate a multi-energy flow physical connection relationship matrix based on the dynamic topology feature matrix, the thermal network balance coefficient matrix, and the natural gas network dynamic gradient matrix, and dynamically optimize the coupling weights of the multi-energy flow physical connection relationship matrix based on the real-time load fluctuation rate to generate a topology sequence diagram; In this step, the multi-energy flow physical connection relationship matrix refers to the coupling relationship matrix integrating the power, heat, and gas networks, which characterizes the physical constraints and energy transfer paths of multi-energy flow interaction.

[0070] In the embodiment of the present invention, the dynamic topology feature matrix of the power network, the thermal network balance coefficient matrix of the thermal network, and the natural gas network dynamic gradient matrix of natural gas are mapped into a three-dimensional tensor according to nodes, so as to generate a physical connection relationship matrix; based on the real-time load fluctuation rate, the elastic net regression is used to dynamically adjust the matrix weights, set its objective function to minimize the multi-energy flow transmission loss, and set the elastic net regularization parameter to screen key coupling factors, and finally generate a topology sequence diagram.

[0071] Taking the multi-energy network regulation in the industrial park as an example, 3 abnormal connections are identified and reconstructed through the dynamic topology feature matrix; 2 unbalanced branches are marked by the thermal network balance coefficient matrix (such as the temperature deviation rate difference is 3.1 °C), and the valve opening is adjusted to 65%; the natural gas network dynamic gradient matrix predicts that the pressure decay rate is 0.02 MPa / h, triggering the compressor frequency modulation. Combining the above results, the elastic net regression is used to determine the coupling weights of the power network: thermal network: natural gas network as 0.6:0.3:0.1, so as to construct a topology sequence diagram.

[0072] In the embodiment of the present invention, by constructing a multi-energy flow physical connection relationship matrix, the limitations of traditional single-network analysis are solved, the energy transmission path is optimized, the electro-thermal coupling efficiency is improved, and the loss of the natural gas network is reduced at the same time; combined with the elastic net regression to dynamically adjust the coupling weights, the topology reorganization speed is improved when the load changes suddenly, and the abnormal recovery time is shortened; parameters such as node voltage deviation and temperature gradient are introduced, so that the present invention has a minute-level response ability; combined with the prediction results of the long short-term memory network and the hydraulic balance mark, the stability under abnormal conditions is improved.

[0073] The present invention provides a specific embodiment, step 114, using the spectral clustering algorithm and cosine similarity measure to perform dynamic slicing processing on the spatial distribution matrix to generate an initial slice set, which specifically includes the following steps: Step 131: Based on the spatial distribution matrix, real-time status operation parameters, and time decay factor, construct a node similarity matrix characterizing the coupling strength between nodes, and the time decay factor is dynamically adjusted according to the real-time load fluctuation rate; In this step, the time decay factor is a dynamic parameter used to quantify the decay degree of the coupling strength between nodes with the change of time or the load fluctuation rate, reflecting the timeliness of the coupling relationship. In the embodiments of the present invention, it is dynamically adjusted according to the real-time load fluctuation rate. The node similarity matrix is a symmetric matrix characterizing the coupling strength between nodes, and its element values are jointly determined by the spatial distribution matrix, real-time state parameters, and time decay factor.

[0074] In the embodiments of the present invention, parameters such as the spatial distribution matrix, real-time voltage deviation rate, and thermal return water temperature deviation are used to calculate the node state correlation through cosine similarity, and combined with the time decay factor, a node similarity matrix characterizing the coupling strength between nodes is constructed; among them, the time decay factor is calculated by combining the load fluctuation rate and historical load data.

[0075] Step 132: Generate a normalized coupling relationship matrix based on the node similarity matrix and node degree distribution characteristics; In this step, the node degree distribution characteristics are used to describe the statistical distribution characteristics of the number of node connections (degrees) in the network, reflecting the global structure of the multi-energy flow network topology. The normalized coupling relationship matrix refers to a matrix obtained by standardizing the node similarity matrix to eliminate the influence of node degree differences on the multi-energy flow coupling strength.

[0076] In the embodiments of the present invention, based on the node similarity matrix, the number of connections (degrees) of each node is calculated to generate a diagonal matrix, and the diagonal elements of which are the sum of the degrees of each node; each element of the original matrix is divided by the product of the square roots of the corresponding node degrees to eliminate the excessive influence of high-connected nodes on the coupling relationship; through statistical tests, it is verified whether the node degrees conform to the preset distribution (such as scale-free characteristics), and if they deviate, the normalization coefficient is adjusted to ensure the rationality of the network structure.

[0077] Step 133: Use the manifold learning algorithm to perform dimensionality reduction processing on the normalized coupling relationship matrix to generate a dimensionality-reduced feature matrix; In the embodiments of the present invention, the manifold learning algorithm (such as the isometric mapping algorithm) is used to calculate the low-dimensional embedding based on the node similarity matrix, and the geodesic distance between nodes is retained; the normalized coupling relationship matrix is subjected to eigenvalue decomposition, and the eigenvectors corresponding to the first k largest eigenvalues are extracted to form a low-dimensional feature matrix; it is projected into a two-dimensional space through principal component analysis to observe whether the clustering structure is separated (such as the visualization of smart city energy slices). After verifying that the clustering structure has been separated, a dimensionality-reduced feature matrix is generated.

[0078] Step 134: Based on the dimensionality-reduced feature matrix, combine the improved clustering algorithm integrating the balanced regularization term and the alternating direction multiplier method to optimize the standard deviation of the multi-energy flow coupling strength within the slice and the cross-slice channel occupancy ratio constraint, and generate an initial slice set; In this step, the standard deviation of the multi-energy flow coupling strength is used to measure the degree of dispersion of the multi-energy flow coupling strength within the shard, and is used to optimize the stability of energy interaction within the shard. The cross-shard channel ratio constraint is used to limit the ratio of the number of connection channels between different shards to prevent the over-concentration or dispersion of the energy transmission path.

[0079] In the embodiment of the present invention, the improved clustering algorithm integrating the balanced regularization term integrates the L2 regularization term into the objective function. By penalizing the dispersion of the standard deviation of the coupling strength within the shard, it suppresses the influence of abnormal shards on the overall structure, and at the same time constrains that the cross-shard channel ratio does not exceed the threshold. The L2 regularization balances the coupling strength of each node within the shard by shrinking the weights rather than making them zero, thus improving the stability of the model. The alternating direction method of multipliers decomposes the shard optimization problem into two sub-problems: shard division and multiplier variable update. Shard division: Based on the current multiplier variable, minimize the weighted sum of the standard deviation within the shard and the cross-shard channel ratio. Multiplier update: Dynamically adjust the Lagrange multiplier according to the degree of constraint violation to force the shard division to satisfy the global constraint. Distributed optimization is achieved through alternating iteration, and the convergence speed is faster than that of traditional centralized methods.

[0080] In the embodiment of the present invention, the time decay factor is introduced to adjust the coupling weight in real time according to the load fluctuation, adapting to the time-varying characteristics of the energy network. The node similarity matrix integrates the spatial topology and real-time parameters to solve the problem that traditional static models cannot capture dynamic interactions. Through the standard deviation constraint and channel ratio limit, a robust structure with high aggregation inside and low coupling outside the energy shard is achieved, improving the multi-energy flow collaborative transmission efficiency.

[0081] The present invention provides a specific embodiment. Step 102: Based on the set of dynamic sharding rules, calculate a dynamic threshold. When any load factor exceeds the dynamic threshold, generate a dynamic adjustment instruction for the shard boundary. The load factors include the node voltage deviation rate, the node pressure fluctuation gradient, and the return water temperature deviation rate, and specifically include the following steps: Step 201: Based on the set of dynamic sharding rules, combine the dynamic mean, dynamic standard deviation, and elastic coefficient of the load factor within the sliding time window to calculate the dynamic threshold; In this step, the dynamic mean refers to the moving average of the load factor (such as CPU usage rate, energy demand, etc.) within the sliding time window, reflecting the short-term trend of the load. The dynamic standard deviation is a volatility index of the load factor within the sliding time window, used to quantify the degree of deviation of the load from the mean. The elastic coefficient is a weight parameter for adjusting the sensitivity of the dynamic threshold, and is dynamically adjusted according to the load fluctuation characteristics (such as dynamic standard deviation, historical extreme values).

[0082] In the embodiments of the present invention, the exponential weighted moving average method is used to calculate the dynamic mean value to eliminate the interference of instantaneous fluctuations; the dynamic standard deviation is used to quantify the volatility of the load deviating from the mean value; the elasticity coefficient dynamically adjusts the sensitivity according to the historical extreme value of the load and the dynamic standard deviation (for example, when k = 1.5, the threshold is relaxed to tolerate fluctuations). Combining the above, the dynamic threshold is calculated to realize the adaptive change of the threshold with the system state.

[0083] Step 202: Based on the multi-energy flow coupling intensity, calculate the standard deviation of the multi-energy flow coupling intensity corresponding to each shard, and select the target shard whose multi-energy flow coupling intensity standard deviation is higher than the preset standard value; In the embodiments of the present invention, the standard deviation of the multi-energy flow coupling intensity of each shard is calculated through the multi-energy flow coupling intensity, and the target shard with a multi-energy flow coupling intensity higher than the preset standard value (such as 0.1) is screened. This process combines the hash sharding algorithm (such as consistent hashing) to locate the highly discrete shards, ensuring that the coupling imbalance area is preferentially processed during shard rebalancing.

[0084] Step 203: According to the dynamic threshold and the elasticity coefficient, calculate the load overflow ratio to generate a shard shrinkage coefficient or expansion coefficient, and update the energy interaction weight matrix of adjacent shards in combination with the graph neural network to obtain the updated energy interaction weight matrix; In this step, the energy interaction weight matrix is a weighted graph describing the energy transmission ability between shards.

[0085] In the embodiments of the present invention, based on the dynamic threshold and the elasticity coefficient, the deviation ratio of the current load relative to the dynamic threshold is calculated to obtain the load overflow ratio. If the load overflow ratio > 0, an expansion coefficient is generated to expand the shard capacity proportionally; if the load overflow ratio < 0, a shrinkage coefficient is generated to reduce redundant resources; the edge weights are calculated using the graph neural network, and by introducing a pruning threshold, weak similar edges are removed, and the interaction weights of low-load shards are preferentially enhanced to obtain the updated energy interaction weight matrix, optimizing the energy transmission efficiency.

[0086] Step 204: According to the updated energy interaction weight matrix, set the transmission power gradient and the lowest occupancy ratio threshold of the cross-shard energy interaction channel during the shard boundary adjustment process; In this step, the transmission power gradient refers to the maximum allowable power change rate of the cross-shard channel, which is used to prevent network oscillations caused by sudden changes in transmission power. The lowest occupancy ratio threshold refers to the minimum occupancy ratio requirement of the cross-shard channel in the total transmission capacity, avoiding energy islanding between shards and ensuring system redundancy.

[0087] In the embodiments of the present invention, the transmission power gradient is set to ±10% / second (to prevent power mutations), and the lowest occupancy ratio threshold ≥ 15% (for example, the core heat source shard in the industrial park needs to retain 20% channel redundancy).

[0088] Step 205: Based on a preset conflict arbitration rule, when it is detected that any load factor exceeds the dynamic threshold, combine the shard shrinkage coefficient or expansion coefficient, the updated energy interaction weight matrix, the transmission power gradient, and the minimum ratio threshold to generate a dynamic adjustment instruction for the shard boundary; In this step, the preset conflict arbitration rule is a decision-making logic for resolving resource competition and conflicts during the shard adjustment process.

[0089] In an embodiment of the present invention, based on a preset conflict arbitration rule, when it is detected that any load factor exceeds the threshold, the alternating direction method of multipliers is used to decompose and optimize the problem, including a shard division sub-problem and a multiplier update sub-problem. Specifically, it includes minimizing the weighted sum of the within-shard standard deviation and the cross-shard channel ratio, adjusting the Lagrange multiplier according to the degree of constraint violation, and forcing the satisfaction of global constraints. Finally, a new shard scheme is generated by an extreme gradient boosting classifier within 10 seconds, and the weight adjustment is coordinated by a game theory model, and finally a dynamic adjustment instruction for the shard boundary is generated.

[0090] In an embodiment of the present invention, the shard expansion or contraction is triggered by the load overflow ratio, and the edge weights of the graph neural network are dynamically updated to achieve load balancing; the pruning threshold filters out noise edges (such as weakly connected lines in the power grid), improving the resource allocation priority of high-load shards; by combining the shard shrinkage coefficient or expansion coefficient, the stability and efficiency of shard adjustment are ensured. Through the synergistic effect of the dynamic threshold, the graph neural network, and the pruning strategy, the load response speed and resource utilization rate of the multi-energy flow network are improved. The preset conflict arbitration rule effectively avoids resource competition failures and improves the success rate of abnormal rollback.

[0091] Figure 2 The following is a schematic structural diagram of a distributed big data real-time stream processing method and system provided by an embodiment of the present invention, as Figure 2 shown, the system includes: A generation module 21, configured to generate a topological sequence graph based on a power network, a thermal network, a natural gas network, and real-time operating state parameters, and generate a set of dynamic shard rules for a multi-energy flow coupling network based on the topological sequence graph and real-time operating state parameters. The set of dynamic shard rules includes a power network topological structure, multi-energy flow coupling data, and dynamic shard triggering conditions; A calculation module 22, configured to calculate a dynamic threshold based on the set of dynamic shard rules, and generate a dynamic adjustment instruction for the shard boundary when any load factor exceeds the dynamic threshold. The load factors include a node voltage deviation rate, a node pressure fluctuation gradient, and a return water temperature deviation rate; A determination module 23, configured to construct a hierarchical collaborative optimization architecture based on the dynamic adjustment instruction and the multi-energy flow coupling data to determine the target energy interaction constraint and the target timing optimization result between shards; A processing module 24 is configured to generate distributed dynamic sharding configuration instructions for a multi-energy flow coupling network based on the target energy interaction constraint, the target timing optimization result, and the dynamic adjustment instruction, and drive the data flow processing units within the shards to perform distributed big data real-time stream processing by using the distributed dynamic sharding configuration instructions.

[0092] Figure 2 The described distributed big data real-time stream processing system can execute Figure 1 the distributed big data real-time stream processing method described in the illustrated embodiment. The implementation principle and technical effects will not be elaborated further. For the distributed big data real-time stream processing method and system in the above embodiments, the specific manners in which each module and unit perform operations have been described in detail in the embodiments related to the method, and will not be elaborated here.

[0093] In a possible design, Figure 2 the distributed big data real-time stream processing method and system in the illustrated embodiment can be implemented as a computing device, such as Figure 3 shown, the computing device may include a storage component 31 and a processing component 32; The storage component 31 stores one or more computer instructions, and the one or more computer instructions are called and executed by the processing component 32.

[0094] The processing component 32 is configured to: generate a topological sequence diagram based on the power network, the thermal network, the natural gas network, and the real-time operating state parameters; generate a set of dynamic sharding rules for the multi-energy flow coupling network based on the topological sequence diagram and the real-time operating state parameters, where the set of dynamic sharding rules includes the power network topology structure, the multi-energy flow coupling data, and the dynamic sharding trigger condition; calculate a dynamic threshold based on the set of dynamic sharding rules, and generate a dynamic adjustment instruction for the sharding boundary when any load factor exceeds the dynamic threshold, where the load factor includes the node voltage deviation rate, the node pressure fluctuation gradient, and the return water temperature deviation rate; construct a hierarchical collaborative optimization architecture based on the dynamic adjustment instruction and the multi-energy flow coupling data to determine the target energy interaction constraint and the target timing optimization result between shards; generate distributed dynamic sharding configuration instructions for the multi-energy flow coupling network based on the target energy interaction constraint, the target timing optimization result, and the dynamic adjustment instruction, and drive the data flow processing units within the shards to perform distributed big data real-time stream processing by using the distributed dynamic sharding configuration instructions.

[0095] Among them, the processing component 32 may include one or more processors to execute computer instructions to complete all or part of the steps in the above methods. Of course, the processing component may also be implemented by one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors or other electronic components for executing the above methods.

[0096] The storage component 31 is configured to store various types of data to support the operation of the terminal. The storage component may be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0097] Of course, the computing device may also necessarily include other components, such as input / output interfaces, display components, communication components, etc.

[0098] The input / output interface provides an interface between the processing component and the peripheral interface module, and the above peripheral interface module may be an output device, an input device, etc.

[0099] The communication component is configured to facilitate communication between the computing device and other devices in a wired or wireless manner, etc.

[0100] Among them, the computing device may be a physical device or an elastic computing host provided by a cloud computing platform, etc. At this time, the computing device may refer to a cloud server, and the above processing component, storage component, etc. may be basic server resources leased or purchased from a cloud computing platform.

[0101] The embodiment of the present invention also provides a computer storage medium storing a computer program, and when the computer program is executed by a computer, it can implement the Figure 1 distributed big data real-time stream processing method shown in the above embodiments.

[0102] Those skilled in the art can clearly understand that for the convenience and brevity of description, the specific working processes of the above-described systems, devices, and units can refer to the corresponding processes in the foregoing method embodiments, and will not be described herein again.

[0103] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. A person of ordinary skill in the art can understand and implement it without creative labor.

[0104] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on such an understanding, the essence of the above technical solution, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0105] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements for some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A distributed big data real-time stream processing method, characterized in that: include: Based on the power network, the heat network, the natural gas network and the real-time operation status parameters, a topology sequence diagram is generated, and based on the topology sequence diagram and the real-time operation status parameters, a dynamic slicing rule set of the multi-energy flow coupling network is generated, wherein the dynamic slicing rule set includes the power network topology structure, the multi-energy flow coupling data and the dynamic slicing triggering conditions; Based on the dynamic slicing rule set, a dynamic threshold is calculated, and when any load factor exceeds the dynamic threshold, a dynamic adjustment instruction of the slicing boundary is generated, wherein the load factor includes a node voltage deviation rate, a node pressure fluctuation gradient, and a return water temperature deviation rate; Based on the dynamic adjustment instructions and the multi-energy flow coupling data, a hierarchical collaborative optimization architecture is constructed to determine target energy interaction constraints and target timing optimization results between slices; Based on the target energy interaction constraint, the target timing optimization result and the dynamic adjustment instruction, a distributed dynamic sharding configuration instruction of the multi-energy flow coupling network is generated, and the distributed dynamic sharding configuration instruction is used to drive the data flow processing unit in the shard to perform distributed big data real-time flow processing.

2. The method according to claim 1, characterized in that Based on the dynamic adjustment instruction and the multi-energy flow coupling data, a hierarchical collaborative optimization architecture is constructed to determine the target energy interaction constraints and target timing optimization results between the slices, including: Based on the core layer node betweenness in the power network and the multi-energy flow coupling data, a dynamic weight matrix is ​​generated by combining the multi-energy flow coupling factor weight and the line capacity weight; A distributed consistency optimization algorithm is used to solve the output priorities of photovoltaic and energy storage devices in the shard in parallel, generate timing optimization instructions for the devices in the shard, and combine the pre-regulation instructions of cogeneration to generate the verified shard boundary power demand constraints; Based on the verified shard boundary power demand constraint and the dynamic weight matrix, collaboratively optimize the cross-shard energy interaction rules to generate inter-shard energy interaction constraints and cross-shard timing collaboration rules; Based on the inter-shard energy interaction constraints and the cross-shard timing coordination rules, when the difference between the core layer node betweenness and the preset betweenness value exceeds the limit threshold, the topological structure of the shard is reconstructed, and the reorganized shard energy interaction constraints and timing optimization rules are generated in combination with the dynamic weight matrix and the pre-adjustment instructions; When an abnormality is detected in the node voltage deviation coefficient or the thermal return water temperature within the slice, the dynamic weight matrix is ​​frozen using the reorganized slice energy interaction constraints and the timing optimization rules, and the adjacent slice transmission margins are redistributed to generate safe mode energy interaction constraints and abnormal timing optimization results. When the abnormality is resolved, the dynamic weight matrix is ​​optimized based on the safe mode energy interaction constraints to generate target energy interaction constraints and target timing optimization results.

3. The method according to claim 2, characterized in that A distributed consistency optimization algorithm is used to solve the output priorities of photovoltaic and energy storage devices in the shard in parallel, generate timing optimization instructions for the devices in the shard, and combine the pre-regulation instructions of cogeneration to generate the verified shard boundary power demand constraints, including: Based on the irradiance fluctuation coefficient of photovoltaic equipment, the discrete gradient of the state of charge of energy storage equipment and the topological connectivity of equipment, a dynamic priority weight matrix of equipment in the shard is constructed; Performing distributed consistency optimization on the dynamic priority weight matrix using an alternating direction multiplier method to generate timing optimization instructions; The ratio of the return water temperature extreme difference within the sliding time window to the set threshold is used as the reference sensitivity, and the reference sensitivity and the historical time lag data are combined to generate a time lag compensation coefficient, and the reference sensitivity and the time lag compensation coefficient are convolved to generate the intensity parameter of the pre-regulation instruction; Using a bidirectional long short-term memory network, performing cross-time step correlation analysis on the intensity parameters of the timing optimization instruction and the pre-adjustment instruction to obtain a correlation analysis result, and calculating a preliminary boundary power demand constraint based on a load factor, the dynamic priority weight matrix and the correlation analysis result; A constraint validity verification mechanism is constructed, a feasibility index is defined, and the feasibility index is verified using the constraint validity verification mechanism. When the feasibility index is lower than a preset threshold, the dynamic priority weight matrix is ​​reconstructed to generate a verified boundary power demand constraint.

4. The method according to claim 1, characterized in that: Based on the power network, the heat network, the natural gas network and the real-time operation status parameters, a topology sequence diagram is generated. Based on the topology sequence diagram and the real-time operation status parameters, a dynamic slicing rule set of the multi-energy flow coupling network is generated. The dynamic slicing rule set includes the power network topology structure, the multi-energy flow coupling data and the dynamic slicing triggering conditions, including: Obtain the topological connection relationship of the power network, the hydraulic balance parameters of the thermal network, and the pressure and flow dynamic relationship of the natural gas network to construct the physical connection relationship of the multi-energy flow coupling network, and generate a topological sequence diagram by combining the real-time operating state parameters and the physical connection relationship; Based on the real-time operating status parameters, the node voltage deviation rate of the power network, the node return water temperature of the thermal network, and the node pressure fluctuation gradient of the natural gas network are predicted in time series to obtain multiple node status prediction values; The plurality of node state prediction values ​​are trained by using elastic network regression to determine the weight of the multi-energy flow coupling factor, and based on the weight of the multi-energy flow coupling factor, the plurality of node state prediction values ​​are linearly combined to calculate the multi-energy flow coupling strength, and a spatial distribution matrix characterizing the multi-energy flow coupling relationship is constructed based on the topological sequence diagram and the multi-energy flow coupling strength, and the multi-energy flow coupling data is generated by combining the multi-energy flow coupling strength and the real-time state operation parameters; Using a spectral clustering algorithm and a cosine similarity metric, the spatial distribution matrix is ​​dynamically sliced ​​to generate an initial slice set; A dynamic sharding trigger condition set is generated by using an extreme gradient boosting classifier trained based on historical load data, and a dynamic sharding rule set of a multi-energy flow coupling network is generated by combining the multi-energy flow coupling data, the initial sharding set and the dynamic sharding trigger condition set.

5. The method according to claim 4, characterized in that The topological connection relationship of the power network, the hydraulic balance parameters of the thermal network, and the pressure and flow dynamic relationship of the natural gas network are obtained to construct the physical connection relationship of the multi-energy flow coupling network. Combined with the real-time operating state parameters and the physical connection relationship, a topological sequence diagram is generated, including: Collecting power network equipment connection relationship data, parsing the physical connection path from the distribution transformer to the user node, generating a power network adjacency matrix, combining circuit breaker status data and node voltage deviation rate, optimizing the power network adjacency matrix, and generating a dynamic topology feature matrix containing voltage deviation attributes; According to the supply and return water temperature data and flow time series data, the return water temperature deviation rate is predicted, and a standardized temperature deviation rate sequence is generated to calculate the temperature deviation rate difference of adjacent nodes. Based on the temperature deviation rate difference, the hydraulic balance state of the thermal branches in the thermal network is marked to generate a thermal network balance coefficient matrix; Based on the pressure monitoring data of the natural gas network, the node pressure distribution data is generated, and the pipeline pressure decay rate and flow fluctuation gradient are jointly predicted and processed by combining the Weibull distribution model and the long short-term memory network to generate the dynamic gradient matrix of the natural gas network; Based on the dynamic topological characteristic matrix, the thermal network balance coefficient matrix and the natural gas network dynamic gradient matrix, a multi-energy flow physical connection relationship matrix is ​​generated. Based on the real-time load fluctuation rate, the coupling weight of the multi-energy flow physical connection relationship matrix is ​​dynamically optimized to generate a topological sequence diagram.

6. The method according to claim 4, characterized in that The spatial distribution matrix is ​​dynamically sliced ​​using a spectral clustering algorithm and cosine similarity measurement to generate an initial slice set, including: Based on the spatial distribution matrix, the real-time state operating parameters and the time decay factor, a node similarity matrix characterizing the coupling strength between nodes is constructed, and the time decay factor is dynamically adjusted according to the real-time load fluctuation rate; Based on the node similarity matrix and node degree distribution characteristics, a normalized coupling relationship matrix is ​​generated; Using a manifold learning algorithm to perform dimensionality reduction processing on the normalized coupling relationship matrix to generate a dimensionality-reduced feature matrix; Based on the reduced-dimensional feature matrix, the improved clustering algorithm with integrated balanced regularization term and the alternating direction multiplier method are combined to optimize the standard deviation of the multi-energy flow coupling intensity within the shard and the cross-shard channel ratio constraints to generate an initial shard set.

7. The method according to claim 1, characterized in that Based on the dynamic sharding rule set, a dynamic threshold is calculated. When any load factor exceeds the dynamic threshold, a dynamic adjustment instruction of the sharding boundary is generated. The load factor includes a node voltage deviation rate, a node pressure fluctuation gradient, and a return water temperature deviation rate, including: Based on the dynamic sharding rule set, combined with the dynamic mean, dynamic standard deviation and elasticity coefficient of the load factor in the sliding time window, a dynamic threshold is calculated; Based on the multi-energy flow coupling strength, the standard deviation of the multi-energy flow coupling strength corresponding to each slice is calculated, and the target slice whose standard deviation of the multi-energy flow coupling strength is higher than a preset standard value is selected; According to the dynamic threshold and the elastic coefficient, the load overflow ratio is calculated to generate a shard contraction coefficient or an expansion coefficient, and the energy interaction weight matrix of adjacent shards is updated in combination with the graph neural network to obtain an updated energy interaction weight matrix; According to the updated energy interaction weight matrix, setting the transmission power gradient and minimum proportion threshold of the cross-shard energy interaction channel during the shard boundary adjustment process; Based on the preset conflict arbitration rules, when any load factor is detected to exceed the dynamic threshold, a dynamic adjustment instruction for the slice boundary is generated in combination with the slice contraction coefficient or expansion coefficient, the updated energy interaction weight matrix, the transmission power gradient and the minimum share threshold.

8. A distributed big data real-time stream processing system, characterized in that: include: A generation module, for generating a topology sequence diagram based on the power network, the heat network, the natural gas network and the real-time operation status parameters, and generating a dynamic slicing rule set of the multi-energy flow coupling network based on the topology sequence diagram and the real-time operation status parameters, wherein the dynamic slicing rule set includes the power network topology structure, the multi-energy flow coupling data and the dynamic slicing triggering condition; A calculation module, configured to calculate a dynamic threshold based on the dynamic sharding rule set, and generate a dynamic adjustment instruction for a sharding boundary when any load factor exceeds the dynamic threshold, wherein the load factors include a node voltage deviation rate, a node pressure fluctuation gradient, and a return water temperature deviation rate; A determination module, configured to construct a hierarchical collaborative optimization framework based on the dynamic adjustment instruction and the multi-energy flow coupling data to determine target energy interaction constraints and target timing optimization results between slices; A processing module is used to generate a distributed dynamic sharding configuration instruction for a multi-energy flow coupling network based on the target energy interaction constraint, the target timing optimization result and the dynamic adjustment instruction, and use the distributed dynamic sharding configuration instruction to drive the data flow processing unit within the shard to perform distributed big data real-time flow processing.

9. A computing device, characterized in that It comprises a processing component and a storage component; the storage component stores one or more computer instructions; the one or more computer instructions are used to be called and executed by the processing component to implement a distributed big data real-time stream processing method as described in any one of claims 1 to 7.

10. A computer storage medium, characterized in that: A computer program is stored, and when the computer program is executed by a computer, a distributed big data real-time stream processing method as described in any one of claims 1 to 7 is implemented.

Images