Mechanism integrated multi-objective optimization method and system for energy system

Abstract

The application discloses a mechanism-fused multi-objective optimization method and system for an energy system, and relates to the field of control system optimization. The method comprises the following steps: acquiring multi-source operation data of each energy device in the energy system and time-of-use electricity price information; constructing energy efficiency mechanism models of the energy devices, and constructing a multi-device coupling mechanism constraint model based on the energy efficiency mechanism models and energy flow, information flow and control flow among the energy devices; constructing a comprehensive benefit return function; based on a system state space, outputting initial action variables by an intelligent agent with the goal of maximizing the comprehensive benefit return function; performing feasibility checking on the initial action variables, clipping the action variables that violate the constraint conditions to the feasible region to obtain clipped action variables, and receiving feedback information; updating the system state space based on the feedback information, and entering the iterative optimization of the next time. The application can improve the robustness of multi-device collaborative optimization in a time-varying environment.

Description

Technical Field

[0001] This application relates to the field of control system optimization, and in particular to a mechanism-integrated multi-objective optimization method and system for energy systems. Background Technology

[0002] Regional energy stations provide cooling and heating to urban areas through the coordinated operation of electromechanical equipment such as chillers, heat pumps, water pumps, and energy storage devices. The setting of equipment operating parameters directly determines the system's energy efficiency. In actual operation, the load fluctuates continuously with the outdoor environment and user-side energy consumption patterns. The various devices form a strong coupling relationship through hot and cold medium pipelines and electrical connections, making parameter optimization a high-dimensional nonlinear dynamic optimization problem. In existing operation and regulation, the parameter settings of different devices are mostly completed independently by their respective control loops, lacking an effective means of coordinating and regulating the parameters of multiple devices at the system level. As a result, the overall system energy efficiency often deviates from the optimal range.

[0003] To address the issue of low energy efficiency caused by the coupling of multiple devices, existing technologies have proposed an optimization and adjustment method based on a system-wide energy balance model. This method takes the entire energy station as the object, establishes a system energy balance equation encompassing the input-output relationships of each device, and solves for all device parameter setpoints in one go under given boundary conditions, with the objective function of minimizing total system energy consumption. Chilled water outlet temperature, pump frequency, unit load rate, and energy storage charging / discharging power are treated as unified optimization variables, and incorporated into equipment safety boundaries and supply-demand balance constraints, thus theoretically achieving coordinated determination of multiple device parameters.

[0004] However, the outdoor temperature and humidity of energy stations and user load demand fluctuate significantly throughout the day, while the equipment energy efficiency characteristic parameters (such as the relationship between the unit performance coefficient and load rate and temperature) on which the above methods rely are usually fixed in the model based on nameplate data under nominal operating conditions. When the boundary conditions deviate from the modeled operating conditions, a deviation occurs between the fixed energy efficiency parameters and the actual operating characteristics, causing the parameter setpoints obtained by the optimization model to deviate from the true optimal operating point, reducing the robustness of multi-equipment collaborative optimization in time-varying environments. Summary of the Invention

[0005] This application provides a mechanism-integrated multi-objective optimization method and system for energy systems, which addresses the technical problem of how to improve the robustness of multi-device collaborative optimization in time-varying environments.

[0006] In a first aspect, embodiments of this application provide a mechanism-integrated multi-objective optimization method for energy systems, including: Acquire multi-source operation data and time-of-use electricity price information for various energy devices within the energy system; Based on multi-source operation data, an energy efficiency mechanism model for energy equipment is constructed. Based on each energy efficiency mechanism model and the energy flow, information flow and control flow between each energy equipment, a multi-equipment coupling mechanism constraint model is constructed. The multi-equipment coupling mechanism constraint model includes system-level supply and demand balance constraints, energy efficiency mechanism relationships between each energy equipment, equipment safe operation range, and state transition relationships. Based on time-of-use electricity pricing information and equipment coordination parameters of the energy system, a comprehensive benefit-return function including energy efficiency, cost, and equipment items is constructed. Based on the current state space of the energy system, a pre-trained reinforcement learning agent outputs the initial action variables of the energy system with the objective of maximizing the function value of the comprehensive benefit reward function. Using the multi-device coupling mechanism constraint model as a constraint condition, the feasibility of the initial action variables is verified, and the action variables that violate the constraint conditions are clipped to the feasible region to obtain the clipped action variables. The trimmed action variables are sent to the corresponding energy devices for execution, and feedback information is received from each energy device. The system state space is updated based on the feedback information, and the iteration optimization begins at the next time step.

[0007] Optionally, based on time-of-use electricity pricing information and equipment coordination parameters of the energy system, a comprehensive benefit return function is constructed, including an energy efficiency item, a cost item, and an equipment item. This includes: using the ratio of the system's supplied energy to its total power consumption as the energy efficiency item based on the equipment coordination parameters; determining the real-time electricity price corresponding to the current moment based on the time-of-use electricity pricing information, and subtracting the demand response revenue at the current moment from the product of the real-time electricity price and the total power consumption to obtain the cost item; calculating the total number of start-ups and shutdowns of each energy device within a preset time window based on the equipment coordination parameters, and calculating the total start-up and shutdown loss value of the energy system based on the preset single start-up and shutdown loss coefficient and the total number of start-ups and shutdowns; calculating the total efficiency deviation loss value of the energy system based on the actual load rate of each energy device and the preset high-efficiency load rate range; constructing the equipment item based on the total start-up and shutdown loss value and the total efficiency deviation loss value; calculating the first product of the energy efficiency item and a preset first weight, the second product of the cost item and a preset second weight, and the third product of the equipment item and a preset third weight; and subtracting the second and third products from the first product to obtain the comprehensive benefit return function.

[0008] Optionally, before outputting the initial action variables of the energy system based on the current system state space of the energy system by a pre-trained reinforcement learning agent with the objective of maximizing the function value of the comprehensive benefit reward function, the method further includes: using a multi-device coupling mechanism constraint model as a simulation training environment, and having the reinforcement learning agent output training action variables based on the current simulation state space with the objective of maximizing the function value of the comprehensive benefit reward function; verifying whether the values ​​of each training parameter to be adjusted in the training action variables fall within the safe operating range of the corresponding energy device; if they fall within, they are retained; if they exceed, the values ​​of the training parameters to be adjusted are adjusted to the boundary extreme values ​​of the safe operating range of the device, generating preliminary training pruning actions; and calculating the energy supply and load requirements of the energy system when running according to the preliminary training pruning actions based on the multi-device coupling mechanism constraint model. Find the training imbalance between different energy sources. When the training imbalance exceeds the preset training tolerance, decrease the output adjustment amplitude of each energy source in ascending order of sensitivity to the training imbalance until the training imbalance falls within the preset training tolerance, thus obtaining the pruned training action variables. Construct a potential function to evaluate the system state evolution, using the preset optimal energy efficiency as a reference. Calculate the first function value in the simulation state space at the current moment and the second function value in the simulation state space at the next moment based on the potential function. Superimpose the function difference between the first and second function values ​​onto the comprehensive benefit reward function to generate the shaped comprehensive benefit reward function. Update the reinforcement learning agent based on the shaped comprehensive benefit reward function, repeating the iteration until the policy converges, thus obtaining the pre-trained reinforcement learning agent.

[0009] Optionally, based on the current state space of the energy system, a pre-trained reinforcement learning agent outputs initial action variables of the energy system with the objective of maximizing the function value of the comprehensive benefit return function. These include: identifying the current electricity price period based on time-of-use pricing information; when the electricity price period is a low-price off-peak period, outputting a first coordinated adjustment parameter with the objective of maximizing the function value of the comprehensive benefit return function, and using this first coordinated adjustment parameter as the initial action variable of the energy system. This first coordinated adjustment parameter is used to increase the loading rate of the production capacity equipment, enabling the equipment to increase its output beyond meeting current load demand, and inputting the additional output capacity into the energy storage device to execute a charging command; when the electricity price period is a high-price peak period, outputting a second coordinated adjustment parameter with the objective of maximizing the function value of the comprehensive benefit return function and minimizing cost and equipment costs, and using this second coordinated adjustment parameter as the initial action variable. This second coordinated adjustment parameter is used to drive the energy storage device to execute an energy release command to replace part of the real-time capacity of the production capacity equipment and reduce the loading rate of the production capacity equipment; during the energy release command period, the loading rate of the production capacity equipment whose capacity is not replaced is constrained within a preset high-efficiency loading rate range.

[0010] Optionally, the multi-device coupling mechanism constraint model is used as a constraint condition to perform feasibility verification on the initial action variables. Action variables that violate the constraint conditions are trimmed to the feasible region to obtain trimmed action variables. This includes: verifying whether the values ​​of each parameter to be adjusted in the initial action variables fall within the safe operating range of the corresponding energy device. If they fall within, they are retained; if they exceed, the values ​​of the parameters to be adjusted are adjusted to the boundary extreme values ​​of the safe operating range of the device, generating a preliminary trimming action; based on the multi-device coupling mechanism constraint model, the imbalance between the energy supply and load demand of the energy system is calculated when the preliminary trimming action is performed; when the imbalance exceeds the preset tolerance, the output adjustment range of the corresponding energy device is trimmed according to the order of sensitivity of each energy device's adjustment action to the imbalance from low to high, based on the preset trimming step size, until the imbalance falls within the preset tolerance, to obtain the trimmed action variables.

[0011] Optionally, before updating the system state space based on feedback information and entering the iterative optimization of the next time step, the method further includes: extracting multi-source feedback data of each energy device from the feedback information, and calculating the deviation between the expected performance coefficient of the multi-source feedback data and the actual operating performance coefficient based on the energy efficiency mechanism model of the energy device to obtain the operating deviation value; when the operating deviation value exceeds a preset anomaly judgment threshold for a consecutive preset number of sampling periods, the corresponding energy device is determined to be an operating abnormal device, and an anomaly detection alarm signal for the operating abnormal device is triggered, and the alarm level is determined according to the degree to which the operating deviation value exceeds the anomaly judgment threshold; in response to the anomaly detection alarm signal, an anomaly detection alarm signal is triggered. Abnormal parameters of frequently detected alarm signals are used as target analysis nodes. Based on the transmission directions of energy flow, information flow, and control flow in the multi-device coupling mechanism constraint model, a reverse causal tracing graph is established with the target analysis node as the search starting point. Tracing upstream along the connection edges of the reverse causal tracing graph, the multi-source feedback data of each upstream associated device is used as the correlation parameters. The contribution of each correlation parameter to the deviation of the target analysis node from the normal state is calculated, and the nodes corresponding to the correlation parameters whose contribution exceeds the preset contribution threshold are used as root cause nodes, and a physical fault propagation path is constructed. Based on the alarm level, the correlation parameters of the root cause node, and the physical fault propagation path, an interpretable diagnostic report of the abnormally operating equipment is generated.

[0012] Optionally, the system state space is updated based on feedback information to proceed to the next iteration optimization, including: extracting efficiency degradation features of malfunctioning equipment from the fault interpretable diagnostic report and mapping the efficiency degradation features to state penalty variables; updating the state penalty variables and feedback information synchronously to the system state space of the next time step; during the iteration optimization process of the next time step, based on the system state space containing state penalty variables, dynamically reducing the loading priority of malfunctioning equipment in the global unit combination through a reinforcement learning agent; and, under the premise of satisfying the constraints of the multi-equipment coupling mechanism constraint model, reallocating the load of normal energy equipment through a reinforcement learning agent to generate safe and coordinated operation action variables.

[0013] Secondly, embodiments of this application provide a mechanism-integrated energy system multi-objective optimization system, which includes: one or more processors and a memory; the memory is coupled to one or more processors, the memory is used to store computer program code, the computer program code includes computer instructions, and one or more processors invoke the computer instructions to cause the mechanism-integrated energy system multi-objective optimization system to perform the method described in the first aspect and any possible implementation thereof.

[0014] Thirdly, embodiments of this application provide a computer-readable storage medium including instructions that, when executed on a mechanism-integrated energy system multi-objective optimization system, cause the mechanism-integrated energy system multi-objective optimization system to perform the method described in the first aspect and any possible implementation thereof.

[0015] Fourthly, embodiments of this application provide a computer program product containing instructions that, when the computer program product is run on a mechanism-integrated energy system multi-objective optimization system, cause the mechanism-integrated energy system multi-objective optimization system to perform the method described in the first aspect and any possible implementation thereof.

[0016] In summary, one or more technical solutions provided in this application have at least the following technical effects or advantages: 1. By pre-training reinforcement learning agents and constructing a multi-device coupling mechanism constraint model that includes system-level supply and demand balance constraints, equipment energy efficiency mechanism relationships, equipment safe operating ranges, and state transition relationships, the model is used as a constraint condition for feasibility verification after the agent outputs initial action variables. Action variables that violate the constraints are clipped to the feasible region before being issued for execution. This ensures that every control command output by the reinforcement learning agent physically satisfies the equipment safety boundary and system supply and demand balance. This overcomes the defect of pure data-driven methods that may output parameter settings that violate physical constraints under varying operating conditions, and improves the robustness of the energy system under wide time-varying operating conditions.

[0017] 2. By constructing a comprehensive benefit-reward function that includes energy efficiency, cost, and equipment components, the cost component calculates electricity costs based on time-of-use pricing, and the equipment component represents equipment lifespan degradation using a weighted sum of start-up and shutdown losses and efficiency deviation losses. During training, the reward function is shaped by the difference between the simulation state space at the current and next time steps on the potential function, ensuring that every exploration action of the agent during training is within the physically feasible region. Simultaneously, the potential function shaping injects physical mechanism cognition into the reward signal to accelerate the convergence of the policy to the efficient feasible region, thereby improving the safety, convergence speed, and sample efficiency of reinforcement learning policy training.

[0018] 3. By outputting corresponding coordinated adjustment parameters based on electricity price periods during the online deployment phase, the loading rate of production capacity equipment is increased during off-peak hours, and the additional energy generated is input into the energy storage device for charging. During peak hours, the energy storage device is driven to release energy to replace part of the real-time capacity of the production capacity equipment and reduce the equipment loading rate. At the same time, the unreplaced production capacity equipment is constrained within the preset high-efficiency loading rate range. This achieves coordinated optimization of energy storage arbitrage revenue and unit high-efficiency operation, avoiding the remaining equipment from operating in the inefficient zone or frequent start-stop due to load replacement by energy storage, and reducing the cost of purchasing high-priced electricity during peak hours and the mechanical wear of equipment.

[0019] 4. By calculating the deviation between the actual operating performance coefficient and the expected performance coefficient based on the energy efficiency mechanism model, abnormal equipment is identified. The alarm level is determined according to the degree of deviation. Based on the energy flow, information flow and control flow transmission direction in the multi-equipment coupling mechanism constraint model, the contribution of each upstream related parameter to the abnormal parameter is traced. The node corresponding to the parameter whose contribution exceeds the threshold is regarded as the root cause node, and an interpretable diagnostic report containing alarm level, root cause parameter and fault propagation path is generated. At the same time, the efficiency degradation characteristics of abnormal equipment are mapped into state penalty variables and updated to the system state space at the next moment. This enables the reinforcement learning agent to automatically reduce the loading priority of abnormal equipment and reallocate the load. This realizes the active intervention of fault diagnosis results in control strategy, overcomes the defects of existing technologies where diagnosis and control are disconnected and diagnostic results are only used for alarms and cannot automatically drive strategy adjustment, and improves the autonomous adaptability of energy system under equipment performance degradation conditions. Attached Figure Description

[0020] Figure 1 This is a flowchart illustrating the mechanism fusion-based multi-objective optimization method for energy systems provided in this application embodiment; Figure 2 This is another schematic diagram of the mechanism fusion-based multi-objective optimization method for energy systems provided in the embodiments of this application; Figure 3 This is a schematic diagram of the coupling relationship of energy devices provided in the embodiments of this application; Figure 4 This is a schematic diagram of the process for constructing a comprehensive benefit return function provided in an embodiment of this application; Figure 5 A flowchart illustrating the output working variables provided in this application embodiment; Figure 6 This is a schematic diagram of the process of training a reinforcement learning agent provided in an embodiment of this application; Figure 7 This is a flowchart illustrating the exception feedback provided in an embodiment of this application; Figure 8 This is a schematic diagram of the structure of a mechanism-integrated multi-objective optimization system for energy systems provided in an embodiment of this application.

[0021] Explanation of reference numerals in the attached drawings: 401, Central Processing Unit; 402, Read-Only Memory; 403, Random Access Memory; 404, Bus; 405, Input / Output Interface; 406, Input Section; 407, Output Section; 408, Storage Section; 409, Communication Section; 410, Driver; 411, Removable Media. Detailed Implementation

[0022] To enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments.

[0023] In the description of the embodiments of this application, the words "for example" or "for instance" are used to indicate examples, illustrations, or explanations. Any embodiment or design that is described as "for example" or "for instance" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design options. Rather, the use of the words "for example" or "for instance" is intended to present the relevant concepts in a specific manner.

[0024] In the description of the embodiments of this application, the term "multiple" means two or more. For example, multiple systems means two or more systems, and multiple screen terminals means two or more screen terminals. Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the indicated technical features. Thus, a feature defined with "first" or "second" may explicitly or implicitly include one or more of that feature. The terms "comprising," "including," "having," and variations thereof all mean "including but not limited to," unless otherwise specifically emphasized.

[0025] This application provides a mechanism-integrated multi-objective optimization method and system for energy systems, which addresses the technical problem of how to improve the robustness of multi-device collaborative optimization in time-varying environments.

[0026] Figure 1 This is a flowchart illustrating the mechanism fusion-based multi-objective optimization method for energy systems provided in this application embodiment.

[0027] This application discloses a mechanism-integrated multi-objective optimization method for energy systems, such as... Figure 1 As shown, the steps include the following.

[0028] S101. Obtain multi-source operation data and time-of-use electricity price information of various energy devices in the energy system.

[0029] Specifically, multi-source operational data from various energy devices within the energy system is collected via fieldbus or sensor networks. This includes equipment operating parameters, energy consumption and electrical data, and boundary conditions. Equipment operating parameters refer to measurable physical quantities of each energy device during operation, including at least the chilled water supply and return temperatures and cooling water inlet and outlet temperatures of chillers or heat pumps, the inlet and outlet pressures and flow rates of circulating water pumps, the operating frequency and current of each device, the state of charge or cold storage status of energy storage devices, and the operating / stop status indicators of each device. Energy consumption and electrical data refer to the active power and cumulative power consumption of each energy device, obtained through the power measurement modules built into smart meters or frequency converters. Boundary conditions refer to external factors affecting the load demand of the energy system, including at least outdoor dry-bulb temperature, outdoor relative humidity, and the current total system load demand. Time-of-use pricing information refers to the peak-valley time-of-use pricing policy published by the local power management department, including at least the division of three time intervals—peak, normal, and valley periods—and their corresponding price values, as well as the effective start and end dates of the pricing policy. In addition, nameplate parameters for each energy device are acquired. These parameters include at least rated cooling or heating capacity, rated power, rated flow rate, rated head, and partial load performance curve data provided by the manufacturer, which are used for the subsequent construction of energy efficiency mechanism models. The collected multi-source operating data, after denoising and standardization, are aligned by timestamps to form basic data for the system state space, containing both device-level and system-level parameters, for use in subsequent steps.

[0030] For example, see reference Figure 2 , Figure 2This is another flowchart illustrating the mechanism-integrated multi-objective optimization method for energy systems provided in this application. Taking a regional cooling system as an example, the cooling system includes chillers, cooling pumps, and energy storage tanks. The system acquires the chilled water supply and return temperatures (7℃ / 12℃), cooling water inlet and outlet temperatures (32℃ / 37℃), and evaporator and condenser flow rates (both 300 m³ / h) of two centrifugal chillers via a Modbus bus with a 1-minute sampling period. The system also reads the active power of each unit (per unit) using a smart meter. The system uses a 380kW generator and its cumulative electricity consumption (current daily cumulative electricity consumption is 120kWh). The outdoor dry-bulb temperature (35℃) and relative humidity (65%) are obtained through outdoor temperature and humidity sensors. Based on the time-of-use electricity price information, it is confirmed that the current time (10:00 AM) is in the peak period (9:00-12:00 PM), and the electricity price is 1.2 yuan / kWh. At the same time, the current cooling rate of the chilled water tank is read as 80%. The above data is standardized (standardized with a mean of 0 and a variance of 1) to construct the system state space.

[0031] S102. Construct energy efficiency mechanism models corresponding to energy equipment based on multi-source operation data, and construct multi-equipment coupling mechanism constraint models based on each energy efficiency mechanism model and the energy flow, information flow and control flow between each energy equipment. The multi-equipment coupling mechanism constraint models include system-level supply and demand balance constraints, energy efficiency mechanism relationships between each energy equipment, equipment safe operation range and state transition relationships.

[0032] Specifically, energy efficiency mechanism models for each energy device are constructed based on multi-source operational data, referencing... Figure 2 For refrigeration equipment, a mechanistic energy efficiency model is constructed. Based on the rated cooling capacity Qevap, rated power, and partial load performance curves in the equipment nameplate parameters, a bivariate polynomial fitting method is used to construct a model with the partial load rate PLR ​​and cooling water inlet temperature T. chw and cooling water outlet temperature T cw The energy efficiency ratio function (f) is the independent variable. chiller (PLR, T) chw T cw )=Q evap / W in This function uses the cooling capacity Q. evap With input electrical power W in The ratio of the two values ​​is used as the coefficient of performance (COP). For water pump power consumption models constructed for distribution equipment, based on the power-flow-head relationship in fluid mechanics and combined with the characteristics of variable frequency speed control, a mapping relationship (W) between power and flow ratio and speed ratio is established. pump =ρgQH / η pump ρ is the density of the medium, g is the acceleration due to gravity, Q is the flow rate, H is the head, and η is the displacement. pump(For pump efficiency). For energy storage devices, an energy storage power consumption model is constructed, and a dynamic balance equation (Uenergy) of the state of charge over time is established based on the principle of energy conservation. t+1 =U t +η ch P ch Δt−P dis Δt / η dis U t For the energy stored at time t, U t+1 P stores energy for the next time step t+1. ch With P dis These represent the charging and releasing power, η. ch With η dis Here, Δt represents the charge / discharge efficiency (where Δt is the time step). The equation is corrected using a charge / discharge efficiency coefficient, which ranges from 0.75 to 0.95 depending on the device type and operating conditions.

[0033] Based on this, refer to Figure 3 , Figure 3 This is a schematic diagram of the coupling relationship of energy equipment provided in the embodiments of this application. It integrates the energy efficiency mechanism models of each individual device and constructs a multi-device coupling mechanism constraint model based on the energy flow, information flow, and control flow between each energy device. The energy flow is established based on the flow direction of the hot and cold media in the pipeline network and the heat transfer path, determining the supply and demand energy balance equation between each production capacity device, energy storage device, and load end: , To supply total energy to the system ( (For the capacity of the kth unit), L t For load demand, Q loss For transmission and distribution losses. The information flow is established based on the sensor deployment locations and data transmission paths, determining the acquisition, transmission, and aggregation relationships of each operating parameter. The control flow is established based on the control command issuance paths, determining the coordination logic of each adjustment command among the devices. This multi-device coupling mechanism constraint model includes at least four components: system-level supply and demand balance constraints composed of the above-mentioned supply and demand energy balance equations, which limit the total relationship between the output and load of each device in equation form; energy efficiency mechanism relationships composed of the energy efficiency mechanism functions of each individual device, used to describe the physical mapping between device inputs and outputs; the safe operating range of the device determined by the device's factory safety thresholds, including upper and lower limits of temperature, pressure, flow, and load rate; and the state transition relationships derived from the energy balance equations and the dynamic characteristics of the devices, used to predict the system state at the next moment based on the current state and candidate actions.

[0034] Through the above embodiments, the energy efficiency mechanism model constructed based on thermodynamics, fluid mechanics principles, and equipment nameplate data can accurately describe the physical mapping between equipment input and output. Building upon this, a multi-equipment coupling mechanism constraint model is constructed by integrating three types of coupling relationships: energy flow, information flow, and control flow. This model unifies system-level supply and demand balance constraints, equipment safe operating ranges, and state transition relationships into a computable action domain boundary. This model provides a physical basis for action pruning of reinforcement learning agents, ensuring that each action variable output by the agent in the subsequent parameter optimization process theoretically satisfies equipment physical constraints and system energy conservation, thereby guaranteeing the safety and feasibility of the optimization strategy in practical engineering deployments.

[0035] S103. Based on time-of-use electricity price information and equipment coordination parameters of the energy system, construct a comprehensive benefit return function that includes energy efficiency items, cost items, and equipment items.

[0036] Specifically, the energy efficiency item is calculated based on the equipment coordination parameters of the energy system; a higher value indicates a higher overall energy efficiency level. The cost item is calculated based on time-of-use electricity pricing information and equipment coordination parameters; a lower value indicates lower electricity costs. The equipment item is calculated based on equipment coordination parameters. The weighted combination of the energy efficiency, cost, and equipment items is used as the comprehensive benefit return function. This can be achieved through the following steps, which are explained in detail below.

[0037] Based on the equipment coordination parameters of the energy system, the ratio of the system's energy supply to its total power consumption is used as the energy efficiency item. The real-time electricity price corresponding to the current moment is determined based on time-of-use pricing information. The product of the real-time electricity price and the total power consumption is subtracted from the demand response revenue at the current moment to obtain the cost item. Based on the equipment coordination parameters, the total number of start-ups and shutdowns of each energy device within a preset time window is counted. Based on the preset single start-up / shutdown loss coefficient and the total number of start-ups / shutdowns, the total start-up / shutdown loss value of the energy system is calculated. Based on the actual load rate of each energy device and the preset high-efficiency load rate range, the total efficiency deviation loss value of the energy system is calculated. Equipment items are constructed based on the total start-up / shutdown loss value and the total efficiency deviation loss value. The first product of the energy efficiency item and the preset first weight, the second product of the cost item and the preset second weight, and the third product of the equipment item and the preset third weight are calculated. The first product is subtracted from the second and third products to obtain the comprehensive benefit return function.

[0038] For details, please refer to Figure 2 and Figure 4 , Figure 4 This is a flowchart illustrating the construction of a comprehensive benefit return function provided in this application embodiment. Based on the equipment coordination parameters of the energy system, the system supply energy Q of the energy system is... supply Total power consumption (Power consumption of the unit W) k Total + Power consumption of water pump (W) pumpAuxiliary machine power consumption (W) aux The ratio of ) is used as the energy efficiency item ( The higher the ratio, the more energy is produced per unit of electrical energy consumed.

[0039] The real-time electricity price is determined based on time-of-use (TOU) pricing information. This TOU pricing information is determined by the peak-valley TOU pricing policy issued by the local power management department, and includes at least three time intervals: peak period, normal period, and valley period, along with their corresponding electricity prices. The real-time electricity price is then determined by finding the corresponding price within the current time interval. The real-time electricity price is compared with the total power consumption over a preset time step. The product of these is the current electricity cost, minus the current demand response revenue. , to obtain the cost item ( Demand response revenue refers to the economic incentives an energy system receives when participating in grid demand response projects, calculated based on the actual load reduction and the agreed compensation unit price. The cost item quantifies the system's net electricity expenditure per unit of time.

[0040] Based on equipment coordination parameters, the total number of start-stop cycles for each energy device within a preset time window is calculated. The preset time window is a predetermined duration interval preceding the current moment, the length of which is set according to the equipment type and operating characteristics, preferably between 1 hour and 24 hours. The cumulative number of times each device switches from a stopped state to an operating state or vice versa within this time window is also counted. This is based on a preset single start-stop loss coefficient λ1 and the total number of start-stop cycles N. sw The total start-up and shutdown loss value of the energy system is calculated. The single start-up and shutdown loss coefficient is set according to the equipment type and mechanical structure. Its physical meaning is the economic loss value caused to the equipment by a single start-up and shutdown. The single start-up and shutdown loss coefficient ranges from 5 to 20 for refrigeration equipment, from 1 to 5 for water pumps, and from 10 to 50 for boilers. This is based on the actual load rate (PLR) of each energy device. k With preset high-efficiency loading rate range Calculate the total efficiency deviation from the loss value ( The actual loading rate refers to the percentage of the equipment's current output to its rated output, obtained directly from the equipment's coordination parameters. The preset high-efficiency loading rate range refers to the range of loading rates within which the equipment can maintain high operating efficiency during operation. It is preset according to the equipment performance curve or factory test data. The high-efficiency loading rate range for refrigeration equipment is 60% to 100% of the rated loading rate, and for water pumps, it is 70% to 100% of the rated frequency. For any equipment, if its actual loading rate is higher than the upper limit of the high-efficiency loading rate range, the efficiency deviation is the difference between the actual loading rate and the upper limit of the range. If the actual loading rate is lower than the lower limit of the high-efficiency loading rate range, the efficiency deviation is the difference between the lower limit of the range and the actual loading rate. If the actual loading rate falls within the range, the efficiency deviation is zero. The total efficiency deviation loss value is obtained by summing the deviations of each equipment and multiplying them by the preset efficiency deviation coefficient λ2. The preset efficiency deviation coefficient is set according to the equipment type and is used to convert the dimensionless loading rate deviation into an economic loss value. The sum of the total start-stop loss value and the total efficiency deviation loss value is taken as the equipment item ( This is used to quantify the comprehensive losses caused by start-up and shutdown shocks and inefficient operation of equipment during operation.

[0041] Calculate the energy efficiency item and the preset first weight w eff The first product, the calculated cost term, and the preset second weight w cost The second product, the calculation device term, and the preset third weight w dev The third product, minus the second and third products of the first product, yields the comprehensive benefit return function. The preset first, second, and third weights are real numbers greater than or equal to 0 and less than or equal to 1, with their sum equal to 1. Their specific values ​​are configured according to the operational strategy. The first weight corresponding to the energy efficiency item adjusts the proportion of energy efficiency targets in the overall benefits; the second weight corresponding to the cost item adjusts the proportion of economic targets in the overall benefits; and the third weight corresponding to the equipment item adjusts the proportion of equipment lifespan protection targets in the overall benefits. When the operational strategy focuses on energy efficiency improvement, the first weight is greater than the second and third weights; when the operational strategy focuses on economic optimization, the second weight is greater than the first and third weights; and when the operational strategy focuses on equipment lifespan protection, the third weight is greater than the first and second weights. Under the default operational strategy, the first weight is 0.4, the second weight is 0.4, and the third weight is 0.2.

[0042] Through the above embodiments, the system energy efficiency, electricity costs and demand response benefits under time-of-use pricing, equipment start-up and shutdown losses, and load rate deviation losses are uniformly constructed into a comprehensive benefit return function. This return function serves as the objective function for the reinforcement learning agent's strategy optimization, enabling the agent to adaptively adjust its action strategy based on the current electricity price period and equipment status during the iterative optimization process, thereby maximizing comprehensive benefits while ensuring energy supply quality.

[0043] S104. Based on the current state space of the energy system, the pre-trained reinforcement learning agent outputs the initial action variables of the energy system with the goal of maximizing the function value of the comprehensive benefit reward function.

[0044] Specifically, the system state space is formed by standardizing the multi-source operational data acquired at the current moment into a state feature vector. The state feature vector includes at least the current load demand data, outdoor environmental parameters, current operating parameters of each energy device and the state of charge of the energy storage device, current time-of-use electricity price information, and time features. Among them, the load demand data, outdoor environmental parameters, and time-of-use electricity price information are derived from the boundary conditions obtained in step S101, the current operating parameters of each energy device and the state of charge of the energy storage device are derived from the equipment operating parameters obtained in step S101, and the time features include the current time in the sequence number of the day (such as which sampling point), the category label of whether the current day is a workday or a rest day, and the category label of whether the current day is in the energy supply season or the transition season. A pre-trained reinforcement learning agent refers to an agent that employs the actor-critic architecture in deep reinforcement learning algorithms and undergoes iterative training in a simulation training environment constrained by a multi-device coupling mechanism until the policy network parameters converge. During training, the agent's policy network optimizes by maximizing the function value of the comprehensive benefit reward function. The critic network evaluates the value of actions, and the actor network updates network parameters along the policy gradient direction to improve the expected reward. The initial action variables are the set of raw action instructions directly output by the policy network of the pre-trained reinforcement learning agent based on the current system state space without physical constraint verification. These include at least the setpoints for chilled or hot water outlet temperatures, the operating frequency instructions for each pump, the load rate instructions for each unit, the charging and discharging power instructions for the energy storage device, and the instructions for the number of units to be started. The values ​​of each parameter in the initial action variables are theoretically within the defined range of the action space when the reinforcement learning agent outputs them, but they have not yet undergone feasibility verification by the multi-device coupling mechanism constraint model, and therefore may violate the safety operation boundaries of some devices or system-level supply and demand balance constraints.

[0045] Based on the above embodiments, as an optional embodiment, for Figure 1 Step S104 shown can be implemented through the following steps, which are explained in detail below.

[0046] Based on time-of-use electricity price information, the current electricity price period is identified. When the electricity price period is a low-price off-peak period, a first coordinated adjustment parameter is output with the goal of maximizing the comprehensive benefit return function. This first coordinated adjustment parameter is used as the initial action variable of the energy system to increase the loading rate of the production capacity equipment, enabling the equipment to increase its output beyond meeting the current load demand. The additional output is then input into the energy storage device to execute a charging command. When the electricity price period is a high-price peak period, a second coordinated adjustment parameter is output with the goal of maximizing the comprehensive benefit return function and minimizing cost and equipment costs. This second coordinated adjustment parameter is used as the initial action variable to drive the energy storage device to execute an energy release command to replace part of the real-time production capacity of the production capacity equipment and reduce the loading rate of the equipment. During the energy release command period, the loading rate of the production capacity equipment that is not replaced is constrained within a preset high-efficiency loading rate range.

[0047] For details, see Figure 5 , Figure 5This is a flowchart illustrating the output working variables provided in this application embodiment. Based on the time-of-use electricity price information, the electricity price period of the current moment is identified. The electricity price period refers to the three time intervals—peak period, normal period, and valley period—pre-divided by the time-of-use electricity price policy. The corresponding electricity price period is determined according to the time interval to which the current moment belongs. When it is identified that the current moment is in a low-price valley period, the pre-trained reinforcement learning agent, with the optimization orientation of maximizing the function value of the comprehensive benefit return function and minimizing the cost term, outputs the first collaborative adjustment parameter as the initial action variable. The two objectives are mathematically equivalent to the same optimization direction, that is, both pointing to the strategy tendency of making full use of low-price electricity during valley periods. The first collaborative adjustment parameter is used to increase the loading rate of the capacity equipment, so that the capacity equipment increases its output on the basis of meeting the current load demand, and the surplus capacity of the capacity equipment after meeting the current load demand is introduced into the energy storage device through pipeline switching and valve adjustment to execute the charging command. When it is identified that the current time is during a peak electricity price period, the pre-trained reinforcement learning agent, guided by maximizing the value of the comprehensive benefit return function and minimizing the cost and equipment costs, outputs a second coordinated adjustment parameter as the initial action variable. Minimizing both increases the return function value, both pointing to a strategy of reducing the purchase of high-priced electricity during peak hours and protecting equipment lifespan. The second coordinated adjustment parameter is used to drive the energy storage device to execute an energy release command to replace part of the real-time capacity of the production equipment and reduce the load rate of the production equipment. During the energy release command, the load rate of the production equipment whose capacity is not replaced is constrained within a preset high-efficiency load rate range. That is, after the cold or heat released by the energy storage device has met part of the load demand, the remaining production equipment that still needs to operate should be limited to its high-efficiency operating range to avoid the remaining equipment operating in an inefficient zone due to load reduction. During normal periods, coordinated operation is carried out according to the needs of the energy system.

[0048] Through the above embodiments, the pre-trained reinforcement learning agent identifies the current electricity price period based on time-of-use pricing information. Guided by maximizing the overall benefit return function and minimizing cost and equipment costs, it adaptively outputs coordinated adjustment parameters. This achieves coordinated optimization of energy storage arbitrage profits and efficient equipment operation, effectively reducing overall operating costs and equipment mechanical losses while ensuring energy quality.

[0049] Based on the above embodiments, as an optional embodiment, see [link to embodiment]. Figure 2 ,against Figure 1 Before step S104 shown, steps S201-S204 may be included to pre-train the agent, which will be explained in detail below.

[0050] S201. Using the multi-device coupling mechanism constraint model as the simulation training environment, the reinforcement learning agent outputs training action variables based on the simulation state space at the current moment, with the goal of maximizing the function value of the comprehensive benefit reward function.

[0051] For details, please refer to Figure 6 , Figure 6 This is a flowchart illustrating the training process of a reinforcement learning agent provided in this embodiment, using a multi-device coupling mechanism constraint model as the simulation training environment. Given the current simulation state space and candidate training action variables, this simulation training environment can calculate the simulation state space at the next moment based on the state transition relationship, and calculate the corresponding immediate reward value based on the comprehensive benefit reward function. The simulation state space has the same characteristic dimensions and physical meaning as the real system state space in the online deployment phase. The difference lies in that the data of the simulation state space originates from the state evolution calculation within the simulation training environment, rather than from real-time acquisition by physical sensors. The characteristics of the simulation state space include at least simulation load demand data, simulation outdoor environmental parameters, current operating parameters of each simulation device and the simulation state of charge of the energy storage device, simulation time-of-use electricity price information, and simulation time characteristics. The simulation load demand data is randomly generated by the load model based on historical load data statistical patterns or a preset load curve, and the simulation outdoor environmental parameters are generated by the meteorological model based on design day meteorological data or typical meteorological year data. The reinforcement learning agent outputs training action variables based on the simulation state space at the current moment, with the goal of maximizing the function value of the comprehensive benefit reward function. The reinforcement learning agent adopts an actor-critic architecture. Its policy network takes the simulation state space as input and outputs training action variables. The training action variables are the actions output by the reinforcement learning agent during the training phase. Their data format is consistent with the initial action variables in the online deployment phase, including at least the setpoint of chilled or hot water outlet temperature, the operating frequency command of each water pump, the load rate command of each unit, the charging and discharging power command of the energy storage device, and the command of the number of units to be started. During the training phase, the training action variables output by the policy network are input into the simulation environment, executed after action pruning, and the simulation state space and instant reward value at the next moment are calculated based on the simulation state transition after execution, which are used for parameter updates of the policy network and the critic network.

[0052] The simulation training environment refers to a virtual interactive environment constructed from a multi-device coupling mechanism constraint model, used for offline training of reinforcement learning agents, and possessing state transition and reward calculation functions. The simulation state space refers to the simulated system state feature vectors generated in the simulation training environment, having the same feature dimensions and physical meaning as the real system state space. Training action variables refer to the action variables output by the reinforcement learning agent during the training phase, used for state transitions and policy updates in the simulation environment. The actor-critic architecture refers to an algorithmic framework in reinforcement learning that simultaneously maintains a policy network (actor) and a value function network (critic), where the policy network outputs actions, and the critic network evaluates the value of actions and provides gradient directions for the policy network. Policy network parameters refer to the weight matrix and bias vector of the policy network in the actor-critic architecture, determining the mapping relationship from state to action.

[0053] S202. Verify whether the values ​​of each training parameter to be adjusted in the training action variables fall within the safe operating range of the corresponding energy equipment. If they fall within the range, retain them; if they exceed the range, adjust the values ​​of the training parameters to be adjusted to the boundary extreme values ​​of the safe operating range of the equipment, and generate preliminary training clipping actions.

[0054] Specifically, the system verifies whether the values ​​of each training parameter to be adjusted in the training action variables fall within the safe operating range of the corresponding energy equipment. For the set value of chilled water or hot water outlet temperature, it verifies whether it falls between the preset lower and upper limits of the outlet temperature of the equipment. The lower limit is set to 4°C to 7°C according to the evaporator antifreeze protection requirements, and the upper limit is set to 12°C to 15°C according to the rated operating conditions of the equipment. For the water pump operating frequency command, it verifies whether it falls between the preset minimum and maximum operating frequencies of the water pump. The minimum frequency is set to 25Hz to 30Hz according to the motor heat dissipation and bearing lubrication requirements, and the maximum frequency is set to 50Hz. For the unit load rate command, it verifies whether it falls between the preset minimum and maximum load rates of the unit. The minimum load rate is set to 20% to 30% according to the compressor oil return requirements and to avoid inefficient operation, and the maximum load rate is set to 100%. For the energy storage device charging and discharging power command, it verifies whether its absolute value falls within the preset maximum charging and discharging power range of the energy storage device. The maximum charging and discharging power is set according to the heat exchanger capacity and medium circulation pump capacity of the energy storage device. If the current training parameter value falls within the corresponding safe operating range of the equipment, the parameter value is retained without adjustment. If the current training parameter value exceeds the corresponding safe operating range of the equipment, the training parameter value is adjusted to the boundary extreme value of the safe operating range, that is, when the parameter value is lower than the lower limit of the range, it is adjusted to the lower limit value, and when the parameter value is higher than the upper limit of the range, it is adjusted to the upper limit value. After completing the above verification and adjustment operations for all training parameters, all retained or adjusted parameter values ​​are combined into a preliminary training pruning action. The preliminary training pruning action is the action variable after single-device safety boundary verification during the training phase. Its physical meaning is: the individual parameters of any device meet the safe operating requirements of that device, but the system-level supply and demand balance after combining multiple devices has not yet been verified.

[0055] S203. Based on the multi-device coupling mechanism constraint model, calculate the training imbalance between the energy supply and load demand of the energy system during the operation of the preliminary training pruning action; when the training imbalance exceeds the preset training tolerance, according to the order of sensitivity of each energy device to the training imbalance from low to high, the output adjustment amplitude of the corresponding energy device is gradually reduced until the training imbalance falls within the preset training tolerance, and the pruned training action variables are obtained.

[0056] Specifically, based on the multi-device coupling mechanism constraint model, the training imbalance between the energy supply and load demand of the energy system is calculated during the execution of the preliminary training trimming action. The energy supply refers to the sum of the real-time energy production of all online operating capacity equipment and the energy released by the energy storage device after the execution of the preliminary training trimming action, minus the energy consumed by the energy storage device during charging. This value is calculated by substituting the parameters of each device in the preliminary training trimming action (load rate commands for each unit, charging and discharging power commands for the energy storage device, and commands for the number of units started) into the system-level supply and demand balance constraint in the multi-device coupling mechanism constraint model. Load demand refers to the predicted load value in the current simulation state space. The formula for calculating the training imbalance is: Training imbalance = Energy supply - Load demand. A value greater than zero indicates oversupply, and a value less than zero indicates undersupply. When the training imbalance exceeds the preset training tolerance, a system-level supply and demand balance adjustment process is triggered. The preset training tolerance is a pre-defined range of allowable supply-demand deviations. Its value is set based on system size, load fluctuation characteristics, and control accuracy requirements. For systems with a capacity of 5000kW or less, the preset training tolerance is that the absolute value of the difference between the supplied energy and load demand should not exceed 2% to 5% of the total installed capacity. For systems with a capacity of 5000kW or more, the preset training tolerance is that the absolute value of the difference between the supplied energy and load demand should not exceed 1% to 3% of the total installed capacity. For example, for a system with a total installed capacity of 2000kW, the preset training tolerance can be set to an absolute value of no more than 40kW to 100kW; for a system with a total installed capacity of 10000kW, the preset training tolerance can be set to an absolute value of no more than 100kW to 300kW. When the training imbalance exceeds the aforementioned tolerance range, the change in supplied energy caused by the unit output adjustment of the energy equipment under the current operating conditions is used as the sensitivity. Based on the order of sensitivity from low to high, a higher sensitivity indicates a greater impact of the equipment's output adjustment on the system's supplied energy, while a lower sensitivity indicates a smaller impact. The output adjustment amplitude is reduced by a preset step size, and then the training imbalance is recalculated. If the recalculated training imbalance still exceeds the preset training tolerance, the next equipment with the second lowest sensitivity is selected, and its output adjustment amplitude is reduced. This process is repeated until the training imbalance falls within the preset training tolerance, resulting in the trimmed training action variables.

[0057] S204. Using the preset optimal energy efficiency as a reference, construct a potential function for evaluating the evolution of the system state; based on the potential function, calculate the first function value of the simulation state space at the current time and the second function value of the simulation state space at the next time; superimpose the function difference between the first function value and the second function value onto the comprehensive benefit reward function to generate the shaped comprehensive benefit reward function; update the reinforcement learning agent based on the shaped comprehensive benefit reward function, and repeat the iteration until the policy converges to obtain the pre-trained reinforcement learning agent.

[0058] Specifically, the theoretical optimal system energy efficiency ratio (EER) calculated from the energy efficiency mechanism relationships of each device in the multi-device coupling mechanism constraint model under the current operating conditions is taken as the preset optimal energy efficiency, i.e., the highest energy efficiency level that each production capacity device can achieve under given load demand, environmental conditions, and device combination. Based on the energy efficiency mechanism relationships of each device in the multi-device coupling mechanism constraint model, the difference between the actual system EER and the preset optimal energy efficiency under the current system state is calculated. This difference is mapped to a potential function value through a monotonically decreasing function, such that the potential function reaches its highest value when the actual energy efficiency ratio equals the preset optimal energy efficiency; the larger the difference, the lower the potential function value. Based on the potential function, the state observed from the simulation training environment at the current decision moment is calculated. After executing the pruned training action variables in the simulation training environment, the state at the next moment calculated from the state transition relationship in the multi-device coupling mechanism constraint model is used to obtain the simulation state space at the next moment. The simulation state space at the current moment is input into the potential function to obtain the first function value Φ(s). t The simulation state space at the next moment is input into the potential function to obtain the second function value Φ(s). t+1 The difference between the first function value and the second function value is superimposed on the comprehensive benefit return function to generate the shaped comprehensive benefit return function. (η is the shaping coefficient). When the function difference is positive, it means that the current action causes the system state to evolve closer to the preset optimal energy efficiency, and the reward after shaping is higher than the original reward; when the function difference is negative, it means that the current action causes the system state to deviate from the preset optimal energy efficiency, and the reward after shaping is lower than the original reward.

[0059] The reinforcement learning agent is updated based on the shaped comprehensive benefit reward function, and the simulation state space s at the current moment is updated. t The clipped training action variable a t The comprehensive benefit return function value after shaping and the simulation state space s at the next moment t+1 As a complete experience sample, it is stored in the experience pool; batch experience samples are sampled from the experience pool, and the critic network is updated by minimizing the temporal difference error, while the actor network is updated by the policy gradient direction. Qπ (s,a) represents the action value of taking action a in state s under policy π, in order to improve the expected post-shaping comprehensive benefit reward. Repeat the above iterative process until the policy converges, obtaining the pre-trained reinforcement learning agent. Policy convergence means that the loss function values ​​of the actor network and the critic network no longer decrease significantly over a predetermined number of training epochs, or the average comprehensive benefit reward function value no longer increases significantly over a predetermined number of training epochs. At this point, the policy network parameters are fixed at the convergent value.

[0060] Through the above embodiments, a potential function is constructed with a preset optimal energy efficiency as a reference. The difference between the potential function and the simulation state space at the current and next time steps is used as an exploration compensation term superimposed on the original comprehensive benefit reward function. When an action causes the system state to evolve towards the optimal energy efficiency direction, the potential function difference is positive, and the shaped reward is higher than the original reward, providing positive incentives to the agent. When an action causes the system state to deviate from the optimal energy efficiency direction, the potential function difference is negative, and the shaped reward is lower than the original reward, providing negative penalties to the agent. This mechanism accelerates the convergence of the policy towards the optimal direction without changing the optimal policy, effectively solving the problem of low exploration efficiency in continuous action space of reinforcement learning. At the same time, this method allows the agent to obtain guidance signals from physical mechanisms in the early stage of training, reducing the sample size required for random exploration, improving sample efficiency and training speed. After repeated iterations until the policy converges, a pre-trained reinforcement learning agent is obtained, which has the ability to output the action with the optimal comprehensive benefit under different working conditions.

[0061] S105. Using the multi-device coupling mechanism constraint model as a constraint condition, perform feasibility verification on the initial action variables, and trim the action variables that violate the constraint conditions to the feasible region to obtain the trimmed action variables.

[0062] Specifically, the multi-device coupling mechanism constraint model is used as the constraint condition. The initial action variables are then checked against the safe operating range of each device. This involves verifying whether the values ​​of each parameter to be adjusted in the initial action variables fall within the safe operating range of the corresponding energy device. A system-level supply and demand balance verification is then performed. The action variables after the first-level verification are substituted into the system-level supply and demand balance constraints in the multi-device coupling mechanism constraint model to calculate the imbalance between the system's energy supply and load demand. It is then checked whether this imbalance exceeds a preset tolerance. Action variables that violate the constraints are then trimmed to the feasible region, resulting in trimmed action variables. The action variables obtained through these two levels of trimming simultaneously satisfy both the safe operating boundary constraints of each device and the system-level supply and demand balance constraints.

[0063] Based on the above embodiments, as an optional embodiment, for Figure 1Step S105 shown can be achieved through the following steps, which are explained in detail below.

[0064] The system verifies whether the values ​​of each parameter to be adjusted in the initial action variables fall within the safe operating range of the corresponding energy equipment. If they do, they are retained; otherwise, the values ​​are adjusted to the boundary extremes of the safe operating range, generating a preliminary trimming action. Based on the multi-equipment coupling mechanism constraint model, the system calculates the imbalance between the energy supply and load demand of the energy system during operation according to the preliminary trimming action. When the imbalance exceeds the preset tolerance, the system trims the output adjustment range of the corresponding energy equipment according to the order of sensitivity of each energy equipment's adjustment action to the imbalance from low to high, based on the preset trimming step size, until the imbalance falls within the preset tolerance, thus obtaining the trimmed action variables.

[0065] Specifically, verify whether the values ​​of each parameter to be adjusted in the initial action variables fall within the safe operating range of the corresponding energy equipment. For the chilled water or hot water outlet temperature setpoint, verify whether it falls between the preset lower and upper limits of the outlet temperature for the equipment. The lower limit is set to 4℃ to 7℃ according to the evaporator antifreeze protection requirements, and the upper limit is set to 12℃ to 15℃ according to the equipment's rated operating conditions. For the water pump operating frequency command, verify whether it falls between the preset minimum and maximum operating frequencies for the water pump. The minimum frequency is set to 25Hz to 30Hz according to the motor heat dissipation and bearing lubrication requirements, and the maximum frequency is set to 50Hz. For the unit load rate command, verify whether it falls between the preset minimum and maximum load rates for the unit. Between the rates, the minimum load rate is set to 20% to 30% based on the compressor oil return requirements and to avoid inefficient operation, while the maximum load rate is set to 100%. For the energy storage device's charging and discharging power command, its absolute value is checked to see if it falls within the preset maximum charging and discharging power range of the energy storage device. The maximum charging and discharging power is set based on the heat exchanger capacity and medium circulation pump capacity of the energy storage device. If the current parameter value to be adjusted falls within the corresponding safe operating range of the equipment, the parameter value is retained without adjustment. If the current parameter value to be adjusted exceeds the corresponding safe operating range of the equipment, the parameter value to be adjusted is adjusted to the boundary extreme value of the safe operating range of the equipment, that is, when the parameter value is lower than the lower limit of the range, it is adjusted to the lower limit value, and when the parameter value is higher than the upper limit of the range, it is adjusted to the upper limit value. After completing the above verification and adjustment operations for all parameters to be adjusted, all retained or adjusted parameter values ​​are combined into a preliminary trimming action. Based on the multi-device coupling mechanism constraint model, the imbalance between the energy supply and load demand of the energy system during the operation of the preliminary trimming action is calculated in the same way as in step S203. A value greater than zero indicates an oversupply, and a value less than zero indicates an undersupply. When the imbalance exceeds the preset tolerance, a system-level supply and demand balance adjustment process is triggered. The preset tolerance is a pre-defined range of allowable supply and demand deviations. For systems with a capacity of 5000kW or less, the preset tolerance is that the absolute value of the difference between the supplied energy and the load demand does not exceed 2% to 5% of the total installed capacity of the system. For systems with a capacity of 5000kW or more, the preset tolerance is that the absolute value does not exceed 1% to 3% of the total installed capacity of the system. The output adjustment range of the corresponding energy equipment is adjusted according to the order of sensitivity of each energy equipment to the imbalance from low to high, based on a preset trimming step size. Sensitivity refers to the change in supplied energy caused by a unit output adjustment of a certain equipment under the current operating conditions. This value is obtained by taking the partial derivative of the state transition relationship in the multi-equipment coupling mechanism constraint model or by performing numerical difference. The preset trimming step size is the change in equipment output during each adjustment. This value is set according to the equipment type and control precision. The preset trimming step size for load rate is 1% to 5%, the preset trimming step size for frequency is 1Hz to 2Hz, and the preset trimming step size for power is 10kW to 50kW.If the recalculated imbalance still exceeds the preset tolerance, then select the next device with the second lowest sensitivity and reduce its output adjustment range, repeating the above process until the imbalance falls within the preset tolerance, thus obtaining the trimmed motion variables.

[0066] S106. Send the trimmed action variables to the corresponding energy devices for execution, and receive feedback information from each energy device.

[0067] Specifically, for chiller or heat pump units, the chilled water or hot water outlet temperature setpoint and load rate command are sent to the unit controller via fieldbus, while the number of units in operation is sent to the unit group controller. For circulating water pumps, the operating frequency command is sent to each pump inverter via fieldbus. For energy storage devices, the charging or releasing power command is sent to the energy storage device controller via fieldbus, which adjusts the opening of the charging / releasing valves and the frequency of the medium circulation pump. For boilers or plate heat exchangers, the outlet water temperature setpoint and load rate command are sent to the corresponding controller via fieldbus. During the distribution process, after receiving the command, the controllers of each device execute the corresponding adjustment actions: the unit controller adjusts the compressor guide vane opening or speed to adjust the load rate to the target value; the inverter adjusts the output frequency to adjust the water pump speed to the target frequency; the energy storage device controller adjusts the opening of the charging / releasing valves and the circulation pump frequency to achieve the target charging / releasing power. After the distribution is completed, feedback information from each energy device after executing the tailored action variables is received via fieldbus or sensor network. Feedback information is collected and transmitted by sensors and smart meters of each device. It includes at least the actual operating parameters of each device after executing the command, namely the actual values ​​of the chilled or hot water supply and return temperatures, the actual values ​​of the pump operating frequency, the actual values ​​of the unit load rate, the actual values ​​of the energy storage device's charging and discharging power, and the actual number of units in operation. The collection of feedback information is completed within the first sampling period after the command is issued. The collected data, after noise reduction and timestamp alignment, is stored in the system as the execution result of the action variables. Simultaneously, the actual operating parameters of each device and the energy storage device status from the feedback information are used to update the system state space, forming the state for the next moment to support the next round of decision-making.

[0068] Through the above embodiments, the feedback information is used both to verify whether the action variables are executed correctly and as a data source for updating the system state space, enabling the system to perceive the execution effect of the current action and the actual response status of the device. By synchronously updating the feedback information to the system state space, the system can accurately obtain the state at the next moment, thereby entering the next round of iterative optimization and forming a complete closed-loop control link of "decision → execution → perception → re-decision".

[0069] S107. Update the system state space based on the feedback information and proceed to the next iteration for optimization.

[0070] Specifically, the actual operating parameters and boundary conditions of each device in the feedback information are organized according to the feature dimensions and data format of the system state space. The device state portion in the feature vector at the corresponding moment is replaced with the measured values ​​in the feedback information, and the boundary condition portion is replaced with the actual environmental parameters and load data at the current moment. The updated system state space is the system state space for the next moment. Iterative optimization refers to re-executing the steps of outputting action variables, feasibility verification and pruning, issuing execution and receiving feedback through a pre-trained reinforcement learning agent based on the updated system state space, forming a cyclical closed-loop control process. Each iteration corresponds to a control cycle, and the duration of the control cycle is consistent with the sampling period of the system state space. In the iterative optimization of the next moment, the pre-trained reinforcement learning agent uses the updated system state space as input and repeats the aforementioned steps, thereby achieving rolling optimization control from the current moment to the next moment.

[0071] Through the above embodiments, this process enables the system state space to be continuously updated over time, with each moment's state reflecting the most accurate current system operation. Based on this, using the updated system state space as input, the complete closed-loop process of action decision-making, verification and pruning, execution, and feedback reception is re-executed, with each iteration corresponding to a control cycle. Through this feedback-based state update mechanism, the system can perceive changes in its operating state in real time and make optimal decisions based on the latest state in each control cycle. This enables dynamic adaptive adjustment to load fluctuations, environmental changes, and equipment performance degradation, ensuring the continued effectiveness of the optimization strategy in long-term operation.

[0072] Based on the above embodiments, as an optional embodiment, see [link to embodiment]. Figure 2 and Figure 7 , Figure 7 This is a flowchart illustrating the exception feedback provided in an embodiment of this application, specifically for... Figure 1 Before step S107 shown, steps S301-S304 may also be included, which will be explained in detail below.

[0073] S301. Extract multi-source feedback data from each energy device from the feedback information, and calculate the deviation between the expected performance coefficient of the multi-source feedback data and the actual operating performance coefficient based on the energy efficiency mechanism model of the energy device, to obtain the operating deviation value.

[0074] Specifically, a subset of various data related to the operating status of each energy device is extracted from the feedback information, including at least the actual power consumption and actual energy supply of the device under the current operating conditions. The energy efficiency mechanism model calculates the expected performance coefficient (EPC) of the device under the current operating conditions, i.e., the theoretical energy efficiency ratio, based on the device's current actual operating parameters (such as chilled water inlet and outlet temperatures, cooling water inlet and outlet temperatures, and partial load factor). The actual operating performance coefficient is the measured EPC obtained by dividing the actual energy supply from the multi-source feedback data by the actual power consumption. Subtracting the expected performance coefficient from the actual operating performance coefficient yields the operating deviation value. A positive operating deviation value indicates that the actual operating performance is lower than the expected performance, potentially indicating efficiency degradation or performance deterioration; a value close to zero indicates normal device operation; a negative operating deviation value indicates that the actual operating performance is higher than the expected performance, which is usually uncommon, but may indicate sensor calibration errors or a need to update the mechanism model parameters. The unit of the operating deviation value is dimensionless, and its absolute value reflects the degree to which the actual operating state of the device deviates from the normal state.

[0075] S302. When the operating deviation value exceeds the preset abnormal judgment threshold for a consecutive preset number of sampling periods, the corresponding energy equipment is determined to be an abnormal operating equipment, and an abnormal detection alarm signal for the abnormal operating equipment is triggered. The alarm level is determined according to the degree to which the operating deviation value exceeds the abnormal judgment threshold.

[0076] Specifically, for centrifugal chillers, an abnormality is identified when the operating deviation (the difference between the expected coefficient of performance and the actual coefficient of performance) exceeds 0.3 to 0.5. The specific value is determined within this range based on the unit capacity, refrigerant type, and operating age; the longer the equipment has been in operation, the more lenient the threshold. For water pumps, an abnormality is identified when the operating deviation exceeds 5% to 10%. For cooling towers or boilers, the threshold is 3% to 8%. The judgment process is as follows: within the current moment and the preceding (preset number - 1) consecutive sampling periods, if the operating deviation value at each moment exceeds the preset abnormality judgment threshold, the energy equipment is determined to be operating abnormally. When an abnormally operating device is identified, an abnormality detection alarm signal is triggered for that device. The abnormality detection alarm signal is a digital signal or message flag used to notify the system that the device is operating abnormally. Calculate the multiple by which the operating deviation value exceeds the anomaly judgment threshold. This multiple = operating deviation value / anomaly judgment threshold. When the multiple exceeds 1 or more but is less than the first preset multiple, the alarm level is determined to be a prompt level, indicating that the equipment performance is slightly deviating from the normal range. When the multiple exceeds 1 or more but is less than the second preset multiple, the alarm level is determined to be a warning level, indicating that the equipment performance is moderately deviating from the normal range, and attention is recommended. When the multiple exceeds 1 or more but is greater than the second preset multiple, the alarm level is determined to be an alarm level, indicating that the equipment performance is seriously deviating from the normal range, and immediate investigation is recommended.

[0077] The preset sampling period refers to the pre-defined number of consecutive samples to avoid false triggers caused by fluctuations in single data points. This value is set based on the system sampling frequency and the sensitivity requirements for anomaly detection. The preset anomaly judgment threshold is a pre-defined critical point for distinguishing between normal and abnormal equipment operation. Abnormally operating equipment refers to energy equipment whose actual operating performance consistently falls below the expected performance over multiple consecutive sampling periods, thus being judged as deviating from normal operation. Anomaly detection alarm signal is a digital signal or message flag used to notify the system of operational anomalies. Alarm level refers to the severity level of the anomaly based on the degree to which the operating deviation exceeds the anomaly judgment threshold, including at least a warning level, a preliminary warning level, and an alarm level. The warning level indicates a slight deviation from the normal performance range.

[0078] The above embodiments provide maintenance personnel with differentiated handling criteria, avoiding untimely or over-reactions caused by a single alarm method, and improving the system's operation and maintenance efficiency.

[0079] S303. In response to the anomaly detection alarm signal, the abnormal parameter that triggered the anomaly detection alarm signal is taken as the target analysis node; based on the transmission direction of energy flow, information flow and control flow in the multi-device coupling mechanism constraint model, a reverse causal tracing graph with the target analysis node as the search starting point is established.

[0080] Specifically, in response to the anomaly detection alarm signal triggered when the corresponding energy equipment is determined to be malfunctioning, the abnormal parameters that triggered the anomaly detection alarm signal are used as the target analysis node. Figure 7 Anomalies in the graph, such as when the deviation between the actual operating performance coefficient of a unit and the expected performance coefficient triggers an alarm, are considered abnormal parameters, including the actual energy supplied, actual power consumed, or actual operating temperature of the unit corresponding to that deviation. This is also the starting point for causal tracing; all subsequent upward tracing begins at this node. Using the target analysis node as the starting node, the graph traverses the reverse direction of energy flow, information flow, and control flow, sequentially searching for upstream nodes directly connected to the target analysis node. A direct connection means that there is energy flow transmission (e.g., cooling water flowing from a cooling tower to a chiller), information flow transmission (e.g., a temperature sensor transmitting collected data to a controller), or control flow transmission (e.g., the controller sending a load rate command to the unit) between the two nodes. For each upstream node, the tracing continues upwards in the reverse direction until a boundary with no upstream nodes is reached. Connecting all the above nodes according to the tracing relationship forms a directed graph, which is the reverse causal tracing graph.

[0081] S304. Tracing upstream along the connecting edges of the reverse cause-effect tracing graph, using multi-source feedback data from each upstream associated device as associated parameters; calculating the contribution of each associated parameter to the deviation of the target analysis node from the normal state, identifying the node corresponding to the associated parameter whose contribution exceeds the preset contribution threshold as the root cause node, and constructing a physical fault propagation path; generating an interpretable diagnostic report for the abnormally operating device based on the alarm level, the associated parameters of the root cause node, and the physical fault propagation path.

[0082] Specifically, the reverse causal tracing graph is a directed graph structure formed by traversing the energy flow, information flow, and control flow transmission directions in the multi-device coupling mechanism constraint model from the target analysis node. It includes nodes and connecting edges, where nodes represent energy devices or measurement points, and connecting edges represent the transmission relationship of energy flow, information flow, or control flow between two nodes. Tracing upstream along the connecting edges means starting from the target analysis node and traversing level by level in the reverse direction pointed to by the connecting edges, identifying the energy device corresponding to each upstream node as the upstream associated device. Using the multi-source feedback data of each upstream associated device as association parameters means associating and storing the multi-source feedback data of each traced upstream associated device with that device, forming a set of parameters that have a direct or indirect causal relationship with the target analysis node. When the associated parameter is a continuously adjustable operating parameter (such as temperature setpoint, valve opening, frequency command, etc.), the contribution is obtained by substituting the actual value of the associated parameter at the current moment into the energy efficiency mechanism relationship in the multi-device coupling mechanism constraint model and calculating the partial derivative. The change in the parameter corresponding to the target analysis node caused by the unit change of the parameter is the contribution. When the associated parameter is a discrete state parameter (such as start-stop state, mode switching, etc.), the contribution is obtained by comparing the difference between the parameter corresponding to the target analysis node in the normal state and the abnormal state.

[0083] Nodes corresponding to correlation parameters whose contribution exceeds a preset contribution threshold are designated as root cause nodes. The preset contribution threshold is a pre-defined critical value used to screen major influencing factors; its value is set according to system scale, equipment type, and anomaly detection sensitivity requirements, preferably 30% to 50% of the maximum contribution among all correlation parameters, or set as an absolute threshold. A root cause node refers to a node among all upstream correlated devices whose correlation parameter's contribution to the deviation of the target analysis node from its normal state exceeds the preset contribution threshold, representing the location of the most likely root cause of the anomaly. A physical fault propagation path is constructed, which is a complete path from the root cause node back along the connecting edges of the reverse causal tracing graph (i.e., along the forward direction of energy flow, information flow, or control flow) to the target analysis node. Each node on this path represents a device or measurement point, and adjacent nodes are connected through energy flow, information flow, or control flow. Based on alarm levels, root cause node correlation parameters, and physical fault propagation paths, a fault-explainable diagnostic report is generated, containing at least the following information: anomaly description, i.e., the abnormal parameters corresponding to the target analysis node and their specific values ​​deviating from the normal range; alarm level, i.e., the severity level of the anomaly determined based on the degree to which the operational deviation value exceeds the anomaly judgment threshold; root cause location, i.e., the device name and correlation parameters corresponding to the root cause node; physical fault propagation path, i.e., the complete path from the root cause node to the target analysis node along the energy flow, information flow, or control flow; and handling recommendations, i.e., specific handling measures given based on the type of root cause node and the nature of the correlation parameters. The report is output in a structured data format for operation and maintenance personnel to view or for automatic system retrieval.

[0084] For example, assuming the target analysis node is the chilled water outlet temperature measurement point of chiller unit 2, with an operating deviation of 0.52 and an alarm level of warning, the upstream related devices traced in the reverse causal chain diagram include the cooling tower (related parameter is the cooling tower outlet temperature setpoint), the cooling water valve (related parameter is the valve opening command), the controller (related parameter is the load rate command issued value), and the temperature sensor (related parameter is the calibration offset). The contribution of each related parameter is calculated as follows: for every 1°C change in the cooling tower outlet temperature setpoint, the chilled water outlet temperature of unit 2 changes by 0.3°C, with a contribution of 0.3°C / °C; for every 1% change in the cooling water valve opening command, the chilled water outlet temperature changes by 0.05°C, with a contribution of 0.05°C / °C; for every 1% change in the load rate command issued value, the chilled water outlet temperature changes by 0.02°C, with a contribution of 0.02°C / °C; for every 0.1°C change in the temperature sensor calibration offset, the chilled water outlet temperature directly shifts by 0.1°C, with a contribution of 1.0 (dimensionless). The preset contribution threshold is set to 40% of the maximum contribution of 1.0, i.e., 0.4. Therefore, if the contribution of the temperature sensor calibration offset (1.0) exceeds 0.4, it is identified as a root cause node. If the contribution of the cooling tower outlet water temperature setpoint (0.3) is less than 0.4, it is not considered a root cause node. The physical fault propagation path is: temperature sensor calibration offset (root cause node) → transmitted via information flow (sensor data uploaded to the controller) → offset of chilled water outlet temperature value received by Unit 2 controller → deviation in the actual operating performance coefficient calculation of Unit 2 → increase in operating deviation value → triggering an anomaly alarm (target analysis node). A fault-explainable diagnostic report was generated: The abnormal phenomenon is "the operating deviation value of the chilled water outlet temperature measuring point of chiller unit No. 2 is 0.52, which exceeds the abnormal judgment threshold of 0.4", the alarm level is "early warning level", the root cause is "abnormal calibration offset of chilled water outlet temperature sensor of chiller unit No. 2, contribution 1.0", the physical fault propagation path is "temperature sensor calibration offset → controller received temperature value offset → performance coefficient calculation deviation → increase in operating deviation value", and the handling recommendation is "check the calibration parameters of chilled water outlet temperature sensor of chiller unit No. 2, and it is recommended to recalibrate or replace the sensor".

[0085] The above examples clearly illustrate the abnormal phenomena, alarm levels, root cause location, physical fault propagation paths, and handling suggestions. This enables maintenance personnel to quickly understand the causal mechanism of the abnormality, accurately locate the source of the fault, and take targeted handling measures. It avoids the shortcomings of traditional black-box diagnostic methods that only output "equipment abnormality" without providing root causes and handling basis, and significantly improves the efficiency and accuracy of abnormality handling.

[0086] Based on the above embodiments, as an optional embodiment, for Figure 1 The step S107 shown can be implemented as follows, which will be explained in detail below.

[0087] Efficiency degradation features of malfunctioning equipment are extracted from the fault-explainable diagnostic report and mapped to state penalty variables. The state penalty variables and feedback information are synchronously updated to the system state space at the next time step. During the iterative optimization process at the next time step, based on the system state space containing the state penalty variables, the loading priority of malfunctioning equipment in the global unit combination is dynamically reduced through a reinforcement learning agent. Under the premise of satisfying the constraints of the multi-equipment coupling mechanism constraint model, the load-bearing capacity of normal energy equipment is reallocated through a reinforcement learning agent to generate safe and coordinated operation action variables.

[0088] Specifically, efficiency degradation characteristics of malfunctioning equipment are read from the fault-explainable diagnostic report. These characteristics include operating deviation values, the multiple by which the operating deviation value exceeds the anomaly judgment threshold, and the ratio of the actual operating performance coefficient of the malfunctioning equipment to the preset benchmark performance coefficient. The operating deviation value is extracted from these characteristics, and the multiple is obtained by dividing the operating deviation value by the preset anomaly judgment threshold. This multiple is then processed by a preset piecewise linear mapping function. This function outputs 0 when the multiple is less than or equal to 1, linearly increases to 0.5 when the multiple is greater than 1 and less than or equal to 2, linearly increases to 0.8 when the multiple is greater than 2 and less than or equal to 3, and outputs 1 when the multiple is greater than 3. The value output after the mapping function is used as a state penalty variable. This state penalty variable, along with the actual operating parameters of each device obtained from the feedback information, is written into the corresponding position in the system state space at the next moment as a newly added health state dimension. The system state space after this addition includes the original load demand, environmental parameters, equipment operating parameters, and electricity price information dimensions, as well as the health state dimension corresponding to the malfunctioning equipment. After the system state space is formed in the next moment, the complete system state space containing the health state dimension is input into the policy network of the reinforcement learning agent. When the policy network calculates the action output, the value of the health state dimension is used as a feature of the input layer of the policy network to participate in the forward propagation calculation. This value affects the activation value of each action dimension of the output layer through the nonlinear transformation of the hidden layer, so that the load rate command output value of the abnormal device is suppressed, while the load rate command output value of other healthy devices is increased accordingly. In this way, the reinforcement learning agent can dynamically reduce the loading priority of the abnormal device in the global unit combination. After the reinforcement learning agent outputs new initial action variables, the loading rate instructions of each device in the new initial action variables are substituted into the system-level supply and demand balance constraints in the multi-device coupling mechanism constraint model for verification: First, it is calculated whether the total energy supplied by the system equals the load demand when running according to the new initial action variables. If the total energy supplied is less than the load demand, the loading rate of the healthy device with the highest energy efficiency ratio is increased in descending order of energy efficiency ratio of other energy devices (excluding abnormal devices) at the current loading rate, with each increase being a preset increment step, until the total energy supplied meets the load demand; if the total energy supplied is greater than the load demand, the loading rate of the healthy device with the lowest energy efficiency ratio is decreased in ascending order of energy efficiency ratio of other energy devices (excluding abnormal devices) at the current loading rate, with each decrease being a preset decrement step, until the total energy supplied meets the load demand; The loading rate instructions of each device, the charging and discharging power instructions of the energy storage device, and the instructions of the number of units started after the above adjustments to meet the supply and demand balance are combined into safe collaborative operation action variables.

[0089] Through the above embodiments, under the conditions of satisfying system-level supply and demand balance constraints and the safe operating range of each device, the intelligent agent reallocates the load of other healthy devices and generates safe collaborative operation action variables to avoid or limit the operation of abnormal devices. This process realizes the proactive intervention of fault diagnosis results in control strategies, enabling the system to automatically adjust its operating strategies to ensure energy supply security under equipment performance degradation conditions. It forms a complete self-healing closed loop from "anomaly detection → root cause diagnosis → state feedback → adaptive strategy adjustment", improving the energy system's autonomous adaptability and operational safety under equipment performance degradation conditions.

[0090] The following describes the mechanism-integrated multi-objective optimization system for energy systems in the embodiments of this invention from the perspective of hardware processing. Please refer to [link / reference]. Figure 8 , Figure 8 This is a schematic diagram of the structure of a mechanism-integrated multi-objective optimization system for energy systems provided in an embodiment of this application.

[0091] It should be noted that, Figure 8 The structure of the mechanism-integrated energy system multi-objective optimization system shown is merely an example and should not impose any limitations on the functionality and scope of application of the embodiments of this application.

[0092] like Figure 8 As shown, the mechanism-integrated energy system multi-objective optimization system includes a central processing unit 401, which can perform various appropriate actions and processes based on a program stored in a read-only memory 402 or a program loaded from a storage section 408 into a random access memory 403, such as executing the methods described in the above embodiments. The random access memory 403 also stores various programs and data required for system operation. The central processing unit 401, the read-only memory 402, and the random access memory 403 are interconnected via a bus 404. An input / output interface 405 is also connected to the bus 404.

[0093] The following components are connected to the input / output interface 405: an input section 406 including audio input devices, push-button switches, etc.; an output section 407 including a liquid crystal display (LCD) and audio output devices, indicator lights, etc.; a storage section 408 including a hard disk, etc.; and a communication section 409 including a network interface card such as a LAN (Local Area Network) card, modem, etc. The communication section 409 performs communication processing via a network such as the Internet. A drive 410 is also connected to the input / output interface 405 as needed. A removable medium 411, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on the drive 410 as needed so that computer programs read from it can be installed into the storage section 408 as needed.

[0094] In particular, according to embodiments of the present invention, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of the present invention include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing a computer program for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication section 409, and / or installed from removable medium 411. When the computer program is executed by central processing unit 401, it performs the various functions defined in the present invention. It should be noted that specific examples of computer-readable storage media may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), flash memory, optical fiber, portable compact disc read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this invention, a computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0095] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. Each block in a flowchart or block diagram may represent a module, program segment, or portion of code, which contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those shown in the drawings.

[0096] Specifically, the mechanism-integrated energy system multi-objective optimization system of this embodiment includes a processor and a memory. The memory stores a computer program. When the computer program is executed by the processor, it implements the mechanism-integrated energy system multi-objective optimization method provided in the above embodiment.

[0097] In another aspect, the present invention also provides a computer-readable storage medium, which may be included in the mechanism-integrated energy system multi-objective optimization system described in the above embodiments; or it may exist independently and not assembled into the mechanism-integrated energy system multi-objective optimization system. The storage medium carries one or more computer programs, which, when executed by a processor of the mechanism-integrated energy system multi-objective optimization system, cause the mechanism-integrated energy system multi-objective optimization system to implement the mechanism-integrated energy system multi-objective optimization method provided in the above embodiments.

[0098] The above description is merely an exemplary embodiment of the present invention and should not be construed as limiting the scope of the present invention. Any equivalent changes and modifications made in accordance with the teachings of this invention are still within the scope of this invention. This application aims to cover any variations, uses, or adaptations disclosed in this invention, which follow the general principles of the present invention and include common knowledge or customary techniques in the art not described in this invention. The specification and embodiments are considered exemplary only, and the scope and spirit of the present invention are defined by the claims.

Claims

1. A multi-objective optimization method for energy systems that integrates mechanisms, characterized in that, include: Acquire multi-source operation data and time-of-use electricity price information for various energy devices within the energy system; Based on the multi-source operating data, an energy efficiency mechanism model of the energy equipment is constructed, and based on each energy efficiency mechanism model and the energy flow, information flow and control flow between each energy equipment, a multi-equipment coupling mechanism constraint model is constructed. The multi-equipment coupling mechanism constraint model includes system-level supply and demand balance constraints, energy efficiency mechanism relationships between each energy equipment, equipment safe operating range and state transition relationships. Based on the time-of-use electricity price information and the equipment coordination parameters of the energy system, a comprehensive benefit return function including energy efficiency, cost, and equipment items is constructed. Based on the system state space of the energy system at the current moment, the initial action variables of the energy system are output by a pre-trained reinforcement learning agent with the objective of maximizing the function value of the comprehensive benefit reward function; Using the multi-device coupling mechanism constraint model as a constraint condition, the feasibility of the initial action variables is verified, and the action variables that violate the constraint condition are clipped to the feasible region to obtain the clipped action variables. The trimmed action variables are sent to the corresponding energy devices for execution, and feedback information is received from each of the energy devices. The system state space is updated based on the feedback information, and the iterative optimization proceeds to the next time step.

2. The method according to claim 1, characterized in that, Based on the time-of-use electricity price information and the equipment coordination parameters of the energy system, a comprehensive benefit return function is constructed, including energy efficiency, cost, and equipment items, comprising: Based on the equipment coordination parameters of the energy system, the ratio of the system's energy supply to its total power consumption is used as the energy efficiency item. Based on the time-of-use electricity price information, the real-time electricity price corresponding to the current moment is determined, and the product of the real-time electricity price and the total power consumption is subtracted from the demand response revenue at the current moment to obtain the cost item; Based on the device coordination parameters, the total number of start-stop cycles for each energy device within a preset time window is counted, and based on the preset single start-stop loss coefficient and the total number of start-stop cycles, the total start-stop loss value of the energy system is calculated. Based on the actual loading rate of each energy device and the preset high-efficiency loading rate range, the total efficiency deviation loss value of the energy system is calculated. Equipment items are constructed based on the total start-stop loss value and the total efficiency deviation loss value; Calculate the first product of the energy efficiency item and the preset first weight, the second product of the cost item and the preset second weight, and the third product of the equipment item and the preset third weight; Subtracting the second and third products from the first product yields the comprehensive benefit return function.

3. The method according to claim 1, characterized in that, Before outputting the initial action variables of the energy system based on the system state space of the energy system at the current moment through a pre-trained reinforcement learning agent with the objective of maximizing the function value of the comprehensive benefit reward function, the method further includes: The multi-device coupling mechanism constraint model is used as a simulation training environment. The reinforcement learning agent outputs training action variables based on the simulation state space at the current moment with the objective of maximizing the function value of the comprehensive benefit reward function. Verify whether the values ​​of each training parameter to be adjusted in the training action variables fall within the safe operating range of the corresponding energy equipment. If they fall within the range, retain them; if they exceed the range, adjust the values ​​of the training parameters to be adjusted to the boundary extreme values ​​of the safe operating range of the equipment, and generate preliminary training clipping actions. Based on the multi-device coupling mechanism constraint model, the training imbalance between the energy supply and load demand of the energy system is calculated when the preliminary training pruning action is performed. When the training imbalance exceeds the preset training tolerance, the output adjustment range of the corresponding energy device is gradually reduced in order of increasing sensitivity of the adjustment action of each energy device to the training imbalance until the training imbalance falls within the preset training tolerance, thus obtaining the clipped training action variable. Using the preset optimal energy efficiency as a reference, a potential function is constructed to evaluate the evolution of the system state; Based on the potential function, calculate the first function value of the simulation state space at the current time and the second function value of the simulation state space at the next time. The difference between the first function value and the second function value is superimposed on the comprehensive benefit return function to generate the shaped comprehensive benefit return function. The reinforcement learning agent is updated based on the shaped comprehensive benefit reward function, and the iteration is repeated until the policy converges to obtain the pre-trained reinforcement learning agent.

4. The method according to claim 1, characterized in that, Based on the current state space of the energy system, the pre-trained reinforcement learning agent outputs the initial action variables of the energy system with the objective of maximizing the function value of the comprehensive benefit reward function, including: Based on the time-of-use electricity price information, identify the electricity price period at the current moment; When the electricity price period is a low-price off-peak period, the first coordinated adjustment parameter is output with the goal of maximizing the function value of the comprehensive benefit return function. The first coordinated adjustment parameter is used as the initial action variable of the energy system. The first coordinated adjustment parameter is used to increase the loading rate of the production equipment, so that the production equipment can increase its output on the basis of meeting the current load demand. The additional production capacity after the increased output is input to the energy storage device to execute the charging command. When the electricity price period is a high-price peak electricity period, with the goal of maximizing the function value of the comprehensive benefit return function and minimizing the cost item and the equipment item, a second coordinated adjustment parameter is output, and the second coordinated adjustment parameter is used as the initial action variable. The second coordinated adjustment parameter is used to drive the energy storage device to execute an energy release command to replace part of the real-time capacity of the capacity equipment and reduce the loading rate of the capacity equipment. During the issuance of the energy release command, the loading rate of the capacity equipment that has not been replaced is constrained within a preset high-efficiency loading rate range.

5. The method according to claim 1, characterized in that, The step of using the multi-device coupling mechanism constraint model as a constraint condition to perform feasibility verification on the initial action variables, and pruning action variables that violate the constraint conditions to the feasible region to obtain the pruned action variables includes: Verify whether the values ​​of each parameter to be adjusted in the initial action variables fall within the safe operating range of the corresponding energy equipment. If they do, retain them; if they do not, adjust the values ​​of the parameters to be adjusted to the boundary extreme values ​​of the safe operating range of the equipment, and generate a preliminary trimming action. Based on the multi-device coupling mechanism constraint model, the imbalance between the energy supply and load demand of the energy system is calculated when the initial trimming action is performed. When the imbalance exceeds the preset tolerance, the output adjustment range of the corresponding energy device is trimmed according to the order of sensitivity of the adjustment action of each energy device to the imbalance from low to high, based on the preset trimming step size, until the imbalance falls within the preset tolerance, and the trimmed action variable is obtained.

6. The method according to claim 1, characterized in that, Before updating the system state space based on the feedback information and entering the iterative optimization at the next time step, the method further includes: Extract multi-source feedback data from each of the energy devices from the feedback information, and calculate the deviation between the expected performance coefficient of the multi-source feedback data and the actual operating performance coefficient based on the energy efficiency mechanism model of the energy devices to obtain the operating deviation value. When the operating deviation value exceeds the preset anomaly judgment threshold for a consecutive preset number of sampling periods, the corresponding energy equipment is determined to be an operating abnormal equipment, and an anomaly detection alarm signal is triggered for the operating abnormal equipment. The alarm level is determined according to the degree to which the operating deviation value exceeds the anomaly judgment threshold. In response to the anomaly detection alarm signal, the abnormal parameter that triggered the anomaly detection alarm signal is taken as the target analysis node; Based on the transmission directions of energy flow, information flow and control flow in the multi-device coupling mechanism constraint model, a reverse causal tracing graph is established with the target analysis node as the search starting point. Tracing upstream along the connecting edges of the reverse causal tracing graph, the multi-source feedback data of each upstream associated device is used as the association parameter; Calculate the contribution of each of the associated parameters to the deviation of the target analysis node from the normal state, take the node corresponding to the associated parameter whose contribution exceeds the preset contribution threshold as the root cause node, and construct the physical fault propagation path; Based on the alarm level, the association parameters of the root cause node, and the physical fault propagation path, an interpretable diagnostic report of the malfunctioning equipment is generated.

7. The method according to claim 6, characterized in that, The step of updating the system state space based on the feedback information and entering the next iteration for optimization includes: Extract the efficiency degradation feature of the abnormally operating equipment from the fault interpretable diagnostic report, and map the efficiency degradation feature to a state penalty variable; The state penalty variable and the feedback information are synchronized and updated to the system state space at the next time step; During the iterative optimization process at the next moment, based on the system state space containing the state penalty variable, the reinforcement learning agent dynamically reduces the loading priority of the malfunctioning device in the global unit combination; Under the premise of satisfying the constraints of the multi-device coupling mechanism constraint model, the load-bearing capacity of normal energy equipment is redistributed through the reinforcement learning agent to generate safe and coordinated operation action variables.

8. A multi-objective optimization system for energy systems that integrates mechanisms, characterized in that, The mechanism-integrated energy system multi-objective optimization system includes: one or more processors and a memory; the memory is coupled to the one or more processors, the memory is used to store computer program code, the computer program code includes computer instructions, and the one or more processors invoke the computer instructions to cause the mechanism-integrated energy system multi-objective optimization system to perform the method as described in any one of claims 1-7.

9. A computer-readable storage medium comprising instructions, characterized in that, When the instruction is executed on the mechanism-integrated energy system multi-objective optimization system, the mechanism-integrated energy system multi-objective optimization system performs the method as described in any one of claims 1-7.

10. A computer program product, characterized in that, When the computer program product is run on the mechanism-integrated energy system multi-objective optimization system, the mechanism-integrated energy system multi-objective optimization system performs the method as described in any one of claims 1-7.

Images