A method for collaborative control of a multi-zone heating, ventilation, and air conditioning system of an offshore platform

Abstract

The present application relates to the technical field of ocean platform HVAC system control, and provides a kind of ocean platform multi-zone HVAC system collaborative control method, comprising the following steps: S1: decoupling carbon dioxide concentration and temperature control, respectively mapping as each region air supply and uniform fresh air ratio adjustment, and setting constraint condition and optimization target;S2: acquisition each region temperature, indoor carbon dioxide concentration and outdoor environmental parameter;S3: construct the main controller based on multi-agent double-delay deep deterministic policy gradient algorithm, carry out joint decision to multi-zone air supply;S4: construct the auxiliary controller based on active disturbance rejection control, carry out robust regulation to fresh air ratio;S5: establish the collaborative communication mechanism of main controller and auxiliary controller.The present application can realize HVAC system energy consumption optimization under meeting each region temperature and carbon dioxide concentration constraint condition, improve system online control ability.

Description

Technical Field

[0001] This invention relates to a method for coordinated control of multi-zone HVAC systems on offshore platforms, belonging to the field of offshore platform environmental control technology. Background Technology

[0002] Offshore platforms serve as crucial infrastructure for offshore oil and gas exploration, wind power maintenance, and scientific research operations. Their HVAC systems play a vital role in maintaining suitable temperatures, ensuring air quality, and safeguarding the health and safety of personnel. However, due to the enclosed nature of offshore platforms, the significant variations in personnel density depending on operational tasks, the highly unstable external marine climate conditions, and limited energy supply, HVAC systems often exhibit strong coupling, multiple disturbances, and significant energy-saving requirements during operation. Therefore, achieving efficient, stable, and energy-saving control of multi-zone HVAC systems in complex marine environments has become an urgent technical challenge.

[0003] Existing HVAC system control methods mainly fall into three categories: rule-based control, model-based control, and reinforcement learning control. Rule-based control typically employs fixed airflow, fixed setpoints, and threshold-based switching logic. While simple in structure and easy to implement, this method cannot respond to real-time load and disturbances, making it ill-suited for scenarios like offshore platforms with significant load fluctuations and complex external environments. This often leads to temperature control lag, insufficient or excessive ventilation, resulting in decreased comfort or energy waste. Model-based control constructs a model reflecting the dynamic characteristics of the HVAC system and its environment to describe the impact of different control variables on the indoor state, thereby determining appropriate adjustment methods to achieve the desired indoor environment. This method utilizes the model to structurally reflect the dynamic relationships and variable couplings of the system, achieving a more systematic control design than empirical adjustment. However, model-based control is highly dependent on model accuracy. When the model cannot accurately describe the real dynamics in complex scenarios such as offshore platforms, the control strategy deviates from expectations, leading to decreased control performance.

[0004] In recent years, reinforcement learning has attracted attention in the field of HVAC system control due to its characteristics such as not relying on precise system models and being able to autonomously learn control strategies through interaction with the environment. However, in the control scenario of multi-zone HVAC systems on offshore platforms, existing reinforcement learning methods mostly use a single agent to uniformly control the entire system. As the number of zones increases, the dimensions of the state space and action space expand rapidly, which can easily lead to unstable training processes, slow convergence speeds, and insufficient generalization ability of control strategies. At the same time, multi-zone HVAC systems on offshore platforms typically need to consider multiple objectives such as temperature control, carbon dioxide concentration, and energy consumption. When single-agent reinforcement learning adopts a unified global reward structure, it is difficult to balance the differentiated needs of each zone with the overall system optimization requirements.

[0005] Furthermore, in existing multi-zone HVAC control methods, there are generally strong mutual influences between different control variables, and there is also a certain degree of coupling between the states of different zones. For multi-source disturbances caused by factors such as fluctuations in the number of people, infiltration of the external environment, model uncertainty, and changes in ventilation conditions, existing technologies usually lack a unified and effective real-time coordination and compensation mechanism, making it difficult for the system to simultaneously take into account the environmental index constraints of each zone and the overall operating energy consumption requirements under complex operating conditions.

[0006] Therefore, existing technologies still have problems in multi-zone HVAC systems on offshore platforms, such as significant mutual influence of control variables, easy degradation of control performance under complex disturbances, and insufficient adaptability to multi-zone scenarios. Summary of the Invention

[0007] This invention addresses the challenges of multi-zone HVAC systems on offshore platforms, including numerous zones, frequent personnel changes, strong coupling between ventilation and heat loads, and significant external environmental disturbances. It proposes a collaborative control method for multi-zone HVAC systems on offshore platforms. This method comprehensively improves system operational stability and energy efficiency while meeting temperature and indoor carbon dioxide constraints in each zone. It achieves unified and effective real-time coordination and compensation, simultaneously considering environmental constraints in each zone and overall energy consumption requirements under complex operating conditions.

[0008] The present invention provides a method for coordinated control of multi-zone HVAC systems on marine platforms, comprising the following steps.

[0009] Step S1: Decouple carbon dioxide concentration and temperature control, map them to the air supply volume of each area and the unified fresh air ratio adjustment respectively, and set constraints and optimization objectives;

[0010] The specific process of this step includes:

[0011] Step S11: Decouple temperature control from indoor carbon dioxide concentration control;

[0012] The control targets for indoor temperature and carbon dioxide concentration in each area are processed in a hierarchical manner: the temperature regulation of each area is mapped to the air supply volume control of each area, which is completed by the main controller based on the TwinDelayed Deep Deterministic Policy Gradient (MATD3) algorithm; the carbon dioxide concentration regulation is mapped to the unified fresh air ratio control of the HVAC system, which is completed by the auxiliary controller based on Active Disturbance Rejection Control (ADRC).

[0013] Step S12: Establish indoor temperature and carbon dioxide concentration constraints for each area;

[0014] Suppose the offshore platform comprises N regions, and the indoor temperature in region i is... Set constraints:

[0015] ;

[0016] in, and These are the minimum and maximum allowable temperatures for normal operation of zone i, respectively. With the air handling unit supply air temperature setpoint fixed, the indoor temperature of each zone is adjusted by regulating the corresponding zone's supply air volume. accomplish;

[0017] To ensure regional air quality, carbon dioxide concentrations in each area are monitored. Set constraints:

[0018] ;

[0019] in, To set the upper limit of permissible carbon dioxide concentration in zone i, the fresh air ratio is adjusted. The carbon dioxide concentration in each area is adjusted to meet the corresponding air quality requirements.

[0020] Step S13: Establish an energy consumption model and optimization objectives for the HVAC system;

[0021] Energy consumption of cooling coils in HVAC systems Simultaneously affected by the air supply volume and fresh air ratio in each area, it can be expressed as:

[0022] ;

[0023] in, The specific heat capacity of air, The efficiency coefficient of the cooling coil coil. Temperature of adjacent areas Supply temperature setpoints to the air handling unit;

[0024] Energy consumption of fans in HVAC systems Represented as:

[0025] ;

[0026] in, The rated power of the fan under reference operating conditions. It is the reference total air supply volume of the fan under rated operating conditions;

[0027] The total energy consumption of the HVAC system is:

[0028] ;

[0029] The control optimization objective is to reduce the total energy consumption of the HVAC system while satisfying temperature and carbon dioxide concentration constraints. The optimization objective function is established as follows:

[0030] ;

[0031] Where T is the optimization time domain length, the objective function is used to characterize the energy consumption optimization objective of the HVAC system under the conditions of satisfying temperature and indoor carbon dioxide constraints.

[0032] Step S2: Collect temperature, indoor carbon dioxide concentration and outdoor environmental parameters for each area. The temperature and outdoor environmental parameters for each area are used as part of the state input of the main controller, and the maximum value of indoor carbon dioxide concentration in each area is used as the feedback quantity of the auxiliary controller.

[0033] Step S3: Construct a master controller based on the Twin Delayed Deep Deterministic Policy Gradient (MATD3) algorithm to realize joint decision-making on air supply volume in multiple regions;

[0034] The specific process of this step includes:

[0035] Step S31: Establish a multi-agent airflow control model;

[0036] Each region is treated as a relatively independent control unit, and a reinforcement learning agent is assigned to each region, forming a multi-agent ensemble:

[0037] ;

[0038] in, Only responsible for control decisions in region i;

[0039] Step S32: Construct the state space, action space, and reward function for each agent;

[0040] Define the local observations of agent i in region i at discrete time t as:

[0041] ;

[0042] in, , , Let represent the outdoor ambient temperature at time t, the indoor temperature in region i, and the indoor temperature in region z, respectively. Let M represent the set of all other regions on the offshore platform except region i. Indicates the power of the electrical load. d represents the number of people.t The fresh air ratio is calculated and output by the auxiliary controller. This indicates the total energy consumption of the HVAC system;

[0043] The global state is obtained by stitching together the local observations of all agents:

[0044] ;

[0045] Z i The action at time t is defined as the air supply volume for that area:

[0046] ;

[0047] The joint action is as follows:

[0048] ;

[0049] The reward for each agent in each region at time t is defined as:

[0050] ;

[0051] in, , , The weights are set to balance the penalties for temperature deviation, energy consumption, and motion change, respectively.

[0052] Step S33: The training and parameter update process of the master controller based on the Twin Delayed Deep Deterministic Policy Gradient (MATD3) algorithm;

[0053] The reinforcement learning algorithm used is a multi-agent, dual-delay deep deterministic policy gradient algorithm. During the execution phase, each agent in a region outputs its own air supply control action based solely on its current state. During the training phase, a centralized training and decentralized execution framework is adopted. The air supply decisions of each region are evaluated through global state and joint actions, and the training stability is improved through delayed policy updates and a dual-value network mechanism. A policy network and a dual-value network are constructed for each agent in a region, and the training process is completed using a target network and a delayed update mechanism.

[0054] The training process is completed using a target network and a delayed update mechanism, and the specific steps are as follows:

[0055] (1) Initialize the value network for each regional agent. and policy network and initialize the corresponding target network. and ;

[0056] (2) Initialize the experience recycling pool It is used to store joint experience samples generated during the interaction of multiple agents;

[0057] (3) Collect current local observations of each region. i,t And the local observations of all agents are spliced ​​together to form the global state S. t Let the combined action be A. t ;

[0058] (4) Each regional agent selects an action with detection noise based on the action output by its policy network. Combine the actions of each agent into a joint action A. t It acts on the environment, and after the environment executes the action, it returns an immediate reward r to each agent. i,t Local observation at the next moment i,t+1 and the termination mark D t To further construct the global state S at the next time step t+1 ;

[0059] (5) Store transition tuples In the experience recycling pool Among them, its and sample small batches of samples from them. ;

[0060] (6) Based on the target policy network of each region's agents, obtain the corresponding target actions and add pruned, smoothed noise: Combine all target actions into a target joint action. ;

[0061] (7) Utilize reward r i,t The global state S at the next moment t+1 and joint action of the target Calculate the time-series difference objective value , ;

[0062] (8) Update the two online value network parameters θ of each regional agent. i,1 and θ i,2 Minimize the loss function ;

[0063] (9) If the delayed update condition is met, update the policy network of each regional agent, and then softly update the target network parameters: , ;

[0064] (10) Repeat (4)-(9) until the termination condition is met.

[0065] Step S4: Construct an auxiliary controller based on Active Disturbance Rejection Control (ADRC) to achieve robust adjustment of fresh air supply;

[0066] The specific process of this step includes:

[0067] Step S41: Model the control target and controlled object of the auxiliary controller;

[0068] The fresh air ratio is used as a separate auxiliary control variable and adjusted online by the auxiliary controller.

[0069] The maximum carbon dioxide concentration in each region is selected as the feedback value for the auxiliary controller, i.e.:

[0070] ;

[0071] in, This represents the indoor carbon dioxide concentration in region i at time t; the goal of the auxiliary controller is to adjust the fresh air ratio u=d t This ensures that the output y(t) stably tracks the given reference value r, thereby meeting the indoor air quality control requirements;

[0072] The input-output relationship between the fresh air ratio and the indoor carbon dioxide concentration can be abstracted into a first-order uncertain system:

[0073] ;

[0074] Where x represents the system state variable, y represents the system output, i.e., the indoor carbon dioxide concentration, and u represents the control input, i.e., the fresh air ratio d. t f(x,t,u) represents a comprehensive term that includes both internal system uncertainties and external disturbances; b is the system input gain.

[0075] Step S42: Define the total system disturbance and construct the extended state model;

[0076] Let b0 be an estimate of the input gain b, and let system uncertainties, parameter mismatches, and other disturbances be uniformly grouped into the total system disturbance, defined as:

[0077] ;

[0078] The original system is then rewritten as follows:

[0079] ;

[0080] Further introduce expansion state Construct an extended state model:

[0081] ;

[0082] Where x1 represents the system output state and x2 represents the system total disturbance state;

[0083] Step S43: Design of the linearly extended state observer;

[0084] To estimate the system state and total disturbance in real time, a linear extended state observer is introduced, which takes the following form:

[0085] ;

[0086] Where z1 is the estimate of the system state x1, z2 is the estimate of the total system disturbance x2, and β1 and β2 are the observer gain parameters, which are tuned using the bandwidth method.

[0087] ;

[0088] Where ω0 is the observer bandwidth, used to adjust the response speed and accuracy of the disturbance estimation;

[0089] S44: Design of disturbance compensation control law and linear error feedback control law;

[0090] Based on the online estimation results of the total disturbance by the linear extended state observer, a disturbance compensation control law is constructed:

[0091] ;

[0092] Where u0 is the nominal control input and z2 is the observer's estimate of the total disturbance; substituting this control law into the system, we get:

[0093] ;

[0094] For this approximate object, a linear error feedback control law is further adopted:

[0095] ;

[0096] Where, ω c This provides the controller bandwidth; thus, the auxiliary controller can adjust the fresh air ratio in real time based on the output error without relying on an accurate model, thereby achieving rapid tracking and stable control of indoor carbon dioxide concentration.

[0097] Step S5: Establish a collaborative communication mechanism between the main controller and the auxiliary controller.

[0098] During the real-time operation of the HVAC system, the auxiliary controller calculates the system's fresh air ratio d based on the indoor carbon dioxide concentration feedback signal. tThe fresh air ratio is used as one of the environmental state inputs of the main controller to participate in the joint decision-making of the air supply volume of each zone. The joint action of the air supply volume of each zone output by the main controller acts on the multi-zone HVAC system and is fed back to the auxiliary controller through system state changes. The main controller and the auxiliary controller form an information interaction relationship under a unified closed-loop framework. Among them, the main controller is responsible for the multi-zone air supply volume decision and energy consumption optimization under temperature constraints, while the auxiliary controller is responsible for the real-time adjustment of the fresh air ratio under indoor carbon dioxide concentration constraints, so as to jointly achieve energy-saving control of the multi-zone HVAC system under the premise of meeting temperature and indoor carbon dioxide concentration constraints.

[0099] This invention addresses the multi-zone HVAC system of offshore platforms, proposing an optimized control method based on a combination of multi-agent reinforcement learning and active disturbance rejection control. The following key technologies are proposed for the optimized control of multi-zone HVAC systems on offshore platforms:

[0100] 1. This invention proposes a master-slave collaborative control architecture consisting of a master controller and an auxiliary controller. The supply air volume control and fresh air ratio control in a multi-zone HVAC system are designed in a hierarchical manner. The master controller is responsible for the joint optimization decision of the supply air volume in each zone, while the auxiliary controller is responsible for the unified online adjustment of the fresh air ratio. This architecture breaks through the existing method of incorporating supply air volume and fresh air ratio into a single controller for unified solution, thus improving the feasibility of online optimization control of multi-zone HVAC systems on offshore platforms.

[0101] 2. This invention employs a multi-agent reinforcement learning method to jointly decide on the air supply volume for multiple regions. Each region is assigned a relatively independent agent, and multi-region air supply volume control is achieved through collaborative learning. Compared to traditional control methods that rely on precise physical models, this invention reduces the dependence on high-precision mechanistic models, enabling each controller to autonomously adjust its air supply volume control strategy based on load fluctuations, personnel changes, and environmental disturbances in its respective region.

[0102] 3. This invention designs an auxiliary controller with indoor carbon dioxide concentration as feedback, unifying the coupled effects of personnel fluctuations, external infiltration, model uncertainty, and changes in supply air volume into a single system disturbance, and performing online estimation and real-time compensation through an extended state observer. This method overcomes the problem of difficulty in uniformly handling multiple disturbances in existing carbon dioxide concentration control, and solves the problem of insufficient stability in fresh air ratio adjustment under model uncertainty and operational disturbance conditions.

[0103] 4. This invention establishes a two-way information interaction mechanism between the main controller and the auxiliary controller. The fresh air ratio output by the auxiliary controller serves as part of the input to the main controller's state and participates in the air supply volume decision. The change in air supply volume output by the main controller serves as an important object for disturbance observation and compensation in the auxiliary controller. This mechanism improves upon the problems of information fragmentation and untimely feedback of coupling effects in existing discrete control methods.

[0104] The aforementioned key technologies are not only applicable to multi-zone HVAC systems on offshore platforms, but can also be extended to ship area environmental control and other HVAC systems with multi-zone and multi-variable coupling characteristics through parameter configuration adjustments, thus exhibiting good engineering applicability.

[0105] This invention addresses the problems of strong multi-zone coupling, complex control objects, and significant operational disturbances in multi-zone HVAC systems on offshore platforms. It proposes an optimized control method that balances temperature regulation, indoor carbon dioxide concentration control, and energy-saving requirements, achieving more coordinated and stable control results than existing technologies, and has significant engineering application value. Compared with existing technologies, this invention has the following advantages:

[0106] 1. By handling multi-zone air supply volume control and unified fresh air ratio adjustment in a hierarchical manner, the high-dimensionality, strong coupling, and solution difficulties caused by the existing method of unifying multiple control variables into a single controller for unified solution are avoided. This provides a hierarchical collaborative control scheme that is more suitable for engineering applications for multi-zone HVAC systems on offshore platforms.

[0107] 2. In view of the characteristics of significant changes in the number of people in the area, equipment load fluctuations, and environmental conditions during the operation of offshore platforms, this invention can comprehensively consider information such as outdoor environmental parameters, environmental conditions in each area, and energy consumption of the HVAC system to make joint optimization decisions on the air supply volume in each area. This reduces the reliance on high-precision mechanism models and empirical parameter tuning, and improves the problem of insufficient adaptability of existing control methods under complex operating conditions.

[0108] 3. This invention can perform online estimation and real-time compensation for model uncertainties and operational disturbances, solving the problems of difficulty in uniformly handling multiple types of disturbances and insufficient stability of fresh air ratio adjustment under complex operating conditions in existing carbon dioxide concentration control.

[0109] 4. By establishing an information interaction mechanism between the main controller and the auxiliary controller, the changes in fresh air ratio and air supply volume can be coordinated and adjusted within a unified closed-loop framework. This helps to improve the problem of untimely feedback of coupled information in existing discrete control methods, and optimizes the energy consumption of the HVAC system under the premise of meeting the constraints of regional temperature and indoor carbon dioxide concentration. Attached Figure Description

[0110] Figure 1This is a flowchart of the collaborative control method for multi-zone HVAC systems on marine platforms according to the present invention.

[0111] Figure 2 This is a schematic diagram illustrating the structural principle of a multi-zone HVAC system for an offshore platform.

[0112] Figure 3 This is the architecture diagram of the master-slave collaborative control in this invention. Detailed Implementation

[0113] This invention addresses the challenges of multi-zone HVAC systems on offshore platforms, including numerous zones, frequent personnel changes, strong coupling between ventilation and heat loads between zones, and significant external environmental disturbances. It proposes an optimized control method for multi-zone HVAC systems on offshore platforms based on active disturbance rejection control (ADRC) and multi-agent reinforcement learning collaboration. This method employs a collaborative control structure with a main controller and an auxiliary controller. The main controller, MATD3, is used for joint optimization decisions on the air supply volume of each zone; the auxiliary controller, ADRC, is used for online adjustment of the fresh air ratio of the air handling units. By mapping temperature control and indoor carbon dioxide concentration control to the air supply volume control loop and the fresh air ratio control loop, respectively, hierarchical processing of temperature control and carbon dioxide concentration adjustment is achieved, and collaborative operation is completed within a unified closed-loop framework. This optimizes the energy consumption of the HVAC system while meeting the constraints of multi-zone temperature and indoor carbon dioxide concentration.

[0114] The flowchart of the collaborative control method for multi-zone HVAC systems on offshore platforms of the present invention is as follows: Figure 1 As shown. The following is combined with... Figure 1 as well as Figure 2 The given structure of a multi-zone HVAC system for an offshore platform provides a detailed explanation of the process steps of this invention.

[0115] Step S1: Decouple carbon dioxide concentration and temperature control, map them to the air supply volume of each area and the unified fresh air ratio adjustment respectively, and set constraints and optimization objectives;

[0116] Step S11: Decouple temperature control from indoor carbon dioxide concentration control;

[0117] In multi-zone HVAC systems on offshore platforms, both indoor temperature and carbon dioxide concentration are simultaneously affected by the supply air volume and the fresh air ratio, exhibiting significant coupling. To reduce the control complexity caused by this coupling, this invention employs a hierarchical approach to the two control objectives: indoor temperature regulation is primarily mapped to supply air volume control in each zone, handled by the main controller MATD3; carbon dioxide concentration regulation is primarily mapped to a unified fresh air ratio control, handled by the auxiliary controller ADRC.

[0118] Step S12: Establish indoor temperature and carbon dioxide concentration constraints for each area;

[0119] Suppose the offshore platform comprises N regions, and the indoor temperature in region i is... Set constraints:

[0120] ;

[0121] in, and These are the minimum and maximum allowable temperatures for normal operation in zone i, respectively. With the air handling unit supply air temperature setpoint fixed, the indoor temperature of each zone is adjusted by regulating the corresponding zone's supply air volume. Therefore, the aforementioned temperature constraints are mainly achieved through multi-agent reinforcement learning by adjusting the air volume in each region. To satisfy.

[0122] To ensure regional air quality, carbon dioxide concentrations in each area are monitored. Set constraints:

[0123] ;

[0124] in, To set the upper limit of permissible carbon dioxide concentration in zone i, the fresh air ratio is adjusted. The carbon dioxide concentration in each area is adjusted to meet the corresponding air quality requirements.

[0125] Step S13: Establish an energy consumption model and optimization objectives for the HVAC system;

[0126] Energy consumption of cooling coils in HVAC systems It is also affected by the air supply volume and fresh air ratio in each area. This is expressed as:

[0127] ;

[0128] in, The specific heat capacity of air, The efficiency coefficient of the cooling coil coil. Temperature of adjacent areas Supply temperature setpoints to the air handling unit.

[0129] Fan energy consumption Represented as:

[0130] ;

[0131] in, The rated power of the fan under reference operating conditions. It is the reference total air supply volume of the fan under rated operating conditions. As can be seen from this formula, the fan energy consumption is cubically related to the total air supply volume. Therefore, the air supply volume decision of multiple regional agents has significant coupling characteristics at the energy consumption level.

[0132] The total energy consumption of the HVAC system is:

[0133] ;

[0134] The control optimization objective is to reduce the total energy consumption of the HVAC system while satisfying temperature and carbon dioxide concentration constraints. The optimization objective function is established as follows:

[0135] ;

[0136] Where T is the optimization time domain length, the objective function is used to characterize the energy consumption optimization objective of the HVAC system under the conditions of satisfying temperature and indoor carbon dioxide constraints.

[0137] Step S2: Collect temperature, indoor carbon dioxide concentration and outdoor environmental parameters for each area. The temperature and outdoor environmental parameters for each area are used as part of the state input of the main controller, and the maximum value of indoor carbon dioxide concentration in each area is used as the feedback quantity of the auxiliary controller.

[0138] Step S3: Construct a main controller based on MATD3 to realize joint decision-making on air supply volume in multiple zones;

[0139] Step S31: Establish a multi-agent airflow control model;

[0140] Each region is treated as a relatively independent control unit, and a reinforcement learning agent is assigned to each region, forming a multi-agent ensemble:

[0141] ;

[0142] in, Only responsible for control decisions in region i;

[0143] Step S32: Construct the state space, action space, and reward function for each agent;

[0144] Define the local observations of agent i in region i at discrete time t as:

[0145] ;

[0146] in, , , Let represent the outdoor ambient temperature at time t, the indoor temperature in region i, and the indoor temperature in region z, respectively. Let M represent the set of all other regions on the offshore platform except region i. Indicates the power of the electrical load. d represents the number of people. t The fresh air ratio is calculated and output by the auxiliary controller. This indicates the total energy consumption of the HVAC system;

[0147] The global state is obtained by stitching together the local observations of all agents:

[0148] ;

[0149] Z i The action at time t is defined as the air supply volume for that area:

[0150] ;

[0151] The joint action is as follows:

[0152] ;

[0153] The reward for each agent in each region at time t is defined as:

[0154] ;

[0155] in, , , The weights are set to balance the penalties for temperature deviation, energy consumption, and motion change, respectively.

[0156] Step S33: Training and parameter update process of the master controller based on MATD3;

[0157] The reinforcement learning algorithm used in this invention is MATD3. During the execution phase, each regional agent outputs its own airflow control action based solely on its current state. During the training phase, a centralized training and distributed execution framework is adopted. The airflow decisions of each region are evaluated through global state and joint actions, and training stability is improved through delayed policy updates and a dual-value network mechanism. A policy network and a dual-value network are constructed for each regional agent, and the training process is completed using a target network and a delayed update mechanism. The specific steps are as follows:

[0158] (1) Initialize the value network for each regional agent. and policy network and initialize the corresponding target network. and ;

[0159] (2) Initialize the experience recycling pool It is used to store joint experience samples generated during the interaction of multiple agents;

[0160] (3) Collect current local observations of each region. i,t And the local observations of all agents are spliced ​​together to form the global state S. t Let the combined action be A. t ;

[0161] (4) Each regional agent selects an action with detection noise based on the action output by its policy network. Combine the actions of each agent into a joint action A. t It acts on the environment, and after the environment executes the action, it returns an immediate reward r to each agent. i,t Local observation at the next moment i,t+1 and the termination mark D t To further construct the global state S at the next time step t+1 ;

[0162] (5) Store transition tuples In the experience recycling pool Among them, its and sample small batches of samples from them. ;

[0163] (6) Based on the target policy network of each region's agents, obtain the corresponding target actions and add pruned, smoothed noise: Combine all target actions into a target joint action. ;

[0164] (7) Utilize reward r i,t The global state S at the next moment t+1 and joint action of the target Calculate the time-series difference objective value , ;

[0165] (8) Update the two online value network parameters θ of each regional agent. i,1 and θ i,2 Minimize the loss function ;

[0166] (9) If the delayed update condition is met, update the policy network of each regional agent, and then softly update the target network parameters: , ;

[0167] (10) Repeat (4)-(9) until the termination condition is met.

[0168] Step S4: Construct an ADRC-based auxiliary controller to achieve robust fresh air regulation;

[0169] Step S41: Model the control target and controlled object of the auxiliary controller;

[0170] Since the supply air volume and fresh air ratio have different effects on system performance in a multi-zone HVAC system, the supply air volume mainly affects the thermal comfort of the zone, while the fresh air ratio is more sensitive to the indoor carbon dioxide concentration. Therefore, the fresh air ratio is used as an auxiliary control variable and adjusted online by ADRC.

[0171] Considering that the fresh air ratio adjustment corresponds to a uniform fresh air valve in the air handling unit, to ensure that the carbon dioxide constraint in the most unfavorable area is met, the maximum indoor carbon dioxide concentration in each area is selected as the feedback value for the auxiliary controller, i.e.:

[0172] ;

[0173] in, This represents the indoor carbon dioxide concentration in region i at time t; the goal of the auxiliary controller is to adjust the fresh air ratio u=d t This ensures that the output y(t) stably tracks the given reference value r, thereby meeting the indoor air quality control requirements;

[0174] Considering that indoor carbon dioxide dynamics are affected by various factors such as occupant exhalation, external air infiltration, inter-zone air exchange, and changes in the main controller's air supply volume, it is difficult to establish an accurate analytical model. To facilitate the design of the auxiliary controller, the input-output relationship between the fresh air ratio and indoor carbon dioxide concentration is abstracted as a first-order uncertain system:

[0175] ;

[0176] Where x represents the system state variable, y represents the system output, i.e., the indoor carbon dioxide concentration, and u represents the control input, i.e., the fresh air ratio d. t f(x,t,u) represents a comprehensive term that includes both internal system uncertainties and external disturbances; b is the system input gain.

[0177] Step S42: Define the total system disturbance and construct the extended state model;

[0178] To reduce the impact of model uncertainty and external disturbances on controller design, let b0 be an estimate of the input gain b, and unify system uncertainty, parameter mismatch, and other disturbances into a single total system disturbance, defined as:

[0179] ;

[0180] The original system is then rewritten as follows:

[0181] ;

[0182] Further introduce expansion state Construct an extended state model:

[0183] ;

[0184] Where x1 represents the system output state and x2 represents the system total disturbance state;

[0185] Step S43: Design of the linearly extended state observer;

[0186] To estimate the system state and total disturbance in real time, a linear extended state observer is introduced, which takes the following form:

[0187] ;

[0188] Where z1 is the estimate of the system state x1, z2 is the estimate of the total system disturbance x2, and β1 and β2 are the observer gain parameters, which are tuned using the bandwidth method.

[0189] ;

[0190] Where ω0 is the observer bandwidth, used to adjust the response speed and accuracy of the disturbance estimation;

[0191] S44: Design of disturbance compensation control law and linear error feedback control law;

[0192] Based on the online estimation results of the total disturbance by the linear extended state observer, a disturbance compensation control law is constructed:

[0193] ;

[0194] Where u0 is the nominal control input and z2 is the observer's estimate of the total disturbance; substituting this control law into the system, we get:

[0195] ;

[0196] For this approximate object, a linear error feedback control law is further adopted:

[0197] ;

[0198] Where, ω c This provides the controller bandwidth; thus, the auxiliary controller can adjust the fresh air ratio in real time based on the output error without relying on an accurate model, thereby achieving rapid tracking and stable control of indoor carbon dioxide concentration.

[0199] Step S5: Establish a collaborative communication mechanism between the main controller and the auxiliary controller.

[0200] During the real-time operation of the HVAC system, the auxiliary controller ADRC calculates the fresh air ratio dt based on the indoor carbon dioxide concentration feedback signal, and uses this fresh air ratio as one of the environmental state inputs of the main controller MATD3 to participate in the joint decision-making of the air supply volume of each zone; the joint action of the air supply volume of each zone output by the main controller MATD3 acts on the multi-zone HVAC system and is fed back to the auxiliary controller through state changes. Figure 3 As shown, the main controller and the auxiliary controller form an information interaction relationship within a unified closed-loop framework. The main controller is primarily responsible for multi-zone airflow decisions and energy consumption optimization under temperature constraints, while the auxiliary controller is primarily responsible for real-time adjustment of the fresh air ratio under indoor carbon dioxide concentration constraints. Together, they achieve energy-saving control of the multi-zone HVAC system while meeting temperature and indoor carbon dioxide concentration constraints.

[0201] The following simulation verification will be performed.

[0202] A simulation model of a multi-zone HVAC system for an offshore platform was established using EnergyPlus, with parameters including the number of zones, personnel, equipment load, and outdoor meteorological parameters. Simultaneously, a main controller and auxiliary controller modules implementing the control method of this invention were constructed in a Python environment. The EnergyPlus simulation model and the Python control module were connected via an FMU interface. Input and output variables were defined in EnergyPlus, and the EnergyPlusToFMU tool was used to encapsulate the EnergyPlus simulation model into an FMU model. The FMU model was then loaded into the Python environment using PyFMI, and simulation parameters were set.

[0203] At each control moment, the main controller and auxiliary controller output control values ​​for the air supply volume and fresh air ratio, and input these control values ​​into the FMU model. After performing one simulation step, the FMU model returns status information such as indoor temperature, indoor carbon dioxide concentration, and HVAC system energy consumption for each area. Based on this status information, the control module makes control decisions for the next moment and repeats the above process until the set simulation duration or termination condition is reached, thus completing the simulation verification.

Claims

1. A method for coordinated control of a multi-zone HVAC system on an offshore platform, characterized in that, Includes the following steps: Step S1: Decouple carbon dioxide concentration and temperature control, map them to the air supply volume of each area and the unified fresh air ratio adjustment respectively, and set constraints and optimization objectives; Step S2: Collect temperature, indoor carbon dioxide concentration and outdoor environmental parameters for each area. The temperature and outdoor environmental parameters for each area are used as part of the state input of the main controller, and the maximum value of indoor carbon dioxide concentration in each area is used as the feedback quantity of the auxiliary controller. Step S3: Construct a master controller based on a multi-agent dual-delay deep deterministic policy gradient algorithm to realize joint decision-making on air supply volume in multiple regions; Step S4: Construct an auxiliary controller based on active disturbance rejection control to achieve robust adjustment of fresh air supply; Step S5: Establish a collaborative communication mechanism between the main controller and the auxiliary controller.

2. The method for coordinated control of a multi-zone HVAC system on a marine platform according to claim 1, characterized in that, The specific process of step S1 includes: Step S11: Decouple temperature control from indoor carbon dioxide concentration control; The control targets for temperature and carbon dioxide concentration in each area are processed in a stratified manner: temperature regulation in each area is mapped to air supply volume control in each area, which is completed by the main controller; carbon dioxide concentration regulation is mapped to the unified fresh air ratio control of the HVAC system, which is completed by the auxiliary controller. Step S12: Establish indoor temperature and carbon dioxide concentration constraints for each area; Suppose the offshore platform comprises N regions, and the indoor temperature in region i is... Set constraints: ； in, and These are the minimum and maximum allowable temperatures for normal operation of zone i, respectively. With the air handling unit supply air temperature setpoint fixed, the indoor temperature of each zone is adjusted by regulating the corresponding zone's supply air volume. accomplish; To ensure regional air quality, carbon dioxide concentrations in each area are monitored. Set constraints: ； in, To set the upper limit of permissible carbon dioxide concentration in zone i, the fresh air ratio is adjusted. The carbon dioxide concentration in each area is adjusted to meet the corresponding air quality requirements. Step S13: Establish an energy consumption model and optimization objectives for the HVAC system; Energy consumption of cooling coils in HVAC systems Simultaneously affected by the air supply volume and fresh air ratio in each area, it can be expressed as: ； in, The specific heat capacity of air, The efficiency coefficient of the cooling coil coil. Temperature of adjacent areas Supply temperature setpoints to the air handling unit; Energy consumption of fans in HVAC systems Represented as: ； in, The rated power of the fan under reference operating conditions. It is the reference total air supply volume of the fan under rated operating conditions; The total energy consumption of the HVAC system is: ； The control optimization objective is to reduce the total energy consumption of the HVAC system while satisfying temperature and carbon dioxide concentration constraints. The optimization objective function is established as follows: ； Where T is the optimization time domain length, the objective function is used to characterize the energy consumption optimization objective of the HVAC system under the conditions of satisfying temperature and indoor carbon dioxide constraints.

3. The method for coordinated control of multi-zone HVAC systems on offshore platforms according to claim 1, characterized in that, The specific process of step S3 includes: Step S31: Establish a multi-agent airflow control model; Each region is treated as a relatively independent control unit, and a reinforcement learning agent is assigned to each region, forming a multi-agent ensemble: ； in, Only responsible for control decisions in region i; Step S32: Construct the state space, action space, and reward function for each agent; Define the local observations of agent i in region i at discrete time t as: ； in, , , Let represent the outdoor ambient temperature at time t, the indoor temperature in region i, and the indoor temperature in region z, respectively. Let M represent the set of all other regions on the offshore platform except region i. Indicates the power of the electrical load. d represents the number of people. t The fresh air ratio is calculated and output by the auxiliary controller. This indicates the total energy consumption of the HVAC system; The global state is obtained by stitching together the local observations of all agents: ； Z i The action at time t is defined as the air supply volume for that area: ； The joint action is as follows: ； The reward for each agent in each region at time t is defined as: ； in, , , The set weights are used to balance the temperature deviation penalty, energy consumption penalty, and motion change penalty, respectively. Step S33: Training and parameter update process of the master controller based on the multi-agent dual-delay deep deterministic policy gradient algorithm; The reinforcement learning algorithm used is a multi-agent, dual-delay deep deterministic policy gradient algorithm. During the execution phase, each agent in a region outputs its own air supply control action based solely on its current state. During the training phase, a centralized training and decentralized execution framework is adopted. The air supply decisions of each region are evaluated through global state and joint actions, and the training stability is improved through delayed policy updates and a dual-value network mechanism. A policy network and a dual-value network are constructed for each agent in a region, and the training process is completed using a target network and a delayed update mechanism.

4. The method for coordinated control of multi-zone HVAC systems on offshore platforms according to claim 3, characterized in that, The training process employs a target network and a delayed update mechanism, and the specific steps are as follows: (1) Initialize the value network for each regional agent. and policy network and initialize the corresponding target network. and ; (2) Initialize the experience recycling pool It is used to store joint experience samples generated during the interaction of multiple agents; (3) Collect current local observations of each region. i,t And the local observations of all agents are spliced ​​together to form the global state S. t Let the combined action be A. t ; (4) Each regional agent selects an action with detection noise based on the action output by its policy network. Combine the actions of each agent into a joint action A. t It acts on the environment, and after the environment executes the action, it returns an immediate reward r to each agent. i,t Local observation at the next moment i,t+1 and the termination mark D t To further construct the global state S at the next time step t+1 ; (5) Store transition tuples In the experience recycling pool Among them, its And sample small batches of samples from them. ; (6) Based on the target policy network of each region's agents, obtain the corresponding target actions and add pruned, smoothed noise: Combine all target actions into a target joint action. ; (7) Utilize reward r i,t The global state S at the next moment t+1 and joint action of the target Calculate the time-series difference objective value , ; (8) Update the two online value network parameters θ of each regional agent. i,1 and θ i,2 Minimize the loss function ; (9) If the delayed update condition is met, update the policy network of each regional agent, and then softly update the target network parameters: , ; (10) Repeat (4)-(9) until the termination condition is met.

5. The method for coordinated control of a multi-zone HVAC system on a marine platform according to claim 1, characterized in that, The specific process of step S4 includes: Step S41: Model the control target and controlled object of the auxiliary controller; The fresh air ratio is used as a separate auxiliary control variable and adjusted online by the auxiliary controller. The maximum carbon dioxide concentration in each region is selected as the feedback value for the auxiliary controller, i.e.: ； in, This represents the indoor carbon dioxide concentration in region i at time t; the goal of the auxiliary controller is to adjust the fresh air ratio u=d t This ensures that the output y(t) stably tracks the given reference value r, thereby meeting the indoor air quality control requirements; The input-output relationship between the fresh air ratio and the indoor carbon dioxide concentration can be abstracted into a first-order uncertain system: ； Where x represents the system state variable, y represents the system output, i.e., the indoor carbon dioxide concentration, and u represents the control input, i.e., the fresh air ratio d. t f(x,t,u) represents a comprehensive term that includes both internal system uncertainties and external disturbances; b is the system input gain. Step S42: Define the total system disturbance and construct the extended state model; Let b0 be an estimate of the input gain b, and let system uncertainties, parameter mismatches, and other disturbances be uniformly grouped into the total system disturbance, defined as: ； The original system is then rewritten as follows: ； Further introduce expansion state Construct an extended state model: ； Where x1 represents the system output state and x2 represents the system total disturbance state; Step S43: Design of the linearly extended state observer; To estimate the system state and total disturbance in real time, a linear extended state observer is introduced, which takes the following form: ； Where z1 is the estimate of the system state x1, z2 is the estimate of the total system disturbance x2, and β1 and β2 are the observer gain parameters, which are tuned using the bandwidth method. ； Where ω0 is the observer bandwidth, used to adjust the response speed and accuracy of the disturbance estimation; S44: Design of disturbance compensation control law and linear error feedback control law; Based on the online estimation results of the total disturbance by the linear extended state observer, a disturbance compensation control law is constructed: ； Where u0 is the nominal control input and z2 is the observer's estimate of the total disturbance; substituting this control law into the system, we get: ； For this approximate object, a linear error feedback control law is further adopted: ； Where, ω c This provides the controller bandwidth; thus, the auxiliary controller can adjust the fresh air ratio in real time based on the output error without relying on an accurate model, thereby achieving rapid tracking and stable control of indoor carbon dioxide concentration.

6. The method for coordinated control of a multi-zone HVAC system on a marine platform according to claim 1, characterized in that, The specific process of step S5 includes: During the real-time operation of the HVAC system, the auxiliary controller calculates the system's fresh air ratio d based on the indoor carbon dioxide concentration feedback signal. t The fresh air ratio is used as one of the environmental state inputs of the main controller to participate in the joint decision-making of the air supply volume of each zone. The joint action of the air supply volume of each zone output by the main controller acts on the multi-zone HVAC system and is fed back to the auxiliary controller through system state changes. The main controller and the auxiliary controller form an information interaction relationship under a unified closed-loop framework. Among them, the main controller is responsible for the multi-zone air supply volume decision and energy consumption optimization under temperature constraints, while the auxiliary controller is responsible for the real-time adjustment of the fresh air ratio under indoor carbon dioxide concentration constraints, so as to jointly achieve energy-saving control of the multi-zone HVAC system under the premise of meeting temperature and indoor carbon dioxide concentration constraints.

Images