Self-optimizing control method and system for multi-stage chemical reaction processes

Abstract

The application relates to the field of automatic control technology and discloses a self-optimization control method and system for a multi-stage chemical reaction process, wherein the self-optimization control method for the multi-stage chemical reaction process comprises the following steps: collecting operation data, constructing a directed graph of the multi-stage reaction, and establishing a time delay queue; setting a global scheduler to determine the update time of each stage, and constructing an enhanced state vector; establishing a shadow predictor, and establishing a double-layer self-optimization control structure; calculating a candidate control amount, and performing constraint projection, rate limitation and safety prediction processing to obtain a final control amount; a coordinator executes shared resource allocation optimization, and determines and issues the final control amount at the update time; the application explicitly includes the transmission time delay of the information flow between stages into the control framework by constructing a multi-stage reaction chain graph and establishing an online time delay queue, so that the control system can accurately perceive and predict the influence of the operation of an upstream stage on a downstream stage, and the long-term stable operation of the system is ensured.

Description

Technical Field

[0001] This application relates to the field of automation control technology, and more specifically, to a self-optimizing control method and system for multi-stage chemical reaction processes. Background Technology

[0002] Multistage chemical reaction processes are widely found in industries such as petrochemicals, fine chemicals, pharmaceuticals, and metallurgy. Typical applications include hydrofluoric acid recovery, polymerization reactions, catalytic cracking, and multistage extraction. These processes typically consist of multiple reaction stages connected in series or parallel. These stages are tightly coupled through material flow, energy flow, and information flow. The products of one stage often become the raw materials for the next stage, and fluctuations in process parameters are transmitted and amplified step by step along the reaction chain, forming complex cascade effects.

[0003] A Chinese patent with authorization announcement number CN120178693B discloses a reinforcement learning-based method for optimizing and controlling the chemical reaction in tin smelting. An improved Fourier synthesis method is used to simulate the state data during tin smelting. Subsequently, a reinforcement learning-based process model is constructed, using key process parameters within the furnace as state variables. A dual-network Actor-Critic architecture is employed to achieve dynamic optimization control. A physicochemical model is also introduced, combining process parameters and the thermodynamic equilibrium formula of the chemical reaction to calculate the trend of carbon monoxide generation. This method can intelligently optimize the chemical reaction conditions during tin smelting, increasing tin production, reducing carbon monoxide emissions, and effectively shortening reaction time, thus reducing energy waste.

[0004] However, the state space of existing technologies only contains parameters within a single reactor, failing to characterize material transfer and energy coupling relationships between multiple stages. This makes it difficult to capture the impact of cascade effects on downstream stages. Furthermore, existing technologies optimize each stage independently without considering inter-stage interactions. For example, excessive resource consumption in the upstream stage in pursuit of local optima may lead to insufficient raw materials or decreased quality in the downstream stage, ultimately resulting in increased global energy consumption and product quality fluctuations. Summary of the Invention

[0005] The purpose of this application is to address the problems existing in the control of multi-stage chemical reaction processes, such as strong coupling between stages, information transmission delays, asynchronous dynamic changes, and difficulties in global optimization. It aims to provide a self-optimizing control method and system for multi-stage chemical reaction processes. By establishing a directed graph representation of the multi-stage reaction process and combining an asynchronous update scheduling mechanism, a shadow predictor, and a two-layer self-optimizing control structure, the control strategies for each stage are optimized online and coordinated globally, thereby improving the control performance, disturbance rejection capability, and overall operational efficiency of the multi-stage chemical reaction process.

[0006] This application provides a self-optimization control method for a multi-stage chemical reaction process, including the following steps:

[0007] Collect operational data of a multi-stage chemical reaction process, and construct a directed graph of the multi-stage reaction based on the operational data. The directed graph includes a set of nodes and a set of directed edges. Establish a time-delay queue for each directed edge.

[0008] An independent update cycle is set for each stage. A global scheduler is set according to the update cycle to determine the update time of each stage. After the update time is reached, an enhanced state vector is constructed for each stage based on the running data, the directed graph, and the time-delay queue.

[0009] A shadow predictor is established for each stage based on the enhanced state vector, and a two-layer self-optimizing control structure is established based on the shadow predictor and stage policy of each stage. The two-layer self-optimizing control structure includes a local layer and a global layer. The local layer includes multiple stage policies, and the global layer includes a coordinator.

[0010] At the update time, candidate control quantities are calculated according to the stage strategy, and the candidate control quantities are sequentially subjected to constraint projection, rate limiting and safety prediction processing based on the shadow predictor to obtain the final control quantity;

[0011] When the preset global coordination cycle arrives, the coordinator performs shared resource allocation optimization to obtain the optimal resource quota for each stage, and converts the optimal resource quota into stage constraint updates and stage reference settings for each stage, so that each stage can determine and issue the final control quantity at the update time.

[0012] This application provides a self-optimizing control system for a multi-stage chemical reaction process, which stores computer-readable instructions and, when the computer-readable instructions are read, can execute the aforementioned self-optimizing control method for the multi-stage chemical reaction process; the system includes:

[0013] The data modeling module is used to collect operational data of multi-stage chemical reaction processes and construct a directed graph of the multi-stage reaction based on the operational data. The directed graph includes a set of nodes and a set of directed edges, and a time-delay queue is established for each directed edge.

[0014] The state construction module is used to set an independent update cycle for each stage, set a global scheduler to determine the update time of each stage according to the update cycle, and construct an enhanced state vector for each stage based on the running data, directed graph and time delay queue after the update time is reached.

[0015] The predictive control module establishes a shadow predictor for each stage based on the enhanced state vector, and establishes a two-layer self-optimizing control structure based on the shadow predictor and stage strategy of each stage. The two-layer self-optimizing control structure includes a local layer and a global layer. The local layer includes multiple stage strategies, and the global layer includes a coordinator.

[0016] The control generation module is used to calculate candidate control quantities according to the stage strategy at the update time, and to perform constraint projection, rate limiting and safety prediction processing on the candidate control quantities in sequence based on the shadow predictor to obtain the final control quantity.

[0017] The collaborative output module is used to optimize the allocation of shared resources by the coordinator when the preset global coordination cycle arrives, obtain the optimal resource quota for each stage, and convert the optimal resource quota into stage constraint updates and stage reference settings for each stage, so that each stage can determine and issue the final control quantity at the update time.

[0018] The beneficial effects of this application are as follows: By constructing a multi-stage reaction chain diagram model and establishing an online time-delay queue, this application explicitly incorporates the transmission time delays of material flow, energy flow, and information flow between stages into the control framework. This effectively solves the problem of strong coupling and transmission time delays between stages, which is difficult to handle with traditional control methods, enabling the control system to accurately perceive and predict the impact of upstream stage operations on downstream stages. By adopting an event-triggered mechanism in conjunction with a multi-time-scale clock, the update time of each stage is dynamically determined based on the degree of state change, quality deviation, and cycle arrival conditions. This avoids the waste of computational resources and response lag problems caused by fixed-cycle sampling, allowing fast-variable stages and slow-variable stages to perform differentiated control updates according to their respective dynamic characteristics, thus improving the adaptability of the control system to multi-time-scale processes.

[0019] This application introduces downstream responsibility terms into the local layer reward function, enabling each stage to proactively consider the impact on the quality of downstream stages when optimizing its own control strategy, thus fundamentally avoiding the problem of local optima leading to global deterioration. The global layer coordinator achieves optimal allocation of shared resources across stages through marginal revenue curves and dynamically adjusts the responsibility weights and resource constraints of upstream stages by setting conflict detection indicators, achieving an organic unity between local optimization and global coordination. By establishing a lightweight shadow predictor and combining it with a safety prediction mechanism, the future state is predicted and evaluated before the control quantity is issued. When a potential risk of exceeding limits is detected, the control quantity is automatically adjusted through a conservative scaling strategy to ensure that the process always operates within safety constraints. Simultaneously, through online monitoring of model uncertainty and adaptive parameter adjustment mechanisms, the control system can automatically adopt a conservative strategy when model accuracy decreases and gradually recover its learning ability after conditions recover, ensuring the long-term stable operation of the system. Attached Figure Description

[0020] Figure 1 This is a flowchart illustrating the self-optimization control method for the multi-stage chemical reaction process of this application.

[0021] Figure 2 This is a flowchart illustrating the process by which the final control quantity is obtained in this application.

[0022] Figure 3 This is a module example diagram of the self-optimizing control system for the multi-stage chemical reaction process of this application. Detailed Implementation

[0023] The subject matter described herein will now be discussed with reference to exemplary embodiments. It should be understood that these embodiments are discussed only to enable those skilled in the art to better understand and implement the subject matter described herein, and changes may be made to the function and arrangement of the elements discussed without departing from the scope of this specification. Various processes or components may be omitted, substituted, or added as needed in the examples. Furthermore, features described in some examples may be combined in other examples.

[0024] A self-optimizing control method and system for multi-stage chemical reaction processes, including the following embodiments:

[0025] Example 1:

[0026] Self-optimization control methods for multi-stage chemical reaction processes, such as Figure 1 As shown, it includes the following steps:

[0027] Step 100: Collect operational data of the multi-stage chemical reaction process, construct a directed graph of the multi-stage reaction based on the operational data, the directed graph includes a set of nodes and a set of directed edges, and establish a time-delay queue for each directed edge.

[0028] Real-time operating data for each stage is collected from a distributed control system (DCS) or a programmable logic controller (PLC). The operating data includes stage-specific status data, output quality data, total global energy consumption, disturbance data, and actuator status data.

[0029] Intra-stage state data refers to the measured values ​​used to describe the dynamic behavior of the process at time t in stage i, including temperature, pressure, flow rate, liquid level, etc., where i ranges from 1 to N positive integers, and N is the total number of stages; Output quality data refers to the measured values ​​used to evaluate product performance at time t, including product purity, yield, emission indicators, etc. It should be noted that component concentration is classified according to its role in the control system. When component concentration is used to predict process dynamic changes, it is classified as intra-stage state data; when component concentration is used to evaluate the quality of the final product, it is classified as output quality data; Total global energy consumption refers to the measured values ​​of energy consumption such as steam consumption, electricity consumption, and cooling water consumption at time t; Disturbance data refers to the measured values ​​of external disturbances such as raw material composition fluctuations, ambient temperature, and equipment load at time t; Actuator state data refers to the actual working status feedback values ​​of each actuator at time t in stage i, including feed valve opening, reflux valve opening, steam valve opening, cooling water valve opening, stirring speed, and tower bottom heat load, etc. This data is collected in real time by position sensors or status detection devices on the actuators.

[0030] Control quantity refers to the command value sent by the control system to the actuator, which is used to drive the actuator to adjust the process state. The relationship between control quantity and actuator state data is as follows: control quantity is the expected value calculated and output by the control system, and actuator state data is the actual feedback value after the actuator responds to the control quantity command. Ideally, the two should be consistent, but due to factors such as actuator response delay, mechanical backlash, valve jamming, etc., the actual actuator state data may deviate from the control quantity. During the operation of the control system, actuator state data is used to initialize control quantity, monitor the health status of the actuator, and calculate the increment of control quantity change. The specific application methods will be explained in detail in the following steps.

[0031] A multi-stage chemical reaction process is represented by a directed graph model. This directed graph includes a set of nodes and a set of directed edges. The node set contains N nodes, each representing a reaction stage, with the i-th node representing the i-th reaction stage. Each directed edge in the set of directed edges represents a connection between one stage and another, indicating a material flow, energy flow, or information flow. If a directed edge exists from node i to node j, it indicates that there is such a connection between stage i and stage j. The reason for using a directed graph model to represent a multi-stage chemical reaction process is that a directed graph can clearly depict the topological connections and material and energy transfer directions between each reaction stage. Compared with traditional series or parallel models, the directed graph model can flexibly express complex network structures, including various connection forms such as branching, merging, and looping, thus accurately reflecting the actual flow of materials and energy in the actual chemical process. Another advantage of the directed graph model is that it naturally supports the explicit expression of coupling relationships between stages. Directed edges can clearly identify which stages have direct influences, providing a structured information foundation for subsequent construction of enhanced state vectors and design of coordinated control strategies.

[0032] For each directed edge in the directed graph, an online time-delay queue is established. The time-delay queue includes multiple time-delay states to store historical state information from upstream stages. The output of the time-delay queue is set to the time-delay state transmitted from upstream to downstream stages. Specifically, the time-delay state transmitted from stage i to stage j at time t is equal to the state value of stage i at time t minus the transmission delay. The state value is a vector composed of state data from all stages at a specific time t, arranged in a fixed order according to measurements such as temperature, pressure, flow rate, and liquid level. That is, the values ​​of each measurement at that time are written into the corresponding positions of the vector one by one according to a predetermined order. The first position stores the temperature measurement value, the second position stores the pressure measurement value, and so on until all measurement values ​​are included, thus forming a complete state value vector. The transmission delay refers to the time delay from stage i to stage j. The time delay, determined by physical processes such as material transport time and heat transfer time, is read from the time delay queue. The reason for establishing the time delay queue is that there is an inherent time delay in the material and energy transfer between stages in a multi-stage chemical reaction process. If this time delay characteristic is ignored, the control system will not be able to accurately predict the response behavior of the downstream stage, resulting in a timing misalignment between the control decision and the actual process state. By storing the historical state sequence of the upstream stage, the time delay queue enables the downstream stage to obtain the upstream state information that is physically related to the current moment when performing control calculations, thereby achieving time delay compensation. This design effectively solves the problem of decreased control performance caused by ignoring the inter-stage transmission time delay in traditional control methods. Especially in operating conditions with time delays, such as long pipeline transportation and large-capacity storage tank buffering, the introduction of the time delay queue can improve control accuracy and system stability.

[0033] Step 200: Set an independent update cycle for each stage, set a global scheduler to determine the update time of each stage according to the update cycle, and construct an enhanced state vector for each stage based on the running data, directed graph and time delay queue after the update time is reached.

[0034] An independent update cycle is set for each stage, with the update cycle for fast variable stages being on the order of seconds, and the update cycle for slow variable stages being on the order of minutes or hours. The update cycle of stage i is denoted as the time step of that stage. The update cycle refers to the maximum allowable time interval between two adjacent control updates in that stage, used to ensure that control calculations are performed periodically even when the process state is stable. The update time refers to the specific time point at which the global scheduler triggers the execution of control updates in that stage, and this time point is dynamically determined by the event triggering mechanism. The relationship between the update cycle and the update time is that the update cycle participates in the determination of the update time as one of the triggering conditions. When the time interval between the current time and the previous update time reaches the update cycle, even if other... Even if the triggering condition is not met, a new update will still be triggered. The reason for setting an independent update cycle for each stage is that the dynamic characteristics of different stages in a multi-stage chemical reaction are different. For example, the response time constant of the gas phase reaction stage is usually on the order of seconds, while the response time constant of the liquid phase separation stage may reach the order of minutes or even hours. If a uniform update cycle is used, it will lead to untimely control updates and miss the best adjustment opportunity for fast variable stages, and generate a lot of redundant calculations and waste computing resources for slow variable stages. By configuring an independent update cycle that matches the dynamic characteristics of each stage, the overall computational burden of the system can be effectively reduced while ensuring timely control, and the rational allocation of computing resources can be achieved.

[0035] A global scheduler is configured, employing an event-triggered mechanism to determine the update timing for each stage. At each acquisition moment, the global scheduler checks whether each stage meets the update conditions. When stage i meets any update condition at a given acquisition moment, the global scheduler determines that acquisition moment as the update moment for stage i and immediately triggers the control update for that stage. Update conditions include state change conditions, quality deviation conditions, and cycle arrival conditions. The reason for using an event-triggered mechanism is that traditional periodic triggering control methods cannot adapt to the dynamic fluctuations in the rate of state change during multi-stage chemical reactions. When the process state is stable, periodic triggering generates a large amount of unnecessary control calculations; while when the process state changes drastically, a fixed cycle may not respond in time. The event-triggered mechanism monitors state changes and quality deviations in real time, triggering updates only when there is a genuine control requirement, thereby reducing communication and computational overhead while ensuring control performance. This on-demand triggering method is particularly suitable for distributed control system architectures, effectively reducing network bandwidth usage and controller computational load.

[0036] The state change condition is met when the Euclidean distance between the state value of stage i at the current acquisition time and the state value at the last update time of the stage exceeds the preset state change threshold of the stage. The Euclidean distance refers to the square root of the sum of the squares of the differences between the corresponding components of two vectors. The preset state change threshold is determined based on the physical dimensions and normal fluctuation range of each measured value in the state data of each stage. Specifically, it is obtained by statistically analyzing the standard deviation of each measured value in the historical operation data of the stage and multiplying it by a preset multiplier coefficient. The preset multiplier coefficient is selected based on the fluctuation characteristics of each measured value in the stage and the control sensitivity requirements.

[0037] The quality deviation condition is met when the Euclidean distance between the output quality data at the current acquisition time and the set quality target exceeds the preset quality deviation threshold. The preset quality deviation threshold is determined according to the product quality control accuracy requirements, specifically by analyzing the allowable deviation range of the quality indicators and combining it with the quality qualification judgment criteria in the process specification.

[0038] The cycle is reached when the time interval between the current acquisition time and the last update time of stage i is greater than or equal to the update cycle of that stage.

[0039] When stage i meets any of the above conditions at a certain acquisition time, the global scheduler determines the acquisition time as the update time of stage i, triggers the control update of stage i, and records the time as the latest update time of stage i.

[0040] After determining the update time, control calculation and optimization operations are performed at that time. At the update time, the current enhanced state vector for that stage is first read and updated, and the actuator state data for that stage is also read as a reference for the current control quantity. Then, candidate control quantities are calculated based on the stage policy function, and constraint projection, rate limiting, and safety prediction are sequentially applied to these candidate control quantities to obtain the final control quantity. Simultaneously, the policy gain matrix is ​​updated online at the update time, and the policy parameters are adjusted based on the gradient information of the reward function using a deterministic policy gradient method. Furthermore, the parameters of the shadow predictor are updated at the update time, using a recursive least squares algorithm with a forgetting factor to correct the prediction model parameters based on the latest state observation data. Finally, the final control quantity is distributed to the actuators for that stage, completing the control output for this update time.

[0041] For each stage, an enhanced state vector is constructed, explicitly incorporating the inter-stage coupling relationship and time-delay queue into the state representation. The enhanced state vector of stage i at the update time is constructed as follows: first, the local state vector is used as the starting part of the enhanced state vector; then, each time-delay state in the time-delay state set passed from all upstream stages to stage i is arranged sequentially after the local state vector; next, the target deviation vector of stage i at the update time is arranged after the time-delay state set; finally, the model uncertainty estimation vector of stage i at the update time is arranged after the target deviation vector. The above four parts are combined end-to-end in this fixed order to form a complete enhanced state vector. Among them, the local state vector is a vector after standardizing the state values, used to control the algorithm input. Each component in the local state vector corresponds to a measurement value in the state data within the corresponding stage, i.e., the standardized values ​​of temperature, pressure, flow rate, liquid level, etc. These components are collectively referred to as state variables in the following description; upstream stages refer to all stages in the directed graph that have directed edges pointing to stage i; the target deviation vector is the difference between the current actual value of each state variable at the update time of stage i and the corresponding reference setpoint. The vector formed by these parameters refers to the expected operating values ​​of each state variable in this stage, which are pre-set according to the process specifications and production targets. These values ​​are determined by the process design or dynamically issued by the global layer coordinator based on resource allocation results. The dimension of the model uncertainty estimation vector is the same as that of the local state vector, and each component corresponds to the uncertainty estimate of the corresponding state variable in the local state vector. The reason for constructing the enhanced state vector is that simply using the local state vector cannot reflect the coupling effects between stages, while the core characteristic of a multi-stage chemical reaction process is the strong coupling relationship between each stage. By incorporating the time-delay state of the upstream stage into the enhanced state vector, the control strategy of each stage can perceive the propagation of disturbances from the upstream, thereby achieving a feedforward compensation effect. The reason for incorporating the target deviation vector into the enhanced state vector is that the target deviation directly reflects the urgency of the current control task, enabling the control strategy to adaptively adjust the control intensity according to the magnitude of the deviation. The reason for incorporating the model uncertainty estimation vector into the enhanced state vector is that the model uncertainty reflects the reliability of the current prediction model, enabling the control strategy to automatically take more conservative control actions when the model accuracy decreases, thereby improving the robustness of the system.

[0042] The model uncertainty estimation vector is calculated. Each component of the model uncertainty estimation vector at time t in stage i corresponds to the uncertainty estimate of the corresponding state variable in the local state vector. The calculation method is the square root of the mean of the squares of the differences between the actual value and the predicted value of the state variable at each time within the sliding window. The preset sliding window length is W time steps, and the time step index within the window ranges from 0 to W minus 1. The preset sliding window length is determined based on the statistical characteristics of the prediction error. Specifically, by analyzing the autocorrelation of the prediction error sequence, a window length that can cover the main fluctuation period of the error is selected, and the number of samples within the window meets the stability requirements required for estimating the mean error and the amplitude of error fluctuation, while also taking into account computational efficiency. For fast variable stages, the preset sliding window length can be appropriately reduced to improve the response speed; for slow variable stages, the preset sliding window length can be appropriately increased to improve the estimation stability.

[0043] Step 300: Based on the enhanced state vector, a shadow predictor is established for each stage, and a two-layer self-optimizing control structure is established based on the shadow predictor and stage policy of each stage. The two-layer self-optimizing control structure includes a local layer and a global layer. The local layer includes multiple stage policies, and the global layer includes a coordinator.

[0044] A lightweight shadow predictor is established for each stage to predict the future state of that stage. The shadow predictor adopts a linear regression model based on the operating point linearization, which is suitable for state prediction near the operating point. For strongly nonlinear processes, a piecewise linear model or an extended nonlinear model can be used as an alternative. The shadow predictor of stage i predicts the state of stage i at time t plus the stage update period based on the enhanced state vector and control input at time t. The prediction equation is: the predicted state equals the state transition matrix multiplied by the enhanced state vector, and the control input matrix multiplied by the sum of the control input and the bias vector. Here, the predicted state is the future state vector output by the shadow predictor. The dimension of this vector is the same as the local state vector, and each component corresponds to the predicted value of the corresponding state variable in the local state vector at the future time. The same standardization process as the local state vector is used to maintain the consistency of numerical scale; the state transition matrix, control input matrix, and bias vector are the model parameters of the shadow predictor for this stage; the reason for using a lightweight shadow predictor is that real-time control of multi-stage chemical reaction processes has strict requirements for computational efficiency. Although complex nonlinear prediction models have high accuracy, they are computationally time-consuming and cannot meet the real-time requirements of the second-level update cycle in fast variable stages; linear regression models have a simple structure, fast computation speed, and sufficient prediction accuracy near the operating point, achieving a good balance between computational efficiency and prediction accuracy; another important role of the shadow predictor is to provide state prediction support for safety prediction. By predicting future state trajectories in advance, the control system can take preventive measures before state out-of-bounds occurrence, achieving active safety control. In one embodiment of this application, the formula for calculating the predicted state is as follows:

[0045] ;

[0046] in, Here is the state transition matrix. To enhance the state vector, To control the input matrix, To control the quantity, It is the bias vector; For stage i at time i The predicted state.

[0047] The state transition matrix, control input matrix, and bias vector are combined into a parameter matrix, and a recursive least squares algorithm with a forgetting factor is used for online parameter updates. The reason for using a recursive least squares algorithm with a forgetting factor is that the dynamic characteristics of a chemical reaction process slowly drift due to factors such as catalyst activity decay, equipment aging, and changes in raw material composition. A prediction model with fixed parameters will gradually lose accuracy. The recursive least squares algorithm can continuously correct model parameters based on the latest observation data, ensuring the prediction model always tracks the current process characteristics. The purpose of introducing the forgetting factor is to give higher weight to recent data, enabling the algorithm to adapt to changes in process characteristics more quickly, while gradually reducing the influence of historical data and avoiding interference from outdated data in parameter estimation. Another advantage of this algorithm is its recursive calculation form; each update only needs to utilize the current observation data and the parameter estimate from the previous time step, eliminating the need to store and process large amounts of historical data. This results in low computational complexity and small memory footprint, making it very suitable for online real-time applications. To avoid coupling drift caused by simultaneous model and policy updates, a time-scale separation mechanism is adopted, and the parameter update frequency of the shadow predictor is set... The preset model-policy update frequency ratio is set to a factor of 1.5. This ratio is determined based on the coupling between model parameters and policy parameters, specifically by analyzing the stability of parameter estimation when both are updated simultaneously. This ensures that policy updates are performed on a relatively stable model. The time-scale separation mechanism is employed because if model and policy parameters are updated rapidly simultaneously, they can interfere with each other. Changes in model parameters affect the calculation of policy gradients, while changes in policy parameters alter the closed-loop behavior of the system, affecting model identification. This bidirectional coupling can lead to parameter estimation divergence or oscillation. By reducing the update frequency of model parameters, policy updates are performed on a relatively stable model, thus ensuring the stability of the overall learning process. Furthermore, when the magnitude of model parameter updates exceeds a preset parameter update magnitude threshold, policy updates are paused until the model parameters converge. This threshold is determined based on the norm of the parameter matrix, specifically by analyzing the relationship between the magnitude of model parameter changes and the deviation in policy gradient calculation. A threshold that allows for normal adaptive adjustment of model parameters while ensuring the reliability of policy updates is selected.

[0048] The regression vector is constructed as follows: first, the transpose of the enhanced state vector from the previous update cycle is used as the starting part of the regression vector; then, the transpose of the control variables from the previous update cycle is arranged after the transpose of the enhanced state vector; finally, the constant 1 is arranged after the transpose of the control variables. These three parts are combined end-to-end in this fixed order to form a complete regression vector. The specific implementation of the recursive least squares algorithm is as follows:

[0049] First, calculate the prediction error, which is equal to the actual state value at the current moment minus the product of the regression vector and the parameter matrix at the previous moment.

[0050] Then calculate the gain vector, which is equal to the covariance matrix of the previous time step multiplied by the regression vector, divided by the preset forgetting factor, plus the sum of the transpose of the regression vector multiplied by the covariance matrix of the previous time step and then multiplied by the regression vector.

[0051] Next, the parameter matrix is ​​updated. The new parameter matrix is ​​equal to the parameter matrix of the previous time step plus the product of the gain vector and the transpose of the prediction error.

[0052] Finally, the covariance matrix is ​​updated. The new covariance matrix is ​​equal to the previous covariance matrix minus the result of multiplying the gain vector by the transpose of the regression vector and then by the previous covariance matrix. This difference is then divided by a preset forgetting factor. In one embodiment of this application, the formula for calculating the gain vector is:

[0053] ;

[0054] in, For the stage The gain vector at the current moment, For the stage The covariance matrix at the previous time step, For the stage The regression vector at the previous update time, For the stage The transpose of the regression vector at the previous update time. For the stage The preset forgetting factor.

[0055] The parameter matrix is ​​updated as follows:

[0056] ;

[0057] in, For the stage The parameter matrix updated at the current moment. The parameter matrix of the previous time step. This is the transpose of the prediction error.

[0058] The covariance matrix is ​​updated as follows:

[0059] ;

[0060] in, For the stage The updated covariance matrix at the current moment.

[0061] The initial value of the covariance matrix is ​​set as the identity matrix multiplied by a preset initial covariance coefficient. The preset initial covariance coefficient is determined based on the initial uncertainty of the parameters; the larger the initial uncertainty, the larger the value of the coefficient. The preset forgetting factor is a real number greater than 0 and less than or equal to 1. The preset forgetting factor is determined based on the time-varying rate of the process dynamics, specifically by analyzing the time constant of the model parameter drift in historical data. For rapidly time-varying processes, a smaller forgetting factor is selected to enhance the tracking ability of new data, while for slowly time-varying processes, a larger forgetting factor is selected to maintain the stability of parameter estimation.

[0062] During the system initialization phase, an offline identification method is used to obtain initial estimates of the parameter matrix based on historical operating data. The length of the historical data required for initialization is determined according to the dimension of the parameter matrix. Specifically, the number of historical data samples is not less than a preset multiple of the total number of parameters to be estimated in the parameter matrix. The preset multiple is pre-set based on the coverage of typical operating conditions in the historical operating data, the correlation between the components of the regression vector, the allowable range of offline identification error of the parameter matrix, and the allowable range of prediction state error, so that the parameter matrix obtained by offline identification has a stable estimation result and the initial prediction error of the shadow predictor meets the preset prediction accuracy requirements. The training process of the shadow predictor is divided into two parts: an offline initialization phase and an online update phase.

[0063] During the offline initialization phase, operational data containing various typical operating conditions is first extracted from the historical operational database. The data length should cover at least one complete production cycle to ensure the representativeness of the data. The extracted historical data is preprocessed, including removing outliers, filling missing values, and standardizing. Then, a training dataset is constructed, using the augmented state vector and control quantity at each time step as input features and the actual state value at the next time step as the output label.

[0064] The initial estimate of the parameter matrix is ​​obtained using the batch least squares method. Specifically, the input features of all training samples are stacked row-wise to form a data matrix, and the corresponding output labels are stacked row-wise to form a target matrix. The initial estimate of the parameter matrix is ​​equal to the inverse of the matrix obtained by multiplying the transpose of the data matrix by the inverse of the data matrix, then multiplying by the transpose of the data matrix, and finally multiplying by the target matrix. In one embodiment of this application, the formula for calculating the initial estimate of the parameter matrix is:

[0065] ;

[0066] in, For the stage The initial estimate of the parameter matrix, For the stage The data matrix Transpose the data matrix. This is the inverse matrix of the matrix obtained by multiplying the transpose of the data matrix by the data matrix. For the stage The target matrix.

[0067] During the online update phase, after the system is put into operation, the shadow predictor continuously updates the parameter matrix based on the real-time collected operating data. The update method adopts the above-mentioned recursive least squares algorithm with forgetting factor.

[0068] During the online update phase, no manual intervention is required. The system automatically adjusts the model parameters based on the latest process data, achieving adaptive tracking of changes in process characteristics.

[0069] A two-layer self-optimizing control structure is established, comprising a local layer and a global layer. The local layer includes N stage strategies, each generating a control quantity based on the enhanced state vector of its stage. The stage strategy employs a linear strategy function; the control quantity for stage i equals the strategy gain matrix multiplied by the enhanced state vector of that stage, plus the baseline control quantity. The strategy gain matrix represents the learnable parameters of the strategy for that stage. The baseline control quantity is calculated by the baseline controller based on the actuator state data and process setpoints for that stage, ensuring basic operation is maintained even when the strategy gain matrix is ​​zero. The reason for establishing this two-layer self-optimizing control structure is that multi-stage chemical reaction processes simultaneously require both local control and global coordination, which a single-layer control structure cannot adequately address. The local layer is responsible for rapid response control at each stage, enabling timely responses to state fluctuations and disturbances within each stage. The global layer is responsible for cross-stage resource coordination and target allocation, optimizing overall performance from a global perspective. This hierarchical structure achieves a reasonable decomposition of the control task. The local layer handles local control problems with a faster update frequency, while the global layer handles global optimization problems with a slower coordination frequency. The two layers exchange information through resource quotas and reference setpoints, ensuring both timely control and global coordinated optimization. The use of linear policy functions for stage strategies is due to their simple structure, few parameters, high computational efficiency, and analytical gradient calculation, which facilitates online learning and parameter updates. Although the expressive power of linear policy functions is limited, careful design of enhanced state vectors incorporates key information such as inter-stage coupling, time delay effects, target deviation, and model uncertainty estimation into the input features, enabling the linear policy functions to generate effective control decisions based on these rich features. The introduction of a reference control variable ensures the basic operability of the system. Even when the policy gain matrix has not yet converged in the early stages of policy learning, the system can maintain stable operation under the action of the reference control variable, avoiding safety risks during the learning process.

[0070] The global layer includes a coordinator that operates at a relatively slow preset global coordination cycle. The preset global coordination cycle is much larger than the maximum value among all stage update cycles. The preset global coordination cycle is determined based on the response speed requirements of global resource allocation. Specifically, it is obtained by analyzing the propagation time of the coupling effect between each stage and the execution cycle of resource allocation, so as to ensure that the coordination decision can be adjusted in the next round only after each stage has fully responded. The coordinator is responsible for allocating cross-stage shared resources and setting inter-stage collaboration goals.

[0071] Step 400: At the update time, candidate control variables are calculated according to the stage strategy. Based on the shadow predictor, the candidate control variables are sequentially subjected to constraint projection, rate limiting, and security prediction processing to obtain the final control variable, specifically as follows: Figure 2 As shown.

[0072] When the global scheduler triggers a phase i update, that is, when the update time of that phase is reached, the candidate control quantity is calculated based on the policy function of that phase. The candidate control quantity is equal to the policy gain matrix multiplied by the enhanced state vector of that phase at the update time, plus the baseline control quantity.

[0073] Set a reward function for stage i, which includes downstream responsibility items to address cascading effects. The value of the reward function is equal to the negative sum of the operating cost item, local energy consumption item, global quality item, and predicted quality item. The operating cost item is the operating cost of this stage multiplied by a preset operating cost weighting coefficient. The operating cost includes material consumption, catalyst loss, etc. The preset operating cost weighting coefficient is determined based on the priority of production targets and the proportion of operating costs in the total cost. Specifically, it is obtained by comparing the quality, energy consumption, and cost targets pairwise and calculating the weight vector using the analytic hierarchy process. The local energy consumption item is the local energy consumption of this stage multiplied by a preset local energy consumption weighting coefficient. The preset local energy consumption weighting coefficient is determined based on the importance of energy-saving targets and the economic cost per unit of energy consumption. Specifically, it is obtained by analyzing the marginal impact of energy consumption changes on the total production cost and combining it with the enterprise's energy-saving assessment indicators. The global quality term is the square of the Euclidean distance between the current output quality data and the global quality target multiplied by a preset quality deviation weighting coefficient. The preset quality deviation weighting coefficient is determined based on the degree of impact of product quality on economic benefits, specifically obtained by analyzing the relationship between quality deviation and product degradation or scrapping losses. The predicted quality term is the sum of the squares of predicted quality deviations of all downstream stages multiplied by a preset downstream responsibility weighting coefficient. The preset downstream responsibility weighting coefficient is determined based on the sensitivity of the impact of upstream operations on downstream quality, specifically obtained by analyzing the correlation between changes in upstream operations and downstream quality deviations in historical data. The downstream stage refers to all stages in the directed graph where stage i is the starting point of the directed edge and other stages are the ending points of the directed edges; that is, if there is a directed edge from stage i to stage j, then stage j is the downstream stage of stage i.

[0074] The reason for introducing a downstream responsibility term into the reward function is that, in a multi-stage chemical reaction, the operations of the upstream stage are transmitted to the downstream stage through material and energy flows. If the upstream stage only optimizes its own local performance and ignores the impact on the downstream stage, it may lead to the deterioration of the quality of the downstream stage, thereby affecting the quality of the final product. By adding the predicted quality deviation term of the downstream stage to the reward function of the upstream stage, the upstream stage can anticipate and consider the impact of its operations on the downstream stage when making control decisions, thus actively avoiding adverse effects on the downstream stage. This design effectively solves the cascading effect problem caused by each stage acting independently in traditional decentralized control, and realizes implicit coordination between stages. The calculation method of the predicted quality deviation of each downstream stage is as follows: First, using the shadow predictor of the downstream stage, with the current enhanced state vector and control quantity of the downstream stage as input, the predicted quality deviation from the current time to the current time plus the deviation from stage i to the downstream stage is obtained through iteration. The propagation time lag of each stage is used to determine the predicted state at each moment. In each iteration, the predicted state at the previous moment is used as the enhanced state vector input for the shadow predictor at the next moment. Then, the predicted state at the final moment is converted into the predicted quality output through the quality output mapping function. The quality output mapping function is pre-calibrated according to the process characteristics of each stage. For the reaction stage, the mapping function maps state variables such as temperature, pressure, and residence time to conversion rate or selectivity. For the separation stage, the mapping function maps state variables such as reflux ratio and top temperature to product purity. Finally, the Euclidean distance between the predicted quality output and the quality target of the downstream stage is calculated as the predicted quality deviation. The above four preset weight coefficients are all non-negative real numbers, determined according to the priority of production targets and the relative importance of each indicator. Specifically, they are obtained by comparing the targets such as quality, energy consumption, and cost pairwise and calculating the weight vector through the analytic hierarchy process, or by calibrating based on the experience of process experts and historical optimization results.

[0075] A deterministic policy gradient method is employed to update the policy gain matrix online. The reason for choosing the deterministic policy gradient method is that it can directly perform policy optimization in the continuous action space without discretizing the action space, making it very suitable for optimizing continuous control variables such as valve opening and temperature setpoints in chemical reactions. Compared with stochastic policy gradient methods, the deterministic policy gradient method has a smaller gradient estimation variance, faster convergence speed, and does not require sampling of the policy distribution, resulting in higher computational efficiency. Another advantage of this method is its good compatibility with linear policy functions. Since the gradient of a linear policy function has an analytical form, the calculation of the deterministic policy gradient can be completed efficiently, meeting the requirements for online real-time updates. The phased policy adopts a linear policy function. The gradient of the control quantity with respect to the policy gain matrix is ​​equal to the transpose of the augmented state vector. The gradient of the reward function with respect to the policy gain matrix is ​​calculated using the chain rule, specifically by multiplying the gradient of the reward function with respect to the control quantity by the gradient of the control quantity with respect to the policy gain matrix. The gradient of the reward function with respect to the control quantity is estimated using the numerical difference method, specifically as follows: A small perturbation is applied to each component of the control quantity. The perturbation amount is determined based on the full-scale range of the control quantity and the accuracy requirements of the numerical difference method. Specifically, it is obtained by selecting a value that is sufficiently small to ensure linear approximation accuracy while being sufficiently large to avoid numerical rounding errors. The selection of this perturbation amount needs to take into account the bias and variance of the gradient estimation. The reward function value after applying a positive perturbation and the value after applying a negative perturbation are calculated. The difference in reward function values, divided by twice the perturbation amount, yields the partial derivative of the reward function with respect to that component of the control quantity. The partial derivatives of all components are then arranged sequentially according to the original order of the control components to obtain the gradient vector of the reward function with respect to the control quantity. Specifically, the first position of the gradient vector stores the partial derivative corresponding to the first component of the control quantity, the second position stores the partial derivative corresponding to the second component, and so on, until all partial derivatives are included. The reason for using the numerical difference method to estimate the gradient of the reward function with respect to the control quantity is that the reward function involves multiple stages of prediction quality calculation and complex constraint penalty terms, making its analytical gradient difficult to obtain directly. The numerical difference method applies small perturbations to each component of the control quantity and observes the results. The change in the excitation function value can approximate the gradient direction. Although the accuracy is slightly lower than that of the analytical gradient, it is simple to implement and widely applicable. The update rule for the policy gain matrix is ​​that the new policy gain matrix is ​​equal to the outer product of the gradient vector of the reward function with respect to the control quantity and the transpose of the augmented state vector, multiplied by the preset learning rate, plus the original policy gain matrix. The preset learning rate is a positive real number, which is determined according to the policy convergence speed and stability requirements. Specifically, it is selected by performing parameter sensitivity analysis in the simulation environment to obtain a faster learning speed while ensuring convergence. Too large a learning rate will lead to policy oscillation, and too small a learning rate will lead to slow convergence. The training process of the policy gain matrix adopts an online incremental learning method, which does not require the pre-collection of a large amount of training data.During system initialization, each element of the policy gain matrix is ​​initialized to a small random number close to zero. At this stage, the control input is mainly determined by the baseline control input, and the system operates stably under the action of the baseline controller. As the system runs, the policy gain matrix is ​​incrementally updated at each update time based on the current enhanced state vector and the gradient of the reward function. After multiple updates, the policy gain matrix gradually converges to the optimal value that improves control performance. The advantage of this online learning method is that the policy can continuously adapt to changes in process characteristics without offline retraining, and the learning process is synchronized with normal production operation, without affecting production continuity.

[0076] Candidate control quantities are constrained and projected to obtain projected control quantities, ensuring that the control quantities satisfy actuator constraints. The reason for constraining and projecting candidate control quantities is that the candidate control quantities generated by the strategy function may exceed the physical operating range of the actuator. For example, the valve opening cannot be less than zero or greater than 100%. If control quantities exceeding the range are directly issued to the actuator, it may cause the actuator to fail to respond, or even damage the actuator or cause a safety accident. Constraint projection maps candidate control quantities to the feasible region, ensuring that the issued control quantities are always within the effective operating range of the actuator, which is a basic measure to ensure the safe operation of the system. The control quantity constraint set of stage i is set as the set of all control quantities whose components are between their corresponding lower and upper limits. The projected control quantities are obtained through the projection operator. The projection operator processes each component of the candidate control quantity one by one. If the value of a component is less than the corresponding lower limit, the value of the component is set to the lower limit. If the value of a component is greater than the corresponding upper limit, the value of the component is set to the upper limit. If the value of a component is between the corresponding lower and upper limits, the value of the component remains unchanged.

[0077] Rate limiting is applied to the projected control quantity to obtain the rate-limited control quantity. The purpose of rate limiting is to prevent drastic changes in the control quantity from impacting the actuator and the process. The reason for rate limiting the projected control quantity is that even if the control quantity itself is within the actuator's operating range, if the control quantity changes too quickly, it will adversely affect the actuator and the process. For the actuator, excessively rapid action commands may exceed its mechanical response capability, causing the actuator to be unable to track accurately or to cause mechanical wear. For the process, drastic changes in the control quantity will cause drastic fluctuations in the process state, which may trigger safety interlocks or lead to unstable product quality. Rate limiting, by constraining the change range between two adjacent control quantities, allows the control quantity to change in a smooth and gradual manner, protecting the actuator and maintaining the stable operation of the process. The rate-limited control quantity is equal to the control quantity of the previous update cycle plus an increment, which is the sum of the projected control quantity and the... The result of the difference in control values ​​from the previous update cycle after processing by a saturation function is as follows. The control value from the previous update cycle is obtained by reading the actuator status data at that moment. The saturation function limits the magnitude of the input vector to within the upper limit of the preset control value change rate for this stage. The processing method of the saturation function is as follows: First, the magnitude of the input vector is calculated. If the magnitude of the input vector is less than or equal to the upper limit of the preset control value change rate, the output of the saturation function is equal to the input vector itself. If the magnitude of the input vector is greater than the upper limit of the preset control value change rate, the output of the saturation function is a vector with the same direction as the input vector and a magnitude equal to the upper limit of the preset control value change rate. This output vector is obtained by dividing the input vector by its magnitude and then multiplying it by the upper limit of the preset control value change rate. The upper limit of the preset control value change rate is determined based on the physical response capability of the actuator and the process safety requirements. Specifically, it is obtained by referring to the maximum operating rate in the actuator's technical specifications and combining it with the process requirements for control stability.

[0078] Safety prediction is achieved using shadow predictors. The reason for using shadow predictors is that traditional feedback control methods can only take corrective measures after a state boundary violation occurs. However, certain state boundary violations in chemical reactions can lead to serious safety accidents or irreversible product quality losses, and post-event correction is often too late. Safety prediction, by using shadow predictors to predict the state trajectory over a future period, can identify potential risks before a state boundary violation actually occurs, thus allowing for preventative measures to be taken in advance. This proactive safety control method transforms safety assurance from passive response to proactive prevention, improving system security. A preset prediction time domain is set as the time step for safety prediction. The preset prediction time domain is determined based on the advance warning requirement for safety constraint violations, specifically through analysis... The minimum time required to detect a potential risk of exceeding limits and execute a corrective action is converted into the corresponding number of time steps, while considering the characteristic that prediction accuracy decreases with the number of prediction steps. For each moment within the preset prediction time domain, from step 1 to the last step of the preset prediction time domain, the rate-limited control variable is used as the control input during the prediction period, and the shadow predictor iteratively calculates the prediction state sequence for that stage. If there exists a moment where the predicted state is not within the state safety constraint set for that stage, conservative scaling is performed. The state safety constraint set is set as the set of all states whose components of the state vector are between their corresponding lower and upper safety limits. The distance from the predicted state to the boundary of the safety constraint set is calculated using the maximum violation of the component, specifically as follows:

[0079] For each component of the predicted state vector, if the component exceeds the corresponding safety upper limit, the violation amount of the component is equal to the component value minus the safety upper limit; if the component is below the corresponding safety lower limit, the violation amount of the component is equal to the safety lower limit minus the component value; if the component is between the safety lower limit and the safety upper limit, the violation amount of the component is zero; the distance from the predicted state to the boundary of the safety constraint set is equal to the maximum value among all component violations.

[0080] The final control quantity after conservative scaling equals the rate-limited control quantity multiplied by the conservative term, plus the baseline term. The conservative term is the difference between 1 and the safety conservative coefficient, and the baseline term is the baseline control quantity multiplied by the safety conservative coefficient. The safety conservative coefficient is a real number between 0 and 1. The calculation of the safety conservative coefficient uses a progressive scaling strategy. First, the original conservative coefficient is calculated, which equals the maximum distance from the predicted state to the boundary of the safety constraint set at all times within the preset prediction time domain, divided by the preset safety margin parameter. Then, the original conservative coefficient is smoothed, and the safety conservative coefficient equals the smaller value between the square of the original conservative coefficient and 1. This squaring process ensures that the conservative coefficient increases slowly when the risk of exceeding the limit is small, and only increases rapidly when the risk of exceeding the limit is large. To avoid excessive conservatism, a gradual scaling strategy is adopted because if a simple threshold judgment method is used, the control quantity will change abruptly when the predicted state approaches the safety boundary. This abrupt change itself may cause drastic fluctuations in the process state. The gradual scaling strategy achieves a smooth transition of the control quantity through a square function. Only slight adjustments are made when the risk of exceeding the boundary is small, and contraction is only performed when the risk of exceeding the boundary is large, which ensures both safety and maintains the stability of control. The preset safety margin parameter is determined based on the statistical distribution of the model prediction error. The state safety constraint range is set as the mean of each component of the difference between the upper and lower safety limits. Specifically, it is obtained by calculating the upper bound of the confidence interval of historical prediction errors and leaving an appropriate margin to ensure safety even when there are prediction errors.

[0081] If the predicted state at all times within the preset prediction time domain is within the set of state safety constraints, i.e. there is no risk of exceeding the limit, then the final control quantity is equal to the control quantity after the rate limit.

[0082] Step 500: When the preset global coordination cycle arrives, the coordinator performs shared resource allocation optimization to obtain the optimal resource quota for each stage, and converts the optimal resource quota into stage constraint updates and stage reference settings for each stage, so that each stage can determine and issue the final control quantity at the update time.

[0083] When the preset global coordination cycle arrives, the coordinator performs shared resource allocation optimization. The reason for this optimization is that in multi-stage chemical reactions, each stage typically shares common utility resources such as steam, cooling water, and electricity. The total amount of these resources is limited by equipment capacity and supply capability. If each stage independently determines its resource usage without considering global constraints, the total resource demand may exceed the supply capacity, leading to resource contention and supply shortages. By collecting resource demand information from each stage and optimizing allocation under global constraints, the coordinator can maximize the comprehensive benefits of each stage while meeting total resource constraints, achieving efficient resource utilization. Each stage reports a marginal benefit curve based on its current operating status. This curve represents the expected benefit of stage i when it obtains its resource quota. The resource quota for stage i refers to the amount of resources available within the current global coordination cycle. The shared resource usage allocated to stage i includes one or more of steam, cooling water, and electricity. The marginal revenue curve is fitted using a quadratic function, where the marginal revenue of stage i equals the quadratic coefficient multiplied by the square of the resource quota, the linear coefficient multiplied by the resource quota, and the sum of the constant term. The reason for using a quadratic function to fit the marginal revenue curve is that it can better characterize the economic law of diminishing marginal revenue, that is, as the resource quota increases, the incremental revenue brought by each unit of resource gradually decreases. This diminishing characteristic is consistent with the actual situation of chemical reaction processes. For example, increasing the steam supply can increase the reaction temperature and thus accelerate the reaction rate, but when the temperature approaches the optimal value, the revenue from further increasing the steam supply will decrease. Another advantage of the quadratic function is its convexity, which ensures that the resource allocation optimization problem has a unique global optimal solution, making it easy to solve.

[0084] The quadratic coefficient, linear coefficient, and constant term are the fitting coefficients, which are estimated online by the least squares method based on historical operating data for each stage. The specific estimation method is as follows: Collect data pairs of resource quotas and corresponding actual revenue values ​​within the most recent preset historical window length for that stage. The preset historical window length is determined based on the statistical stability requirements of resource quota and revenue data to ensure that the sample size required for fitting is sufficient and the data is timely.

[0085] Actual revenue refers to the comprehensive economic benefit calculated based on actual production and operation data within a coordination period after obtaining a specific resource quota during that stage. This comprehensive economic benefit is calculated as the product output value during that stage within the coordination period minus the operating costs and local energy consumption costs. Specifically, the product output value equals the product of the qualified product output and the unit price, with the unit price determined based on current market prices or internal accounting standards. Operating costs include material costs and catalyst depletion costs incurred during the coordination period. Material costs are calculated based on the feed flow rate and raw material unit price, while catalyst depletion costs are calculated based on catalyst usage time and replacement cycle. Local energy consumption costs equal the steam, electricity, and cooling water consumption during the coordination period multiplied by their respective values. The energy consumption of each component is obtained by summing the unit energy prices. The energy consumption of each component is directly obtained from the operational data of this stage. The unit energy price is determined according to the accounting standards of the enterprise's public works. The output value, operating cost, and local energy consumption cost of the above products are all normalized according to the length of the coordination cycle, that is, each value is divided by the length of the coordination cycle, so that the actual benefit value is expressed as the comprehensive economic benefit per unit time, thereby eliminating the impact of the difference in the length of different coordination cycles on the benefit value. At the end of each coordination cycle, each stage forms a data pair based on the actual resource quota obtained in the coordination cycle and the actual benefit value calculated in the above manner, and stores the data pair in the historical data buffer of the stage. The historical data buffer is managed in a first-in, first-out manner. When the number of data pairs in the buffer exceeds the preset historical window length, the earliest stored data pair is removed.

[0086] When constructing the data matrix, all data pairs within the most recent preset historical window length are read from the historical data buffer of that stage. Each row of the data matrix consists of the square of the resource quota in the data pair, the resource quota in the data pair, and the constant 1 arranged in sequence. The number of rows in the data matrix is ​​equal to the number of data pairs currently stored in the historical data buffer. When constructing the profit vector, each element of the profit vector is the actual profit value of the corresponding data pair in the historical data buffer. The order of the elements in the profit vector is consistent with the order of the rows in the data matrix. The least squares method is used to solve for the fitting coefficients. The fitting coefficient vector is equal to the inverse matrix obtained by multiplying the transpose of the data matrix and the data matrix, then multiplying by the transpose of the data matrix, and finally multiplying by the profit vector. To ensure that the marginal profit curve is a convex function, if the obtained quadratic coefficient is negative, it is corrected to a preset minimum quadratic coefficient. The preset minimum quadratic coefficient is set to a small positive value according to the numerical stability requirements. In one embodiment of this application, the fitting coefficient vector... The calculation formula is:

[0087] ;

[0088] in, For the stage The data matrix Transpose the data matrix. This is the inverse matrix of the matrix obtained by multiplying the transpose of the data matrix by the data matrix. For the stage The profit vector.

[0089] The coordinator solves a resource allocation optimization problem with resource quotas at each stage as decision variables, based on the marginal revenue curves of each stage. The optimization objective is to maximize the negative value of the sum of marginal revenues across all stages, i.e., to minimize negative revenues. The sum of marginal revenues is the sum of the function values ​​of the marginal revenue curves at a specific resource quota for each stage. Constraints include that the sum of resource quotas across all stages does not exceed the upper limit of the total shared resources. This upper limit is determined based on the equipment capacity and supply capacity of the public works system, specifically by consulting the rated steam production of the steam boiler, the maximum water supply of the cooling water circulation system, and the maximum power supply of the substation. The resource quota is obtained by combining the nameplate parameters of the equipment such as load with the safety margin coefficient in actual operation; the resource quota for each stage is between the lower limit and the upper limit of the resource quota for that stage. The lower limit of the resource quota is determined based on the minimum resource consumption required to maintain the minimum safe operation in that stage, which is obtained by analyzing the minimum resource supply required for each actuator to maintain basic functions under the minimum load condition in that stage. The upper limit of the resource quota is determined based on the maximum processing capacity and safe operation restrictions of the equipment in that stage, which is obtained by referring to the rated capacity of each resource-related equipment in that stage and combining it with the maximum allowable load in the process safety regulations.

[0090] Before solving the resource allocation optimization problem, the fitting quality of the marginal revenue curve is first checked. The goodness-of-fit index is calculated, which is equal to 1 minus the sum of squared residuals divided by the total sum of squares. The sum of squared residuals is the sum of squares of the differences between the actual revenue value and the fitted revenue value of each data point, and the total sum of squares is the sum of squares of the differences between the actual revenue value and the mean revenue value of each data point. When the goodness-of-fit index is less than the preset goodness-of-fit threshold, the fitting quality is deemed unqualified, and a backup resource allocation strategy is adopted. The preset goodness-of-fit threshold is determined according to the fitting accuracy requirements and data quality characteristics to ensure the reliability of resource allocation based on the fitted curve. The backup resource allocation strategy is a historical average allocation strategy, that is, the resource quota of each stage is equal to the average of the resource quota within the most recent preset historical window length of that stage, and the quota of each stage is adjusted proportionally so that its sum does not exceed the upper limit of the total shared resources.

[0091] This optimization problem is a convex quadratic programming problem. Lagrange multipliers are introduced, and the derivative of the resource quota at each stage is taken and set to zero. Combined with the total resource constraint, the value of the Lagrange multipliers is determined through binary search to obtain the optimal resource quota, which is then projected onto the interval between the lower and upper limits of the resource quota at each stage. The reason for using the Lagrange multiplier method combined with binary search to solve the resource allocation optimization problem is that this method can efficiently solve convex optimization problems with equality constraints, and has low computational complexity and fast convergence speed. The physical meaning of the Lagrange multiplier is the marginal price of resources, that is, the increase in total revenue that can be obtained by adding one unit of total resources under the current allocation scheme. Through binary search... The binary search method determines the Lagrange multiplier value that exactly satisfies the total resource constraint, enabling the finding of the optimal allocation scheme with equal marginal returns at each stage, thus achieving Pareto optimal resource allocation. The specific implementation of the binary search is as follows: A lower and upper bound for the Lagrange multiplier are set. The initial value of the lower bound is 0, and the initial value of the upper bound is determined based on the maximum absolute value of the coefficients of the linear terms across all stages and the characteristics of the marginal return curve. Specifically, it is obtained by analyzing the theoretical range of the Lagrange multiplier and selecting an upper bound value that ensures the search range covers the optimal solution. In each iteration, the intermediate value of the Lagrange multiplier is calculated; this intermediate value is equal to half the sum of the lower and upper bounds. Calculate the candidate resource quotas for each stage based on the median value. The candidate resource quota for stage i is equal to the negative value of the linear coefficient of that stage minus the median value, then divided by twice the quadratic coefficient of that stage. Project the candidate resource quotas for each stage into the corresponding lower and upper limit ranges of the resource quotas. If the candidate resource quota is less than the lower limit, take the lower limit value; if it is greater than the upper limit, take the upper limit value; otherwise, leave it unchanged. Calculate the sum of the resource quotas for all stages after projection. If the sum is greater than the upper limit of the total shared resources, update the search lower bound to the median value; if the sum is less than the upper limit of the total shared resources, update the search upper bound to the median value; if the sum is less than the upper limit of the total shared resources, update the search upper bound to the median value. The search ends when the total amount of shared resources is equal to the upper limit; the search ends when the difference between the upper and lower bounds is less than the preset search precision. The preset search precision is determined based on the upper limit of the total amount of shared resources and the resource allocation precision requirements. Specifically, it is obtained by analyzing the minimum effective adjustment amount of the resource quota and selecting a search termination condition that can meet the allocation precision. After the search ends, the final intermediate value is used as the optimal Lagrange multiplier to calculate and output the optimal resource quota for each stage. The coordinator converts the optimal resource quota for each stage into stage constraint updates and stage reference settings, and distributes them to the local layer of each stage to play a constraint and guidance role in the control calculation at subsequent update times.

[0092] Stage constraint update refers to the coordinator dynamically adjusting the upper limit of the resource-related components in the control quantity constraint set used in the constraint projection stage of each stage based on the optimal resource quota obtained in each stage. The purpose of stage constraint update is to transform the global total resource limit into control quantity boundary conditions that can be directly executed at the local layer of each stage, so that when each stage calculates candidate control quantities and performs constraint projection at subsequent update times, the control quantity components related to resource consumption in each stage will not exceed the actuator working range corresponding to the resource quota allocated by the coordinator. The specific implementation of stage constraint update is as follows: The coordinator first determines the new upper limit value of the control quantity components related to resource consumption in each stage based on the optimal resource quota of each stage and the correspondence between resource consumption and control quantity in that stage. This correspondence is determined by the process characteristics and equipment parameters of that stage. For example, the steam resource quota is determined by the steam... The flow characteristic curve of the valve is converted into the upper limit of the steam valve opening, and the cooling water resource quota is converted into the upper limit of the cooling water valve opening through the flow characteristic curve of the cooling water valve. Then, the coordinator sends the calculated new upper limit value to the local layer of this stage. After receiving the new upper limit value, the local layer updates the upper limit of the corresponding component in the control constraint set of this stage to the new upper limit value, while the upper limit of the component unrelated to resource consumption and the lower limit of all components in the control constraint set remain unchanged. In subsequent update times, when this stage calculates candidate control quantities according to the strategy function and performs constraint projection, the projection operator will perform component-by-component truncation processing on the candidate control quantities according to the updated control constraint set, thereby ensuring that the component related to resource consumption in the final control quantity actually issued in this stage does not exceed the range allowed by the resource quota allocated by the coordinator, realizing the effective execution of global resource constraints at the local layer.

[0093] The stage reference setpoint refers to the coordinater's conversion of the optimal resource quota obtained in each stage into the expected operating value of each state variable in that stage through a mapping function from resource quota to control setpoint. This value is then distributed as a new reference setpoint to the local layer of that stage. The purpose of the stage reference setpoint is to provide the local layer of each stage with a control objective that matches the global resource allocation scheme. This ensures that when each stage constructs an enhanced state vector in subsequent updates, the target deviation vector reflects the operating objective set by the coordinator based on the global optimization results, thereby guiding the control strategy of each stage to adjust towards the global optimum. The specific conversion method of the stage reference setpoint is as follows: The mapping function from resource quota to control setpoint is determined based on the resource type and the process characteristics of that stage. For steam resources, this mapping function converts the steam quota into a steam valve opening setpoint based on the flow characteristic curve of the steam valve. For cooling water resources, this mapping function converts the cooling water quota into a cooling water valve opening setpoint based on the flow characteristic curve of the cooling water valve. The above mapping relationship is verified through valve flow... Characteristic curves are obtained through calibration. The coordinator further maps the converted control setpoints to the expected operating values ​​of the corresponding state variables through the process model of this stage. For example, the steam valve opening setpoint is mapped to the expected operating value of the reaction temperature through the heat balance relationship, and the cooling water valve opening setpoint is mapped to the expected operating value of the cooling section outlet temperature through the heat transfer model. The coordinator sends the above expected operating values ​​as new reference setpoints to the local layer of this stage. After receiving the new reference setpoints, the local layer uses the new reference setpoints to replace the original reference setpoints when constructing the enhanced state vector at subsequent update times to calculate the target deviation vector. That is, each component in the target deviation vector is updated to the difference between the current actual value of each state variable in this stage and the new reference setpoints sent by the coordinator. Since the target deviation vector is a component of the enhanced state vector, and the enhanced state vector is the input of the stage strategy function, the update of the reference setpoints will directly affect the result of the strategy function calculating the candidate control quantity, so that the control strategy of each stage can automatically track the global coordination target set by the coordinator.

[0094] Stage constraint updates and stage reference setpoints work together in the control system. Stage constraint updates limit the feasible range of control variables in each stage through hard constraints to ensure that resource usage does not exceed the global allocation limit. Stage reference setpoints adjust the control objectives of each stage through soft guidance to encourage each stage to approach the global optimal operating point. Together, they realize the effective transmission of the optimization decision of the global coordinator to the local control execution.

[0095] A conflict detection metric is set up to detect the negative impact of upstream stage behavior on the quality of downstream stages. The reason for setting up this metric is that, although the reward function already includes downstream responsibility items, due to errors in the prediction model and potential inherent contradictions in the optimization objectives of each stage, upstream operations may still have unexpected negative impacts on the downstream stage. By monitoring the quality deviation of the downstream stage in real time, the conflict detection metric can promptly identify such negative impacts and trigger intervention measures from the coordinator, thus supplementing and strengthening the downstream responsibility mechanism. The conflict detection metric employs a two-way deviation detection mechanism. When the absolute value of the difference between the predicted quality output and the downstream stage's quality target exceeds a preset conflict judgment threshold, the conflict detection metric is set to 1, indicating a conflict has been detected; otherwise, it is set to 0, indicating no conflict has been detected. This two-way detection mechanism considers both excessively low and excessively high quality, where excessively high quality may correspond to undesirable conditions such as over-reaction or excessive impurities. The preset conflict judgment threshold is determined based on the sensitivity of the quality metric and the sensitivity requirements of conflict detection, specifically by analyzing the correlation between quality deviation and upstream operational changes in historical data and combining it with the tolerance range of quality control. When a conflict is detected, the coordinator increases the upstream stage's predicted quality output. Let's define a downstream responsibility weight coefficient. The new preset downstream responsibility weight coefficient is equal to the original preset downstream responsibility weight coefficient multiplied by 1 plus the preset weight increase coefficient. Simultaneously, the resource cap for the upstream stage is tightened. The new resource cap is equal to the original resource cap multiplied by 1 minus the preset resource tightening coefficient. The reason for increasing the preset downstream responsibility weight coefficient when a conflict is detected is that the occurrence of a conflict indicates that the current downstream responsibility weight is insufficient to effectively constrain the behavior of the upstream stage. By dynamically increasing this weight coefficient, the upstream stage's attention to the impact on the downstream can be enhanced, prompting the upstream stage to be more cautious in subsequent decisions. The reason for tightening the upstream stage's resource cap is... The reason is that resource usage is often positively correlated with the degree of impact on downstream processes. By limiting resource usage in the upstream stage, interference to the downstream stage can be reduced from the source. The preset weight increase coefficient is a positive real number, which is determined according to the response intensity requirements based on the severity of the conflict. Specifically, it is obtained by simulation analysis of the impact of different increase rates on the conflict resolution speed and system stability. The preset resource tightening coefficient is a real number greater than 0 and less than 1. This value is determined according to the smoothness requirements of resource constraint adjustment. Specifically, it is obtained by analyzing the impact of resource quota changes on the operating status of each stage and selecting a value that can effectively suppress conflicts while avoiding excessive constraints.

[0096] The model uncertainty estimation vector is continuously monitored at each stage. This monitoring is crucial because the model uncertainty estimation vector is a key indicator of the reliability of the prediction model. A significant increase in the values ​​of each component of the model uncertainty estimation vector indicates that the current prediction model can no longer accurately describe the dynamic behavior of the process, rendering control decisions and safety predictions based on this model unreliable. By continuously monitoring the model uncertainty estimation vector and triggering adaptive operations when it exceeds a threshold, timely responses to changes in process characteristics or model mismatch issues can be provided, ensuring the robustness of the control system. The overall model uncertainty of stage i at time t is set as the mean of each component of the model uncertainty estimation vector for that stage. When the overall model uncertainty of stage i at time t exceeds the preset uncertainty threshold for that stage, and this estimated value is greater than the estimated value from the previous update cycle, adaptive operations are executed. The preset uncertainty threshold is determined based on model accuracy requirements and process characteristics, specifically by statistically analyzing the distribution of overall model uncertainty under normal operating conditions and selecting its mean plus a certain multiple of the standard deviation as the threshold.

[0097] Before performing adaptive operations, the cause of the increase in the uncertainty of the integrated model is first determined. If the rate of change of the process state variables exceeds the preset threshold for the rate of change of the state variables while the uncertainty of the integrated model increases, it is determined that the increase in the uncertainty of the integrated model is caused by dynamic changes in the process, and dynamic tracking operations are performed. Otherwise, it is determined that the increase in the uncertainty of the integrated model is caused by model mismatch, and conservative operations are performed. The significance of distinguishing the cause of the increase in the uncertainty of the integrated model is that different coping strategies are required for different causes of the increase in the uncertainty of the integrated model. If the increase in the uncertainty of the integrated model is caused by dynamic changes in the process, it means that the process is undergoing a change in operating conditions or load. At this time, the update speed of the model parameters should be accelerated to track the new process characteristics. If the increase in the uncertainty of the integrated model is caused by model structure mismatch, it means that the current linear model can no longer accurately describe the process behavior. At this time, a conservative strategy should be adopted to reduce the risk. The preset threshold for the rate of change of the state variables is determined based on the nominal value and update cycle of the state variables at each stage, and is specifically obtained by analyzing the statistical distribution of the rate of change of the state variables under normal operating conditions.

[0098] The dynamic tracking operation includes increasing the decay magnitude of the preset forgetting factor. The new preset forgetting factor is equal to the original preset forgetting factor multiplied by the preset forgetting factor decay coefficient. The preset forgetting factor decay coefficient is determined according to the model's tracking response speed requirements to enhance the model's ability to track new data. The preset learning rate is kept constant or appropriately increased to accelerate the policy's adaptation to new operating conditions. The reason for increasing the preset forgetting factor decay magnitude in the dynamic tracking operation is that a smaller forgetting factor can assign higher weights to recent data, allowing the model parameters to adapt to new process characteristics more quickly, thereby shortening the model's tracking response time. The reason for keeping or increasing the preset learning rate is that when the process is in a dynamic change phase, the policy also needs to be adjusted accordingly to adapt to new operating conditions, and a larger learning rate can accelerate the policy's adaptation speed.

[0099] Conservative operations include: reducing the preset learning rate (the new preset learning rate equals the original preset learning rate multiplied by 1 minus the preset learning rate decay coefficient); increasing the safety conservatism coefficient (the new safety conservatism coefficient equals the original safety conservatism coefficient plus the smaller value between the preset conservatism coefficient increment and 1); and extending the preset global coordination period (the new preset global coordination period equals the original preset global coordination period multiplied by 1 plus the preset period extension coefficient). The reason for reducing the preset learning rate in conservative operations is that when the uncertainty of the integrated model increases, the policy gradient calculated based on the model may contain errors. If the policy parameters are continued to be updated with the original learning rate, it may cause the policy to adjust in the wrong direction. Therefore, it is necessary to reduce the learning rate to reduce the magnitude of erroneous updates. Increasing the safety conservatism coefficient... The reason for the conservative coefficient is that increased uncertainty in the integrated model means a decrease in the reliability of safe predictions. Increasing the safety conservative coefficient allows for a larger safety margin in the event of inaccurate predictions. The reason for extending the preset global coordination period is that when the uncertainty of the integrated model at each stage is large, the marginal revenue curves reported at each stage may be inaccurate. Frequent resource reallocation in this case may lead to system oscillations. Extending the coordination period can reduce unnecessary adjustments and maintain system stability. The preset learning rate decay coefficient is a real number greater than 0 and less than 1. This value is determined based on the smoothness requirements of the learning rate adjustment, specifically obtained by analyzing the impact of different decay magnitudes on strategy stability. The preset conservative coefficient increment is a positive real number, and this value is based on... The required increase in safety margin is determined by analyzing the variation of prediction error as the uncertainty of the integrated model increases and selecting a value that can effectively compensate for the increase in prediction error. The preset period extension coefficient is a positive real number, which is determined based on the response characteristics of the coordination frequency adjustment, specifically by analyzing the impact of global coordination period changes on the overall system performance. When performance degradation is detected to exceed the preset degradation threshold (i.e., the difference between the current global objective function value and the global objective function value at a previous time in the evaluation period exceeds the preset degradation threshold), the system reverts to a stable strategy, replacing the current strategy with a saved stable strategy version. The reason for setting up performance degradation detection and strategy rollback mechanisms is that strategy parameters may change during online learning. Gradient estimation errors, model mismatch, and other factors can cause updates in the wrong direction, leading to a continuous deterioration in control performance. Without intervention, this performance degradation can accumulate to an unacceptable level. The policy rollback mechanism, by saving historically stable policy versions, can effectively prevent further performance degradation and ensure the basic operational performance of the system by promptly reverting to a stable policy when performance degradation is detected. The stable policy version is saved when the fluctuation range of the global objective function value is less than a preset stability fluctuation threshold within a consecutive preset stability determination period. The policy gain matrix of each stage is then saved as the stable policy version. The preset stability determination period is determined based on the stability period of the system's dynamic characteristics to ensure that the stability of the policy is confirmed within a sufficiently long observation window.The preset stability fluctuation threshold is determined based on the historical average of the global objective function and the statistical characteristics of the fluctuation of the objective function value during normal operation. Specifically, it is obtained by calculating the standard deviation of the global objective function value under normal operating conditions and selecting a threshold that can distinguish between normal and abnormal fluctuations. The preset degradation threshold is determined based on the tolerance for performance degradation. Specifically, it is obtained by statistically analyzing the fluctuation range of the global objective function value during normal operation and selecting values ​​exceeding the normal fluctuation range as the basis for degradation judgment.

[0100] After reverting to a stable strategy, the recovery evaluation mechanism determines when online learning can be reactivated. The reason for setting up this mechanism is that while strategy rollback ensures basic system performance, the stable strategy version may have been obtained under historical conditions and may not be the optimal strategy for the current situation. If the system remains in a stable strategy for an extended period without reactivating online learning, it will lose its ability to continuously optimize. The recovery evaluation mechanism monitors the uncertainty of the integrated model and the stability of performance indicators, and reactivates online learning when conditions permit, allowing the system to continue pursuing performance optimization while ensuring safety. The specific implementation of the recovery evaluation mechanism is as follows: After rollback, an observation period begins, the length of which is the preset number of recovery observation periods, determined based on the typical time required for the system to recover stability from a disturbance. During the observation period, the uncertainty of the integrated model and the global objective function value are continuously monitored. When both of the following recovery conditions are met simultaneously, online learning is reactivated, and the system recovers its performance. Condition 1 is that the uncertainty of the integrated model remains below a preset uncertainty threshold for a continuous preset number of stable uncertainty periods. The preset number of stable uncertainty periods is determined based on the statistical characteristics of the fluctuation of the integrated model uncertainty. Condition 2 is that the fluctuation amplitude of the global objective function value is less than a preset stable fluctuation threshold for a continuous preset number of stable performance periods. The preset number of stable performance periods is determined based on the fluctuation period of the global objective function value. When online learning is reactivated, the preset learning rate is reset to a lower level than the initial value. The specific reset ratio is determined based on the stability requirements of the system recovery phase. It is obtained by analyzing the convergence behavior of the strategy parameters and the degree of fluctuation of the system state under different reset levels, so as to restore the learning capability in a gradual manner. The reason for resetting the preset learning rate to a lower level when online learning is reactivated is that the system state may not be completely stable when online learning is first resumed. Using a smaller learning rate can avoid drastic changes in the strategy parameters and allow the system to smoothly transition to the normal learning state.

[0101] The system outputs in real time the final control values ​​for each stage, global resource quotas, prediction quality output, risk indicators, and boundary violation warning signals. The risk indicators are constructed by using the comprehensive model uncertainty of stage 1 as the first element of the risk indicator column vector, the comprehensive model uncertainty of stage 2 as the second element, and so on until the comprehensive model uncertainty of stage N is the last element. The comprehensive model uncertainties of each stage are arranged and combined sequentially according to the stage number to form a complete column vector. The final control values ​​for each stage are sent as control values ​​to the actuators of each stage, and the actuators adjust their working state according to the received final control values. In the next acquisition cycle, the actuator status data is read to verify whether the final control quantity is executed correctly. If the deviation between the actuator status data and the issued final control quantity exceeds the preset execution deviation threshold, the actuator abnormal event is recorded and compensated in the subsequent control calculation. The preset execution deviation threshold is determined based on the full scale of the final control quantity and the normal response accuracy of the actuator. Specifically, it is obtained by statistically analyzing the distribution characteristics of the deviation between the actuator status data and the issued control quantity under normal operating conditions and selecting a value that can distinguish between normal execution deviation and abnormal deviation. The process returns to step 100 to continuously acquire running data and continuously optimize the global objective function under the premise of meeting safety constraints.

[0102] To address the computational complexity issue in the fast variable phase, a simplified computation strategy is adopted. When the phase update cycle is less than a preset fast variable cycle threshold, the phase operates in a simplified mode. The preset fast variable cycle threshold is determined based on the controller's computational capabilities and real-time requirements. The simplified mode includes: reducing the shadow predictor parameter update frequency to a preset simplified frequency ratio equal to the normal frequency. This preset simplified frequency ratio is determined based on the controller's computational capabilities and the minimum frequency requirement for model parameter updates, specifically obtained by analyzing the degree of attenuation in model prediction accuracy after reducing the update frequency; using one-sided differencing instead of two-sided differencing for reward function gradient estimation to reduce computational load; and shortening the preset prediction time domain of safety prediction to a preset simplified time domain ratio equal to the normal value. This preset simplified time domain ratio is determined based on the minimum early warning lead requirement for safety prediction and computational resource constraints, specifically obtained by analyzing the change in the effectiveness of safety prediction after shortening the prediction time domain. These simplification measures reduce the computational burden while ensuring basic control performance, making real-time operation on a low-power controller possible.

[0103] The global objective function is the sum of the objective values ​​at each time step within the total optimization time. The objective value at each time step is obtained by a weighted sum of the following five terms: the first term is the square of the Euclidean distance between the output quality data and the quality target setpoint multiplied by a preset quality deviation weighting coefficient; the second term is the total global energy consumption multiplied by a preset energy consumption weighting coefficient; the third term is the square of the Euclidean distance between the final control quantity at the current time step and the final control quantity at the previous time step multiplied by a preset control smoothness weighting coefficient, where the final control quantity at each time step is obtained from the actuator state data at the corresponding time step; the fourth term is the constraint violation penalty function value multiplied by a preset constraint violation weighting coefficient; the fifth term is the sum of downstream responsibility penalties for all stages multiplied by a preset global downstream responsibility weighting coefficient, where the downstream responsibility penalty for each stage is equal to the sum of the squares of the predicted quality deviations of all downstream stages in that stage. The specific calculation method for the constraint violation penalty function is as follows: for all stages, calculate the square of the portion of the state value exceeding its state safety constraint upper limit; if the state value of that stage... If the upper limit is not exceeded, the value is set to 0; simultaneously, the square of the portion of the state value in this stage that is lower than the lower limit of its state safety constraint is calculated, and if the state value in this stage is not lower than the lower limit, the value is set to 0; finally, the above calculation results of all stages are summed to obtain the constraint violation penalty function value, that is, a secondary penalty is applied to the state that exceeds the safety constraint boundary; the state safety constraint boundary is determined by the lower and upper limits of each state component; the above five preset weight coefficients are all non-negative real numbers, determined according to the priority of the global optimization objective, specifically obtained through the weight allocation method in multi-objective optimization. Among them, the preset constraint violation weight coefficient is set relatively large to emphasize the importance of safety constraints, the preset quality deviation weight coefficient and the preset energy consumption weight coefficient are balanced according to the relative importance of production efficiency and energy saving objectives, the preset control smoothness weight coefficient is determined according to the actuator wear and control stability requirements, and the preset global downstream responsibility weight coefficient is consistent with the preset downstream responsibility weight coefficient in the local reward function to ensure the consistency between local optimization and global optimization objectives.

[0104] By cyclically executing the above steps, the global energy consumption and quality indicators of the multi-stage chemical reaction process can be self-optimized and controlled, avoiding local optima that lead to global deterioration and maintaining safe and stable operation.

[0105] Example 2:

[0106] See Figure 3 As shown, a self-optimizing control system for a multi-stage chemical reaction process is provided, which stores computer-readable instructions and, when the computer-readable instructions are read, can execute the aforementioned self-optimizing control method for the multi-stage chemical reaction process. The system includes:

[0107] The data modeling module 101 is used to collect operational data of a multi-stage chemical reaction process and construct a directed graph of the multi-stage reaction based on the operational data. The directed graph includes a set of nodes and a set of directed edges, and a time-delay queue is established for each directed edge.

[0108] The state construction module 102 is used to set an independent update cycle for each stage, set a global scheduler to determine the update time of each stage according to the update cycle, and construct an enhanced state vector for each stage based on the running data, directed graph and time delay queue after the update time is reached.

[0109] The predictive control module 103 establishes a shadow predictor for each stage based on the enhanced state vector, and establishes a two-layer self-optimizing control structure based on the shadow predictor and stage strategy of each stage. The two-layer self-optimizing control structure includes a local layer and a global layer. The local layer includes multiple stage strategies, and the global layer includes a coordinator.

[0110] The control generation module 104 is used to calculate candidate control quantities according to the stage strategy at the update time, and to perform constraint projection, rate limiting and safety prediction processing on the candidate control quantities in sequence based on the shadow predictor to obtain the final control quantity.

[0111] The collaborative output module 105 is used to perform shared resource allocation optimization by the coordinator when the preset global coordination cycle arrives, obtain the optimal resource quota for each stage, and convert the optimal resource quota into stage constraint updates and stage reference settings for each stage, so that each stage can determine and issue the final control quantity at the update time.

[0112] The embodiments of this application have been described above, but these embodiments are not limited to the specific implementation methods described above. The specific implementation methods described above are merely illustrative and not restrictive. Those skilled in the art can make more equivalent embodiments based on the guidance of these embodiments, and all of them are within the protection scope of these embodiments.

Claims

1. A self-optimization control method for a multi-stage chemical reaction process, characterized in that, Includes the following steps: Collect operational data of a multi-stage chemical reaction process, and construct a directed graph of the multi-stage reaction based on the operational data. The directed graph includes a set of nodes and a set of directed edges. Establish a time-delay queue for each directed edge. An independent update cycle is set for each stage. A global scheduler is set according to the update cycle to determine the update time of each stage. After the update time is reached, an enhanced state vector is constructed for each stage based on the running data, the directed graph, and the time-delay queue. A shadow predictor is established for each stage based on the enhanced state vector, and a two-layer self-optimizing control structure is established based on the shadow predictor and stage policy of each stage. The two-layer self-optimizing control structure includes a local layer and a global layer. The local layer includes multiple stage policies, and the global layer includes a coordinator. At the update time, candidate control quantities are calculated according to the stage strategy, and the candidate control quantities are sequentially subjected to constraint projection, rate limiting and safety prediction processing based on the shadow predictor to obtain the final control quantity; When the preset global coordination cycle arrives, the coordinator performs shared resource allocation optimization to obtain the optimal resource quota for each stage, and converts the optimal resource quota into stage constraint updates and stage reference settings for each stage, so that each stage can determine and issue the final control quantity at the update time.

2. The self-optimization control method for a multi-stage chemical reaction process according to claim 1, characterized in that, Methods for establishing time-delay queues include: The set of nodes in a directed graph includes N nodes, each node being a stage of a reaction. Each directed edge in the set of directed edges represents a connection relationship between one stage and another, where there is a flow of materials, energy, or information. If there is a directed edge from node i to node j, it means that there is a connection relationship between stage i and stage j. For each directed edge, a time delay queue is established. The time delay queue includes multiple time delay states. The time delay state from stage i to stage j at time t is: the state value of stage i at time t minus the propagation time delay. The propagation time delay is the time delay from stage i to stage j. The state value is obtained by arranging and combining the state data within the stage from all running data at time t.

3. The self-optimization control method for a multi-stage chemical reaction process according to claim 2, characterized in that, Methods for determining the update timing of each stage include: The update conditions are determined, including state change conditions, quality deviation conditions, and cycle arrival conditions. The state change condition is that the Euclidean distance between the state value of stage i at the acquisition time and the state value at the previous update time exceeds a preset state change threshold. The quality deviation condition is that the Euclidean distance between the output quality data in the running data at the acquisition time and the quality target set value exceeds a preset quality deviation threshold. The cycle arrival condition is that the time interval between the current acquisition time and the previous update time is greater than or equal to the update cycle of stage i. When stage i satisfies any update condition at the acquisition time, the global scheduler determines that the acquisition time is the update time of stage i.

4. The self-optimization control method for a multi-stage chemical reaction process according to claim 2, characterized in that, Methods for constructing enhanced state vectors include: The state values ​​are standardized to obtain the local state vector, which includes multiple state variables; Construct the target deviation vector based on the current actual values ​​of each state variable at the update time; Calculate the uncertainty estimate for each state variable in the local state vector and construct the model uncertainty estimate vector; The enhanced state vector of stage i at the update time is obtained by combining the local state vector, the time-delayed states of all upstream stages passed to stage i, the target deviation vector of stage i, and the model uncertainty estimation vector of stage i, one end to the other. The upstream stage refers to all stages in the directed graph that have directed edges pointing to stage i.

5. The self-optimization control method for a multi-stage chemical reaction process according to claim 4, characterized in that, Methods for establishing a two-level self-optimizing control structure include: A shadow predictor is established for each stage. The shadow predictor calculates the predicted state of the stage at a future time based on the enhanced state vector of the stage. The predicted state is the state vector of the stage at a future time. Each component in the state vector corresponds to the predicted value of the corresponding state variable in the local state vector at a future time. Construct a local layer and set N stage strategies in the local layer; Construct a global layer, set up a coordinator in the global layer, and make the coordinator run at a preset global coordination cycle; The shadow predictor, local layer, and global layer constitute a two-layer self-optimizing control structure.

6. The self-optimization control method for a multi-stage chemical reaction process according to claim 5, characterized in that, Methods for predicting states during the computation phase include: In the local layer, each stage strategy generates control variables based on the enhanced state vector of the corresponding stage; Based on the enhanced state vector and control quantity of stage i at time t, the predicted state of stage i at time t plus the stage update period is calculated by the shadow predictor of stage i.

7. The self-optimization control method for a multi-stage chemical reaction process according to claim 1, characterized in that, Methods for obtaining the final control quantity include: Set the reward function for stage i, and update the policy gain matrix online using a deterministic policy gradient method based on the reward function; The candidate control quantity is calculated based on the enhanced state vector of stage i at time t and the policy gain matrix of stage i. The candidate control variable is constrained and projected to obtain the projected control variable. Rate limiting is applied to the projected control variable to obtain the rate-limited control variable. The shadow predictor is used to make a safety prediction of the control quantity after rate limiting. If the predicted state exceeds the set of state safety constraints in stage i, conservative scaling is performed to obtain the final control quantity; otherwise, the control quantity after rate limiting is used as the final control quantity.

8. The self-optimization control method for a multi-stage chemical reaction process according to claim 7, characterized in that, Methods for setting the reward function for stage i include: Calculate the operating cost item based on the operating cost of stage i; Calculate the local energy consumption term based on the local energy consumption of stage i; Calculate the global quality item based on the output quality data in the runtime data; The prediction quality deviation of the downstream stage is calculated based on the shadow predictor of the downstream stage i. The weighted sum of the squares of the prediction quality deviations of all downstream stages is calculated to obtain the prediction quality term. The downstream stage refers to all stages in the directed graph that are the starting point of the directed edge i and the ending point of the directed edge i. The reward function for stage i is the negative result of the weighted sum of the operating cost, local energy consumption, global quality, and predicted quality.

9. The self-optimization control method for a multi-stage chemical reaction process according to claim 1, characterized in that, Methods for obtaining the optimal resource allocation at each stage include: When the preset global coordination cycle arrives, the coordinator receives the marginal revenue curves reported by each stage based on the current operating status. The marginal revenue curve of stage i is a quadratic function of the resource quota of stage i. The quadratic coefficient, linear coefficient and constant term of the quadratic function are estimated by stage i based on historical operating data. Based on the marginal revenue curves of each stage, the coordinator obtains the optimal resource quota for each stage under the constraint that the sum of resource quotas in all stages does not exceed the upper limit of the total shared resources, and that the resource quota for each stage is between the lower and upper limits of the resource quota for that stage.

10. A self-optimizing control system for a multi-stage chemical reaction process, characterized in that, It is used to store computer-readable instructions, which, when read, enable the execution of a self-optimizing control method for a multi-stage chemical reaction process as described in any one of claims 1-9; the system includes: The data modeling module is used to collect operational data of multi-stage chemical reaction processes and construct a directed graph of the multi-stage reaction based on the operational data. The directed graph includes a set of nodes and a set of directed edges, and a time-delay queue is established for each directed edge. The state construction module is used to set an independent update cycle for each stage, set a global scheduler to determine the update time of each stage according to the update cycle, and construct an enhanced state vector for each stage based on the running data, directed graph and time delay queue after the update time is reached. The predictive control module establishes a shadow predictor for each stage based on the enhanced state vector, and establishes a two-layer self-optimizing control structure based on the shadow predictor and stage strategy of each stage. The two-layer self-optimizing control structure includes a local layer and a global layer. The local layer includes multiple stage strategies, and the global layer includes a coordinator. The control generation module is used to calculate candidate control quantities according to the stage strategy at the update time, and to perform constraint projection, rate limiting and safety prediction processing on the candidate control quantities in sequence based on the shadow predictor to obtain the final control quantity. The collaborative output module is used to optimize the allocation of shared resources by the coordinator when the preset global coordination cycle arrives, obtain the optimal resource quota for each stage, and convert the optimal resource quota into stage constraint updates and stage reference settings for each stage, so that each stage can determine and issue the final control quantity at the update time.

Images