A deep reinforcement learning-based intelligent optimization method and system for a crude oil unit

Abstract

The application discloses a kind of based on deep reinforcement learning's atmospheric and vacuum distillation unit intelligent optimization method and system, it is related to oil refining chemical industry, including, from the distributed control unit of atmospheric and vacuum distillation unit and real-time database collection historical operation data, and pre-processing, obtain the historical data set after pre-processing;Using the historical data set after pre-processing, process mechanism characteristic network is constructed by graph neural network, the correlation between equipment components is mined and abstract feature representing process principle is extracted, and digital twin environment is constructed;In digital twin environment, meta-reinforcement learning algorithm is used to train deep reinforcement learning agent, so that deep reinforcement learning agent obtains meta-strategy suitable for different crude oil properties.The application realizes safe, smooth, efficient intelligent optimization control through safety mirror image drill and trust degree driven closed-loop operation mechanism, solves the optimization response lag problem caused by manual adjustment model due to the change of traditional optimization technology crude oil property.

Description

Technical Field

[0001] This invention relates to the field of oil refining and chemical engineering, and in particular to an intelligent optimization method and system for atmospheric and vacuum distillation units based on deep reinforcement learning. Background Technology

[0002] In the oil refining and chemical industry, the atmospheric and vacuum distillation unit is the first link in crude oil processing. Its operation optimization has a decisive impact on the economic benefits of the entire plant. Traditional optimization methods mainly rely on real-time optimization technology based on rigorous mechanism models. By establishing a detailed process mechanism model, combining on-site data to correct model parameters online, and using mathematical programming methods to solve the optimization problem with the goal of maximizing economic benefits, the optimization results are finally transmitted to the underlying control system for execution in the form of set values.

[0003] Although real-time optimization technology has been widely used in practice, its inherent characteristics limit its ability to cope with certain specific scenarios. The properties of crude oil processed by atmospheric and vacuum distillation units change frequently, and the differences in physical properties of different crude oils can cause a significant shift in the optimal operating point of the unit. When switching the processed crude oil, RTO systems based on rigorous mechanistic models often require process engineers to manually adjust model parameters or even update the model structure based on the new crude oil's assay data. This process is time-consuming, causing the RTO system to be unable to function effectively during the switching transition period, resulting in the unit operating under suboptimal conditions for a certain period of time. Summary of the Invention

[0004] In view of the aforementioned existing problems, the present invention is proposed.

[0005] Therefore, this invention provides an intelligent optimization method for atmospheric and vacuum distillation units based on deep reinforcement learning, which solves the optimization response lag problem caused by the need for manual model adjustment due to changes in crude oil properties in existing real-time optimization technologies.

[0006] To solve the above-mentioned technical problems, the present invention provides the following technical solution:

[0007] In a first aspect, the present invention provides an intelligent optimization method for atmospheric and vacuum distillation equipment based on deep reinforcement learning, which includes: collecting historical operation data from the distributed control unit and real-time database of the atmospheric and vacuum distillation equipment, and preprocessing it to obtain a preprocessed historical dataset; using the preprocessed historical dataset, constructing a process mechanism feature network through a graph neural network, mining the correlation between equipment components and extracting abstract features that characterize the process principle, and constructing a digital twin environment;

[0008] In a digital twin environment, a meta-reinforcement learning algorithm is used to train a deep reinforcement learning agent, enabling the deep reinforcement learning agent to acquire meta-policies that adapt to different crude oil properties.

[0009] A reward function with reward items is set for the deep reinforcement learning agent. The physical consistency reward item is calculated through the built-in lightweight mechanism model, which guides the exploration behavior of the deep reinforcement learning agent to conform to the physical laws.

[0010] The trained deep reinforcement learning agent is deployed online and a secure mirror exercise is performed in a digital twin environment to execute and verify the optimized actions as safe and effective.

[0011] After the safety mirror exercise is passed, closed-loop optimization operation is initiated, enabling the deep reinforcement learning agent to generate optimized operation suggestions based on real-time process data and send them to the distributed control unit.

[0012] As a preferred embodiment of the intelligent optimization method for atmospheric and vacuum diversion devices based on deep reinforcement learning described in this invention, the method includes: collecting historical operation data from the distributed control unit and real-time database of the atmospheric and vacuum diversion device, and preprocessing it to obtain a preprocessed historical dataset, comprising the following steps:

[0013] Historical operating data of the atmospheric and vacuum distillation unit are synchronously collected from the distributed control unit and the real-time database to form a raw dataset. The raw dataset is then input into the process rule filter and the isolated forest machine learning model for parallel processing. Outliers are jointly identified and removed to obtain a cleaned dataset.

[0014] Using the cleaned dataset, the ST-KNN algorithm is employed to accurately impute missing values ​​based on the temporal and spatial correlations between variables, generating a complete data matrix.

[0015] Input the complete data matrix into the steady-state discriminant model of sliding window principal component analysis, calculate the window statistic to measure the variation of process variables within the principal component model, and the statistic to measure the degree of deviation of process variables from the principal component model to identify and extract the steady-state operating condition data segment.

[0016] By aggregating steady-state data segments, a preprocessed historical dataset is constructed.

[0017] As a preferred embodiment of the intelligent optimization method for atmospheric and vacuum distillation units based on deep reinforcement learning described in this invention, the method includes the following steps: utilizing preprocessed historical datasets, constructing a process mechanism feature network through a graph neural network, mining the correlation between equipment components and extracting abstract features representing the process principle, and constructing a digital twin environment.

[0018] Using the preprocessed historical dataset, the equipment components in the atmospheric and vacuum distillation unit are defined as nodes in a graph structure, and the connection relationships between material flow and energy flow are defined as edges, thus constructing a graph structure data with node features and edge features.

[0019] The graph structure data of node features and edge features are input into the graph neural network. The message passing mechanism is used for multiple rounds of iterative updates. Each node aggregates the feature information of neighboring nodes and connecting edges, and updates its own feature representation through nonlinear transformation to obtain the node features and edge features of the relationship between device components.

[0020] Based on the node and edge features of the relationships between equipment components, the correlation coefficients and temporal dependencies between manipulated variables, disturbance variables and controlled variables are analyzed to uncover the spatiotemporal relationships between variables.

[0021] From the spatiotemporal correlation between variables, identify and extract feature combinations that conform to the principle of mass conservation and feature combinations that conform to the principle of energy conservation, and form abstract features that characterize the process principle;

[0022] The structural parameters of the simplified mechanism model are initialized using abstract features representing the process principle. The input features of the simplified mechanism model are then compensated for by deep learning bias using the abstract features representing the process principle, thus forming a digital twin environment.

[0023] As a preferred embodiment of the intelligent optimization method for atmospheric and vacuum distillation units based on deep reinforcement learning described in this invention, the method includes the following steps: In a digital twin environment, a meta-reinforcement learning algorithm is used to train the deep reinforcement learning agent, enabling the agent to acquire meta-policies adapted to different crude oil properties.

[0024] In a digital twin environment, the abstract features representing the principles of mass conservation and energy conservation in the process extracted by the process mechanism feature network are used to construct a shared feature space for the meta-task.

[0025] By adjusting the representation of crude oil properties in the abstract features, a meta-task set with different crude oil properties is generated;

[0026] The initial network parameters of the deep reinforcement learning agent are input into the meta-task set, and the inner loop is updated in the meta-task set to obtain temporary network parameters adapted to the properties of crude oil.

[0027] The meta-loss function is calculated based on temporary network parameters. The initial network parameters of the deep reinforcement learning agent are updated and optimized through second-order gradient updates. The inner and outer loop updates are repeated to enable the deep reinforcement learning agent to extract common features that are implicitly related to the process mechanism feature network.

[0028] As a preferred embodiment of the intelligent optimization method for atmospheric and vacuum diversion devices based on deep reinforcement learning described in this invention, the method includes: setting a reward function for the deep reinforcement learning agent, calculating a physical consistency reward term through a built-in lightweight mechanism model, and guiding the exploration behavior of the deep reinforcement learning agent to conform to physical laws, comprising the following steps:

[0029] Based on the goals of optimizing economic benefits, energy consumption, and product quality, a basic reward function is designed to quantify the direct value of actions of deep reinforcement learning agents.

[0030] Based on the physical laws of mass conservation and energy conservation, a lightweight mechanism model is constructed. The state and actions selected by the agent are input into the lightweight mechanism model, and the deviation between the output result and the physical conservation law is calculated to obtain the physical consistency error.

[0031] The results of the basic reward function calculation are combined with the physical consistency error according to weights to form the total reward function;

[0032] The total reward function is used to train the deep reinforcement learning agent. The reward signal received during exploration is used to update the policy network parameters based on the signal provided by the total reward function through the policy gradient algorithm, so as to obtain an economically optimized operation policy that strictly follows the physical laws.

[0033] As a preferred embodiment of the intelligent optimization method for atmospheric and vacuum diverter devices based on deep reinforcement learning described in this invention, the method includes: deploying the trained deep reinforcement learning agent online, conducting a secure mirror exercise in a digital twin environment, and executing optimization actions verified as safe and effective, comprising the following steps:

[0034] The trained deep reinforcement learning agent is deployed to an online server, and a safety mirror simulation controller and a real-time data interface are configured. The safety mirror simulation controller collects real-time process data from the distributed control unit and the real-time database.

[0035] The acquired real-time process data is processed in parallel through dual channels:

[0036] Channel 1 uses an isolated forest anomaly detection, ST-KNN missing value imputation, and sliding window PCA steady-state discrimination process to process real-time process data;

[0037] Channel 2 performs simple filtering on real-time process data;

[0038] Input the pre-processed real-time process data from Channel 1 and the simplified filtered real-time process data from Channel 2 into the digital twin environment respectively;

[0039] The digital twin environment performs extrapolation and prediction on the preprocessed real-time process data and the simplified filtered real-time process data respectively, and obtains two sets of predicted state trajectories and performance index changes;

[0040] Based on the feature correlations extracted from the process mechanism feature network, we analyze the degree of deviation between the variable correlations in the two sets of prediction results and the normal mode of the mass conservation and energy conservation process principle;

[0041] The operating conditions are clustered using historical steady-state operating data, and a threshold curve library is obtained by fitting the fluctuation range of statistical variables within each cluster to the optimal performance boundary.

[0042] The static boundary data in the threshold curve library is matched and interpolated with the real-time operating condition characteristics to obtain dynamic threshold rules; the standard dynamic threshold is obtained by matching the specific values ​​from the threshold curve library based on the real-time operating condition data.

[0043] Based on the dynamic threshold rule, the dynamic safety margin and performance improvement of the two sets of prediction results are calculated.

[0044] As a preferred embodiment of the intelligent optimization method for atmospheric and vacuum diversion devices based on deep reinforcement learning described in this invention, the method includes the following steps: After a safety mirror simulation is passed, closed-loop optimization is initiated, enabling the deep reinforcement learning agent to generate optimization operation suggestions based on real-time process data and distribute them to the distributed control unit:

[0045] After the security mirror exercise assessment is passed, the closed-loop optimization mode is initiated to initialize the optimization cycle, action range limits, and trust index operating parameters.

[0046] The system collects current process data from the distributed control unit and real-time database, performs filtering and standardization preprocessing to form a standardized state vector, inputs the preprocessed standardized state vector into the deep reinforcement learning agent, outputs optimized action suggestions, and evaluates the uncertainty of intelligent decision-making.

[0047] Based on the uncertainty assessment results of the agent's decision-making, the current trust index is calculated and refreshed according to the dynamic update formula;

[0048] Based on the actions executed in the previous cycle, the current agent's suggested actions, and the trust index, calculate the value of the flexible action to be executed in this cycle.

[0049] The final action command, after being softened, is sent to the distributed control unit via a secure communication protocol.

[0050] Secondly, the present invention provides an intelligent optimization system for atmospheric and vacuum distillation apparatus based on deep reinforcement learning, comprising: a data acquisition module, which collects historical operation data from the distributed control unit and real-time database of the atmospheric and vacuum distillation apparatus, and preprocesses it to obtain a preprocessed historical dataset; using the preprocessed historical dataset, a process mechanism feature network is constructed through a graph neural network to mine the correlation between equipment components and extract abstract features representing the process principle, and to construct a digital twin environment;

[0051] The training module, in a digital twin environment, uses a meta-reinforcement learning algorithm to train a deep reinforcement learning agent, enabling the deep reinforcement learning agent to acquire meta-policies that adapt to different crude oil properties.

[0052] The Physical Constraint Reward Shaping Module sets the reward function for the deep reinforcement learning agent and calculates the physical consistency reward through the built-in lightweight mechanism model, guiding the exploration behavior of the deep reinforcement learning agent to conform to physical laws.

[0053] The optimization module deploys the trained deep reinforcement learning agent online, conducts a secure mirror exercise in a digital twin environment, and executes and verifies the optimization actions as safe and effective.

[0054] After the safety mirror exercise is passed, the distribution module starts closed-loop optimization operation, enabling the deep reinforcement learning agent to generate optimized operation suggestions based on real-time process data and distribute them to the distributed control unit.

[0055] Thirdly, the present invention provides a computer device including a memory and a processor, wherein the memory stores a computer program, wherein: when the computer program is executed by the processor, it implements any step of the intelligent optimization method for atmospheric and depressurized pressure devices based on deep reinforcement learning as described in the first aspect of the present invention.

[0056] Fourthly, the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein: when the computer program is executed by a processor, it implements any step of the intelligent optimization method for atmospheric and vacuum diversion devices based on deep reinforcement learning as described in the first aspect of the present invention.

[0057] The beneficial effects of this invention are as follows: a high-quality dataset is constructed through data acquisition and preprocessing, and a digital twin environment integrating mechanism and data-driven approaches is established; meta-reinforcement learning is used to train the agent to obtain meta-policies adapted to different crude oil properties, achieving rapid adaptive optimization; a lightweight mechanism model is introduced to calculate the physical consistency reward term, ensuring that the agent's exploration behavior conforms to physical laws; and a safe, stable, and efficient intelligent optimization control is achieved through a safe mirror simulation and a trust-driven closed-loop operation mechanism, solving the optimization response lag problem caused by the need for manual model adjustment due to changes in crude oil properties in traditional optimization techniques, and significantly improving the device's adaptive capability and operating efficiency under varying operating conditions. Attached Figure Description

[0058] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0059] Figure 1 This is a flowchart of an intelligent optimization method for atmospheric and vacuum diversion devices based on deep reinforcement learning.

[0060] Figure 2 This is a schematic diagram of an intelligent optimization system for an atmospheric and vacuum diversion device based on deep reinforcement learning. Detailed Implementation

[0061] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0062] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the spirit of the invention. Therefore, the invention is not limited to the specific embodiments disclosed below.

[0063] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The phrase "in one embodiment" appearing in different places in this specification does not necessarily refer to the same embodiment, nor is it a single or selective embodiment that is mutually exclusive with other embodiments.

[0064] Reference Figures 1-2 This is one embodiment of the present invention, which provides an intelligent optimization method for atmospheric and vacuum diversion devices based on deep reinforcement learning, comprising the following steps:

[0065] S1. Collect historical operation data from the distributed control unit and real-time database of the atmospheric and vacuum distillation unit, and preprocess it to obtain the preprocessed historical dataset.

[0066] S1.1. Historical operating data of the atmospheric and vacuum distillation unit are synchronously collected from the distributed control unit and the real-time database to form a raw dataset. The raw dataset is then input into the process rule filter and the isolated forest machine learning model for parallel processing. Outliers are jointly identified and removed to obtain a cleaned dataset.

[0067] Furthermore, historical operating data of the atmospheric and vacuum distillation unit are synchronously collected from the distributed control unit and the real-time database to form a raw dataset containing manipulated variables, disturbance variables, and state variables. The raw dataset is simultaneously input into filters based on the absolute upper and lower limits and rate of change limits preset by the process rules, as well as an isolated forest machine learning model that uses random partitioning of the feature space for anomaly detection. The process rule filter removes outliers that are obviously outside the allowable range of the process, while the isolated forest machine learning model identifies hidden anomalies that have abnormal relationships with other variables. The results of the two types of anomaly detection are merged and all abnormal data are removed to obtain a cleaned dataset.

[0068] S1.2 Using the cleaned dataset, the ST-KNN algorithm is employed to accurately impute missing values ​​based on the temporal and spatial correlations between variables, generating a complete data matrix.

[0069] Furthermore, using the cleaned dataset, the spatiotemporal K-nearest neighbor algorithm is employed to find complete data points in the temporal and spatial neighborhoods of the missing data points. The temporal neighborhood is expanded forward and backward by several time steps centered on the missing point (e.g., 60 sampling points in the example). The spatial neighborhood selects several other variables that are most correlated with the missing variable (e.g., 5 variables in the example). The weighted average of the complete data points in these neighborhoods is calculated to fill in the missing values. The weights are determined by both temporal distance and spatial Euclidean distance, ultimately generating a complete data matrix without missing values.

[0070] S1.3 Input the complete data matrix into the steady-state discriminant model of the sliding window principal component analysis, calculate the window statistics to measure the variation of process variables within the principal component model, and the statistics to measure the degree of deviation of process variables from the principal component model to identify and extract the steady-state operating condition data segment.

[0071] Specifically, the expression is:

[0072] ;

[0073] in, The statistic measures the variation of process variables within the principal component model. For the score vector, This is the transpose of the score vector.

[0074] Specifically, the expression is:

[0075] ;

[0076] in, The statistic measures the degree to which process variables deviate from the principal component model. The original sample observation vector, This is the reconstructed value of the original sample observation vector.

[0077] S1.4. Collect steady-state data segments to construct a preprocessed historical dataset.

[0078] Furthermore, all steady-state operating condition data segments extracted through the sliding window principal component analysis steady-state discrimination model are aggregated, and the data segments are spliced ​​and integrated in chronological order. The overlapping parts between segments are removed and the temporal continuity is ensured to construct a preprocessed historical dataset containing only steady-state operating points.

[0079] S2. Using the preprocessed historical dataset, construct a process mechanism feature network through graph neural networks, mine the correlation between equipment components and extract abstract features that characterize the process principle, and build a digital twin environment.

[0080] S2.1 Using the preprocessed historical dataset, the equipment components in the atmospheric and vacuum distillation unit are defined as nodes in a graph structure, and the connection relationships between material flow and energy flow are defined as edges, thus constructing a graph structure data with node features and edge features.

[0081] Furthermore, the graph structure data of node features and edge features are input into the graph neural network. The graph neural network performs multiple rounds of iterative updates through a message passing mechanism. In each round of iteration, each node aggregates the feature information of all its neighboring nodes and connecting edges, combines the aggregated feature vector with the node's own feature representation from the previous round, and generates a new feature representation of the node through a nonlinear transformation function such as the ReLU activation function. At the same time, the edge features are also updated according to the features of the connected nodes. After multiple rounds of iteration, node features and edge features that can accurately represent the complex relationships between device components are obtained.

[0082] S2.2 Input the graph structure data of node features and edge features into the graph neural network, and use the message passing mechanism to perform multiple rounds of iterative updates. Each node aggregates the feature information of adjacent nodes and connecting edges, and updates its own feature representation through nonlinear transformation to obtain the node features and edge features of the relationship between device components.

[0083] Furthermore, the graph structure data of node features and edge features are input into the graph neural network. The graph neural network performs multiple rounds of iterative updates through a message passing mechanism. In each round of iteration, each node aggregates the feature information of all its neighboring nodes and connecting edges, combines the aggregated feature vector with the node's own feature representation from the previous round, and generates a new feature representation of the node through a nonlinear transformation function such as the ReLU activation function. At the same time, the edge features are also updated according to the features of the connected nodes. After multiple rounds of iteration, node features and edge features that can accurately represent the complex relationships between device components are obtained.

[0084] S2.3. Based on the node and edge features of the relationship between equipment components, analyze the correlation coefficients and temporal dependencies between the manipulated variables, disturbance variables and the controlled variables, and explore the spatiotemporal relationships between variables.

[0085] Furthermore, based on the node and edge features of the relationships between equipment components, the Pearson correlation coefficient is used to determine the degree of linear correlation between manipulated variables such as the fuel valve position of the heater, disturbance variables such as crude oil density, and controlled variables such as the tower top temperature. Time series analysis techniques such as mutual information are used to determine the temporal dependence between variables. By combining linear and nonlinear correlation measures, the complex spatiotemporal relationships between manipulated variables, disturbance variables, and controlled variables are extracted from the node and edge features.

[0086] S2.4 From the spatiotemporal correlation between variables, identify and extract feature combinations that conform to the principle of mass conservation and feature combinations that conform to the principle of energy conservation, and form abstract features that characterize the process principle.

[0087] Furthermore, from the spatiotemporal relationships between variables, we identify and extract feature combinations that conform to the principle of mass conservation, the correlation patterns of the sum of total feed flow and total discharge flow, and feature combinations that conform to the principle of energy conservation, such as the correlation patterns of the sum of total heat input and total heat output and heat loss. These core feature combinations representing the conservation laws are normalized and vectorized, ultimately forming a set of low-dimensional, dense abstract feature vectors representing the process principles of mass conservation and energy conservation.

[0088] S2.5. Initialize the structural parameters of the simplified mechanism model using the abstract features representing the process principle, and use deep learning to compensate for the biases of the abstract features representing the process principle to simplify the input features of the mechanism model, thus forming a digital twin environment.

[0089] Furthermore, the structural parameters of the simplified mechanism model are initialized using abstract feature vectors representing the process principle. For example, parameters representing the mass balance relationship in the abstract feature vector are assigned to the material balance equation of the simplified mechanism model, and parameters representing the energy balance relationship are assigned to the energy balance equation. At the same time, the complete abstract feature vector is used as the input feature of the deep learning bias compensation model to predict the deviation between the output of the simplified mechanism model and the true value. The parameterized simplified mechanism model is then integrated with the trained deep learning bias compensation model to form a digital twin environment.

[0090] S3. In a digital twin environment, a meta-reinforcement learning algorithm is used to train a deep reinforcement learning agent, enabling the deep reinforcement learning agent to acquire meta-policies that adapt to different crude oil properties.

[0091] S3.1 In a digital twin environment, the abstract features representing the process principles of mass conservation and energy conservation extracted from the process mechanism feature network are used to construct a shared feature space for the meta-task.

[0092] Furthermore, in a digital twin environment, the abstract features representing the process principle of mass conservation, such as the total material balance relationship vector, and the abstract features representing the process principle of energy conservation, such as the total energy balance relationship vector, extracted from the process mechanism feature network are spliced ​​and normalized to form a high-dimensional shared feature vector. The shared feature vector defines the shared feature space of the meta-task.

[0093] S3.2. By adjusting the representation of crude oil properties in the abstract features, a meta-task set with different crude oil properties is generated.

[0094] Furthermore, by adjusting the specific parameters representing the properties of crude oil in the shared feature vector, and adjusting the numerical range of crude oil density feature value, sulfur content feature value, and light component yield feature value, a large number of processing scenarios with different crude oil properties are simulated and generated in the digital twin environment. Each scenario constitutes an independent meta-task, and all meta-tasks are aggregated to form a meta-task set.

[0095] S3.3 Input the initial network parameters of the deep reinforcement learning agent into the meta-task set, and perform an inner loop update in the meta-task set to obtain temporary network parameters adapted to the properties of crude oil.

[0096] Furthermore, the initial policy network parameters and initial value network parameters of the deep reinforcement learning agent are input into the meta-task set, and an inner loop update is performed on each specific meta-task in the meta-task set. The inner loop update process includes the agent interacting with the digital twin environment a small amount of time to collect experience data, and updating the initial policy network parameters and initial value network parameters one or more times through gradient descent to obtain temporary policy network parameters and temporary value network parameters adapted to the specific crude oil properties.

[0097] S3.4 Calculate the meta-loss function based on temporary network parameters, optimize the initial network parameters of the deep reinforcement learning agent through second-order gradient updates, and repeat the inner loop update and outer loop update iterations to enable the deep reinforcement learning agent to extract the meta-strategy of common features implicitly related to the process mechanism feature network.

[0098] The meta-loss function expression is:

[0099] ;

[0100] in, For the meta-loss function, For the initial policy network, For value network parameters, The number of meta-tasks. For the meta-task index, For specific losses of the meta-task, For the first Each meta-task In order to be in Temporary policy network parameters adapted quickly on the meta-task. In order to be in Temporary value network parameters after rapid adaptation on the meta-task.

[0101] Furthermore, based on the temporary policy network parameters and temporary value network parameters obtained after adaptation on all meta-tasks, a meta-loss function is calculated, which is the sum of the loss functions on all meta-tasks. The second-order gradient of the meta-loss function with respect to the initial policy network parameters and initial value network parameters of the deep reinforcement learning agent is calculated using automatic differentiation techniques. The initial policy network parameters and initial value network parameters are updated using the Adam optimizer, with the update amount being the product of the outer loop learning rate and the second-order gradient value. The inner loop update and outer loop update iterations are repeated to enable the deep reinforcement learning agent to extract cross-task common features implicitly related to the process mechanism feature network, thereby obtaining a meta-policy that can adapt to different crude oil properties.

[0102] S4. Set a reward function for the deep reinforcement learning agent, calculate the physical consistency reward through the built-in lightweight mechanism model, and guide the exploration behavior of the deep reinforcement learning agent to conform to the physical laws.

[0103] S4.1 Based on the goals of economic benefits, energy consumption and product quality optimization, design a basic reward function to quantify the direct value of the actions of the deep reinforcement learning agent.

[0104] Furthermore, based on the optimization goals of economic benefits, energy consumption, and product quality, a basic reward function is constructed. The direct value of the deep reinforcement learning agent's actions is quantified into a weighted combination of instantaneous profit change, increased processing capacity, reduced energy consumption costs, and the squared deviation of the controlled variable from the target value. Instantaneous profit change is calculated from product value and raw material and utility costs. Increased processing capacity reflects the effect of device load optimization. Energy consumption costs include fuel gas and electricity consumption. The squared deviation of the controlled variable from the target value encourages edge control of product quality. Each component is balanced through preset weight coefficients to ultimately form the output value of the basic reward function.

[0105] S4.2 Based on the physical laws of mass conservation and energy conservation, a lightweight mechanism model is constructed. The state and action selected by the agent are input into the lightweight mechanism model, and the deviation between the output result and the physical conservation law is calculated to obtain the physical consistency error.

[0106] Specifically, the expression is:

[0107] ;

[0108] in, For physical consistency error, As a quality reference value weight, As the weight of the energy reference value, For mass imbalance quantity, This is an energy imbalance quantity. For quality reference value, This is an energy reference value.

[0109] Furthermore, the theoretical principle of the law of conservation of mass states that the total material flow rate entering the atmospheric and vacuum distillation unit must be equal to the total material flow rate leaving the atmospheric and vacuum distillation unit. The theoretical principle of the law of conservation of energy states that the total heat energy input into the atmospheric and vacuum distillation unit must be equal to the sum of the total heat energy output from the atmospheric and vacuum distillation unit and the heat loss. The state and actions selected by the intelligent agent are input into the lightweight mechanism model. Based on the theoretical principles of the laws of conservation of mass and energy, the lightweight mechanism model obtains the expected ideal output value. The ideal output value calculated by the lightweight mechanism model is compared with the theoretical values ​​of the laws of conservation of mass and energy that the actual physical process must follow, and the absolute deviation between the two is calculated. The absolute deviation is the physical consistency error.

[0110] S4.3 Combine the calculation results of the basic reward function with the physical consistency error according to the weights to form the total reward function.

[0111] Furthermore, the results of the basic reward function calculation and the physical consistency error are linearly combined according to preset weights to form the total reward function. The results of the basic reward function calculation reflect the economic value of the deep reinforcement learning agent's actions, while the physical consistency error characterizes the severity of the action's violation of physical laws. The total reward function is calculated by subtracting the product of the physical consistency penalty coefficient and the physical consistency error from the results of the basic reward function calculation. The physical consistency penalty coefficient is much larger than the weights of each component in the basic reward function. This combination method ensures that the physical consistency error dominates the total reward function, allowing the deep reinforcement learning agent to prioritize satisfying physical constraints during the optimization process. The output value of the total reward function will serve as the target signal for the deep reinforcement learning agent's policy update.

[0112] S4.4. The total reward function is used to train the deep reinforcement learning agent. The reward signal received during exploration is used to update the policy network parameters based on the signal provided by the total reward function through the policy gradient algorithm, so as to obtain an economically optimized operation policy that strictly follows the physical laws.

[0113] Furthermore, the total reward function is used in the training process of the deep reinforcement learning agent. When the deep reinforcement learning agent explores in the digital twin environment, it receives the reward signal calculated by the total reward function. The proximal policy optimization algorithm is adopted to optimize the policy gradient. Based on the reward signal provided by the total reward function, the policy gradient is calculated and the policy network parameters of the deep reinforcement learning agent are updated. The policy gradient update formula is that the gradient of the policy network parameters is equal to the product of the policy gradient and the difference between the timely reward and the baseline value. The timely reward is provided by the total reward function. Through iterative updates, the policy network parameters converge, and finally, an operational policy that pursues both economic optimization and strict adherence to physical laws is obtained.

[0114] S5. Deploy the trained deep reinforcement learning agent online, conduct a secure mirror exercise in a digital twin environment, and execute optimized actions that are verified to be safe and effective.

[0115] S5.1 Deploy the trained deep reinforcement learning agent to the online server, configure the safety mirror exercise controller and real-time data interface, and the safety mirror exercise controller collects real-time process data from the distributed control unit and the real-time database.

[0116] Furthermore, the trained deep reinforcement learning agent is deployed to an online server in the production environment, and a safety mirror simulation controller and a real-time data interface are configured. The safety mirror simulation controller reads real-time data of manipulated variables from the distributed control unit through the OPC UA protocol, obtains historical trend data of controlled variables from the real-time database through the database interface, and integrates them to form the current real-time process status data of the atmospheric and vacuum distillation unit. Clock synchronization among all components ensures data timestamp alignment. The safety mirror simulation controller collects and updates real-time process status data at a fixed frequency.

[0117] S5.2. Perform parallel dual-channel processing on the acquired real-time process data:

[0118] Channel 1 uses an isolated forest anomaly detection, ST-KNN missing value imputation, and sliding window PCA steady-state discrimination process to process real-time process data.

[0119] Channel 2 performs simple filtering on real-time process data.

[0120] Furthermore, the collected real-time process data is processed in parallel through two channels. Channel 1 uses the isolated forest anomaly detection algorithm to identify and remove outliers in the real-time process data, uses the ST-KNN algorithm to fill in the missing values ​​with spatiotemporal features, and uses the sliding window PCA algorithm to identify and filter steady-state conditions, outputting a preprocessed real-time state vector. Channel 2 performs simple moving average filtering on the real-time process data, outputting filtered and smoothed real-time data. The processing of the two channels is carried out synchronously and in parallel, and the timestamps of the output results are aligned.

[0121] S5.3 Input the preprocessed real-time process data from Channel 1 and the simplified filtered real-time process data from Channel 2 into the digital twin environment respectively.

[0122] Furthermore, the real-time state vector after preprocessing in Channel 1 and the real-time process data after simple filtering in Channel 2 are respectively input into the digital twin environment. The digital twin environment receives data from the two different processing methods as input, in preparation for inference and prediction.

[0123] S5.4 The digital twin environment performs simulations and predictions on the preprocessed real-time process data and the simplified filtered real-time process data respectively, and obtains two sets of predicted state trajectories and performance index changes.

[0124] The digital twin environment performs extrapolation and prediction on the preprocessed real-time state vector and the simplified filtered real-time data respectively. By using a hybrid digital twin model, the state trajectory and performance index changes at multiple future time steps are formed, resulting in two complete sets of prediction results.

[0125] Furthermore, based on the feature correlations extracted from the process mechanism feature network, the degree of deviation between the variable correlations in the two sets of prediction results and the normal pattern of the process principle of mass conservation and energy conservation is analyzed, and abnormal correlation patterns that violate the conservation law are detected.

[0126] S5.5. Based on the feature correlation extracted from the process mechanism feature network, analyze the degree of deviation between the variable correlation in the two sets of prediction results and the normal mode of the mass conservation and energy conservation process principle.

[0127] Furthermore, based on the abstract features representing the mass conservation and energy conservation process principles extracted by the process mechanism feature network, the variable correlations in the two sets of prediction results obtained by the digital twin environment through the inference and prediction of the real-time state vector after preprocessing of channel one and the real-time process data after simple filtering of channel two are analyzed. The dynamic correlation patterns between the manipulated variables, disturbance variables and controlled variables in the prediction results are similar to the normal patterns of the mass conservation process principle and the normal patterns of the energy conservation process principle learned by the process mechanism feature network. The feature space distance or cosine similarity between the two sets of prediction results and the normal process principle patterns are obtained, and the degree of deviation is quantified.

[0128] S5.6. Perform operating condition clustering using historical steady-state operating data, and obtain a threshold curve library by fitting the fluctuation range of statistical variables within each cluster to the optimal performance boundary.

[0129] Furthermore, multi-condition cluster analysis is performed using historical steady-state operating data. Within each cluster, the allowable fluctuation range and optimal performance boundary of key variables are statistically analyzed. A multinomial fitting method is used to generate threshold curves under different operating conditions, and a threshold curve library is constructed.

[0130] S5.7 Match and interpolate the static boundary data in the threshold curve library with the real-time operating condition characteristics to obtain dynamic threshold rules; obtain the standard dynamic threshold by matching the specific values ​​from the threshold curve library based on the real-time operating condition data.

[0131] Based on the dynamic threshold rule, the dynamic safety margin and performance improvement of the two sets of prediction results are calculated;

[0132] The dynamic safety margin expression is:

[0133]

[0134] in, For the current moment No. The controlled variable in the future... The dynamic safety margin at each step of the prediction time. For the first Hard constraint limits for each controlled variable For the current moment of the digital twin environment Predicted future Step 1 The value of the controlled variable, For the current moment, To predict the step size.

[0135] The expression for the performance improvement is:

[0136] ;

[0137] in, From the current moment The initial percentage improvement in predictive performance. For the current moment The initial cumulative discount prediction reward for the next N steps. Rewards are accumulated based on recent historical average discounts;

[0138] Furthermore, based on real-time operating condition characteristics, the most similar operating condition clusters are matched from the threshold curve library, and the dynamic safety threshold and performance improvement threshold under the current operating condition are calculated through interpolation. Based on the dynamic threshold rules, the dynamic safety margin and performance improvement magnitude of the two sets of prediction results are calculated. The dynamic safety margin reflects the degree of conformity between the predicted state and the constraint boundary, and the performance improvement magnitude reflects the improvement level of the predicted performance relative to the historical benchmark.

[0139] S5.8. An approval execution decision signal shall be generated if and only if the deduction results of both channels meet the standard dynamic threshold requirements and there is no violation of the process principle.

[0140] Furthermore, an approval execution decision signal is generated if and only if the dynamic safety margin of the two channels in all prediction time domains is higher than the dynamic safety threshold, the performance improvement exceeds the performance improvement threshold, and there is no violation of process principles; otherwise, a rejection execution decision signal is generated and an alarm is triggered.

[0141] S6. After the safety mirror exercise is passed, the closed-loop optimization operation is started, which enables the deep reinforcement learning agent to generate optimization operation suggestions based on real-time process data and send them to the distributed control unit.

[0142] S6.1 After the security mirror exercise assessment is passed, the closed-loop optimization mode is started to initialize the optimization cycle, action range limit and trust index operation parameters.

[0143] Furthermore, after the safety mirror exercise assessment is passed, the operator confirms the activation of the closed-loop optimization mode, initializes the optimization cycle to a fixed time interval, sets the action amplitude limit of each manipulated variable to the maximum allowable change, initializes the trust index to the median value, and writes all parameters into the closed-loop operation configuration file. The safety mirror exercise controller then switches to the closed-loop optimization operation state. The optimization cycle is longer than the adjustment time of the underlying control loop, ensuring that the system reaches a steady state after the actions are executed.

[0144] S6.2 Collect current process data from the distributed control unit and real-time database, perform filtering and standardization preprocessing to form a standardized state vector, input the preprocessed standardized state vector into the deep reinforcement learning agent, output optimization action suggestions, and evaluate the uncertainty of intelligent decision-making.

[0145] Furthermore, at the beginning of each optimization cycle, real-time data of the manipulated variables are read from the distributed control unit, and historical data of the controlled variables are obtained from the real-time database, which are then integrated to form the current process data. The current process data is smoothed using a moving average filtering method, followed by mean-variance standardization to form a standardized state vector. The standardized state vector is then input into the deep reinforcement learning agent, and optimization action suggestions are calculated and output through the policy network. At the same time, the uncertainty of the deep reinforcement learning agent's decision-making is evaluated through an ensemble learning method.

[0146] S6.3 Based on the uncertainty assessment results of the agent's decision-making, calculate and refresh the current trust index according to the dynamic update formula.

[0147] The expression for the trust index is:

[0148] ;

[0149] in, This represents the trust level index at the current moment. The trust index at the previous moment. Forgetting factor, This represents the uncertainty assessment value for the decision at the current moment. This is the uncertainty scaling factor.

[0150] S6.4. Combine the actions executed in the previous cycle, the current agent's suggested actions, and the trust index to calculate the value of the flexible action to be executed in this cycle.

[0151] The expression for the flexible action value is:

[0152] ;

[0153] in, This represents the value of the flexible action to be executed at the current moment. The value of the action that was executed in the previous moment. This represents the dynamic trust index at the current moment. This represents the change in action suggested by the agent at the current moment.

[0154] S6.5. The final action command after softening is sent to the distributed control unit through a secure communication protocol.

[0155] Furthermore, the softened final action command is converted into a setpoint format recognizable by the distributed control unit (DCU) and sent to the advanced control layer setpoint interface of the DCU via the OPC UA secure communication protocol. After receiving the setpoint command, the DCU executes the operation and returns a command reception confirmation signal. The secure mirror simulation controller records the command issuance time, content, and confirmation status. The OPC UA protocol uses certificate encryption and authentication mechanisms to ensure the security and reliability of the communication process. The amplitude of the final action command is constrained by the action amplitude limit to ensure the smoothness of a single operation.

[0156] This embodiment also provides an intelligent optimization system for atmospheric and vacuum diversion devices based on deep reinforcement learning, including:

[0157] The data acquisition module collects historical operation data from the distributed control unit and real-time database of the atmospheric and vacuum distillation unit, and preprocesses it to obtain a preprocessed historical dataset. Using the preprocessed historical dataset, a process mechanism feature network is constructed through a graph neural network to mine the correlation between equipment components and extract abstract features that characterize the process principle, thus constructing a digital twin environment.

[0158] The training module, in a digital twin environment, uses a meta-reinforcement learning algorithm to train a deep reinforcement learning agent, enabling the deep reinforcement learning agent to acquire meta-policies that adapt to different crude oil properties.

[0159] The Physical Constraint Reward Shaping Module sets the reward function for the deep reinforcement learning agent and calculates the physical consistency reward through the built-in lightweight mechanism model, guiding the exploration behavior of the deep reinforcement learning agent to conform to physical laws.

[0160] The optimization module deploys the trained deep reinforcement learning agent online, conducts a secure mirror exercise in a digital twin environment, and executes and verifies the optimization actions as safe and effective.

[0161] After the safety mirror exercise is passed, the distribution module starts closed-loop optimization operation, enabling the deep reinforcement learning agent to generate optimized operation suggestions based on real-time process data and distribute them to the distributed control unit.

[0162] This embodiment also provides a computer device applicable to the intelligent optimization method for atmospheric and vacuum diversion devices based on deep reinforcement learning, comprising: a memory and a processor; the memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions to implement the intelligent optimization method for atmospheric and vacuum diversion devices based on deep reinforcement learning as proposed in the above embodiment.

[0163] The computer device can be a terminal, comprising a processor, memory, communication interface, display screen, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, carrier networks, NFC (Near Field Communication), or other technologies. The display screen can be an LCD screen or an e-ink screen. The input devices can be a touch layer covering the display screen, buttons, a trackball, or a touchpad on the computer device's casing, or an external keyboard, touchpad, or mouse.

[0164] This embodiment also provides a storage medium storing a computer program, which, when executed by a processor, implements the intelligent optimization method for atmospheric and vacuum devices based on deep reinforcement learning as proposed in the above embodiments. The storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as Static Random Access Memory (SRAM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Erasable Programmable Read Only Memory (EPROM), Programmable Red-Only Memory (PROM), Read-Only Memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0165] In summary, this invention constructs a high-quality dataset through data acquisition and preprocessing, and establishes a digital twin environment that integrates mechanism and data-driven approaches. It employs meta-reinforcement learning to train agents to obtain meta-policies adapted to different crude oil properties, achieving rapid adaptive optimization. By introducing a lightweight mechanism model to calculate physical consistency rewards, it ensures that the agent's exploration behavior conforms to physical laws. Through a safe mirror simulation and a trust-driven closed-loop operation mechanism, it achieves safe, stable, and efficient intelligent optimization control, solving the optimization response lag problem caused by the need for manual model adjustments due to changes in crude oil properties in traditional optimization techniques. This significantly improves the device's adaptive capability and operational efficiency under varying operating conditions.

[0166] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A method for intelligent optimization of atmospheric and vacuum diversion devices based on deep reinforcement learning, characterized in that: include, Historical operation data is collected from the distributed control unit and real-time database of the atmospheric and vacuum distillation unit and preprocessed to obtain a preprocessed historical dataset. Using preprocessed historical datasets, a process mechanism feature network is constructed through graph neural networks to mine the correlations between equipment components and extract abstract features representing the process principle, thus building a digital twin environment, including the following steps: Using the preprocessed historical dataset, the equipment components in the atmospheric and vacuum distillation unit are defined as nodes in a graph structure, and the connection relationships between material flow and energy flow are defined as edges, thus constructing a graph structure data with node features and edge features. The graph structure data of node features and edge features are input into the graph neural network. The message passing mechanism is used for multiple rounds of iterative updates. Each node aggregates the feature information of neighboring nodes and connecting edges, and updates its own feature representation through nonlinear transformation to obtain the node features and edge features of the relationship between device components. Based on the node and edge features of the relationships between equipment components, the correlation coefficients and temporal dependencies between manipulated variables, disturbance variables and controlled variables are analyzed to uncover the spatiotemporal relationships between variables. From the spatiotemporal correlation between variables, identify and extract feature combinations that conform to the principle of mass conservation and feature combinations that conform to the principle of energy conservation, and form abstract features that characterize the process principle; The abstract features representing the process principle are used to initialize the structural parameters of the simplified mechanism model. The abstract features representing the process principle are then used to perform deep learning bias compensation on the input features of the simplified mechanism model to form a digital twin environment. In a digital twin environment, a meta-reinforcement learning algorithm is used to train a deep reinforcement learning agent, enabling the agent to acquire a meta-policy adapted to different crude oil properties. This includes the following steps: In a digital twin environment, the abstract features representing the principles of mass conservation and energy conservation in the process extracted by the process mechanism feature network are used to construct a shared feature space for the meta-task. By adjusting the representation of crude oil properties in the abstract features, a meta-task set with different crude oil properties is generated; The initial network parameters of the deep reinforcement learning agent are input into the meta-task set, and the inner loop is updated in the meta-task set to obtain temporary network parameters adapted to the properties of crude oil. The meta-loss function is calculated based on temporary network parameters. The initial network parameters of the deep reinforcement learning agent are updated and optimized through second-order gradient. The inner loop update and outer loop update iterations are repeated to enable the deep reinforcement learning agent to extract the meta-strategy of common features implicitly related to the process mechanism feature network. A reward function is set for the deep reinforcement learning agent, and a physical consistency reward is calculated through a built-in lightweight mechanism model to guide the exploration behavior of the deep reinforcement learning agent to conform to physical laws. This includes the following steps: Based on the goals of optimizing economic benefits, energy consumption, and product quality, a basic reward function is designed to quantify the direct value of actions of deep reinforcement learning agents. Based on the physical laws of mass conservation and energy conservation, a lightweight mechanism model is constructed. The state and actions selected by the agent are input into the lightweight mechanism model, and the deviation between the output result and the physical conservation law is calculated to obtain the physical consistency error. The results of the basic reward function calculation are combined with the physical consistency error according to weights to form the total reward function; The total reward function is used to train the deep reinforcement learning agent. The reward signal received during exploration is used to update the policy network parameters based on the signal provided by the total reward function through the policy gradient algorithm, so as to obtain an economically optimized operation policy that strictly follows the physical laws. The trained deep reinforcement learning agent is deployed online and a secure mirror exercise is performed in a digital twin environment to execute and verify the optimized actions as safe and effective. After the safety mirror exercise is passed, closed-loop optimization operation is initiated, enabling the deep reinforcement learning agent to generate optimized operation suggestions based on real-time process data and send them to the distributed control unit.

2. The intelligent optimization method for atmospheric and vacuum diversion devices based on deep reinforcement learning as described in claim 1, characterized in that: Historical operating data is collected from the distributed control unit and real-time database of the atmospheric and vacuum distillation unit, and preprocessed to obtain a preprocessed historical dataset, including the following steps: Historical operating data of the atmospheric and vacuum distillation unit are synchronously collected from the distributed control unit and the real-time database to form a raw dataset. The raw dataset is then input into the process rule filter and the isolated forest machine learning model for parallel processing. Outliers are jointly identified and removed to obtain a cleaned dataset. Using the cleaned dataset, the ST-KNN algorithm is employed to accurately impute missing values ​​based on the temporal and spatial correlations between variables, generating a complete data matrix. Input the complete data matrix into the steady-state discriminant model of sliding window principal component analysis, calculate the window statistic to measure the variation of process variables within the principal component model, and the statistic to measure the degree of deviation of process variables from the principal component model to identify and extract the steady-state operating condition data segment. By aggregating steady-state data segments, a preprocessed historical dataset is constructed.

3. The intelligent optimization method for atmospheric and vacuum diversion devices based on deep reinforcement learning as described in claim 2, characterized in that: The trained deep reinforcement learning agent is deployed online and subjected to a secure mirror exercise in a digital twin environment. Optimization actions verified as safe and effective are executed, including the following steps: The trained deep reinforcement learning agent is deployed to an online server, and a safety mirror simulation controller and a real-time data interface are configured. The safety mirror simulation controller collects real-time process data from the distributed control unit and the real-time database. The acquired real-time process data is processed in parallel through dual channels: Channel 1 uses an isolated forest anomaly detection, ST-KNN missing value imputation, and sliding window PCA steady-state discrimination process to process real-time process data; Channel 2 performs simple filtering on real-time process data; Input the pre-processed real-time process data from Channel 1 and the simplified filtered real-time process data from Channel 2 into the digital twin environment respectively; The digital twin environment performs extrapolation and prediction on the preprocessed real-time process data and the simplified filtered real-time process data respectively, and obtains two sets of predicted state trajectories and performance index changes; Based on the feature correlations extracted from the process mechanism feature network, we analyze the degree of deviation between the variable correlations in the two sets of prediction results and the normal mode of the mass conservation and energy conservation process principle; The operating conditions are clustered using historical steady-state operating data, and a threshold curve library is obtained by fitting the fluctuation range of statistical variables within each cluster to the optimal performance boundary. The static boundary data in the threshold curve library is matched and interpolated with the real-time operating condition characteristics to obtain dynamic threshold rules; the standard dynamic threshold is obtained by matching the specific values ​​from the threshold curve library based on the real-time operating condition data. Based on the dynamic threshold rule, the dynamic safety margin and performance improvement of the two sets of prediction results are calculated; An approval execution decision signal is generated if and only if the deduction results of both channels meet the standard dynamic threshold requirements and there is no violation of process principles.

4. The intelligent optimization method for atmospheric and vacuum diversion devices based on deep reinforcement learning as described in claim 3, characterized in that: After the safety mirror exercise is passed, closed-loop optimization operation is initiated, enabling the deep reinforcement learning agent to generate optimization operation suggestions based on real-time process data and distribute them to the distributed control unit. This includes the following steps: After the security mirror exercise assessment is passed, the closed-loop optimization mode is initiated to initialize the optimization cycle, action range limits, and trust index operating parameters. The system collects current process data from the distributed control unit and real-time database, performs filtering and standardization preprocessing to form a standardized state vector, inputs the preprocessed standardized state vector into the deep reinforcement learning agent, outputs optimized action suggestions, and evaluates the uncertainty of intelligent decision-making. Based on the uncertainty assessment results of the agent's decision-making, the current trust index is calculated and refreshed according to the dynamic update formula; Based on the actions executed in the previous cycle, the current agent's suggested actions, and the trust index, calculate the value of the flexible action to be executed in this cycle. The final action command, after being softened, is sent to the distributed control unit via a secure communication protocol.

5. An intelligent optimization system for an atmospheric and vacuum diversion device based on deep reinforcement learning, based on the intelligent optimization method for an atmospheric and vacuum diversion device based on deep reinforcement learning as described in any one of claims 1 to 4, characterized in that: include, The data acquisition module collects historical operation data from the distributed control unit and real-time database of the atmospheric and vacuum distillation unit, and preprocesses it to obtain a preprocessed historical dataset. Using the preprocessed historical dataset, a process mechanism feature network is constructed through a graph neural network to mine the correlation between equipment components and extract abstract features that characterize the process principle, thus constructing a digital twin environment. The training module, in a digital twin environment, uses a meta-reinforcement learning algorithm to train a deep reinforcement learning agent, enabling the deep reinforcement learning agent to acquire meta-policies that adapt to different crude oil properties. The Physical Constraint Reward Shaping Module sets the reward function for the deep reinforcement learning agent and calculates the physical consistency reward through the built-in lightweight mechanism model, guiding the exploration behavior of the deep reinforcement learning agent to conform to physical laws. The optimization module deploys the trained deep reinforcement learning agent online, conducts a secure mirror exercise in a digital twin environment, and executes and verifies the optimization actions as safe and effective. After the safety mirror exercise is passed, the distribution module starts closed-loop optimization operation, enabling the deep reinforcement learning agent to generate optimized operation suggestions based on real-time process data and distribute them to the distributed control unit.

6. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that: When the processor executes the computer program, it implements the steps of the intelligent optimization method for atmospheric and vacuum diversion devices based on deep reinforcement learning as described in any one of claims 1 to 4.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by the processor, it implements the steps of the intelligent optimization method for atmospheric and vacuum diversion devices based on deep reinforcement learning as described in any one of claims 1 to 4.

Images