Intelligent scheduling method and system for crude oil of refinery plant

Abstract

The invention belongs to the field of intelligent control scheduling, and provides a refinery crude oil intelligent scheduling method and system, and the method comprises the steps: constructing a refinery crude oil scheduling simulation environment; markov processing is carried out on a refinery crude oil scheduling problem, and a corresponding relation between the Markov processing and the refinery crude oil scheduling simulation environment data is determined; determining the reward of the refinery scheduling problem according to the corresponding relation; constructing a strategy network and a value network of deep reinforcement learning; training a deep reinforcement learning model according to the refinery crude oil dispatching simulation environment data, the corresponding relation and the reward; and according to the trained deep reinforcement learning model, a refinery crude oil intelligent scheduling decision is made. According to the intelligent scheduling method and system for the refinery crude oil, the refinery crude oil scheduling task is converted into Markov decision for modeling, and the SAC algorithm network is adopted for deep reinforcement learning training, so that the solving efficiency of the refinery crude oil scheduling decision is greatly improved.

Description

Technical Field

[0001] The present invention belongs to the field of intelligent control scheduling, and in particular relates to a refinery crude oil intelligent scheduling method and system. Background Art

[0002] The scheduling problem of coastal crude oil refineries mainly considers the large-scale network scheduling problem under the topological structure of crude oil from tankers to terminal tank areas, the interaction between terminal tank areas and commercial tank areas, the terminal tank areas through long-distance pipelines to plant tank areas, and the plant tank areas to atmospheric and vacuum transposition. In actual business scenarios, a series of complex nonlinear constraints such as oil types, physical properties, and long-distance pipelines need to be considered. Therefore, the above problems are generally modeled as large-scale nonlinear mixed integer programming problems. Generally speaking, large-scale mixed integer programming problems are NP-hard problems, and existing solution software cannot effectively and quickly solve the above large-scale mixed integer programming problems. Usually, in order to reduce the solution scale and improve the computational efficiency, the crude oil scheduling model is simplified, which will cause the model to be unable to fully and effectively describe the actual business scenario of crude oil scheduling. Therefore, how to quickly and effectively solve large-scale mixed integer problems based on the actual business logic of refinery crude oil scheduling is of great significance to the effectiveness and optimality of crude oil scheduling.

[0003] Existing crude oil scheduling models are often modeled based on mathematical programming. If this method is used to model, if the modeling effectiveness is pursued, the problem may be large in scale and cannot be effectively solved. If the model is simplified, it may not be able to effectively describe the business scenario, resulting in invalid solution results. Existing models are built based on specific business scenarios and have poor generalization. When the crude oil scheduling scenario changes slightly, the model solution efficiency will be greatly reduced. Summary of the invention

[0004] In view of the above problems, the present invention proposes a refinery crude oil intelligent scheduling method, the method comprising:

[0005] Build a refinery crude oil scheduling simulation environment;

[0006] Markovizing the refinery crude oil scheduling problem, and determining the corresponding relationship between the Markovizing and the refinery crude oil scheduling simulation environment data;

[0007] determining a reward for the refinery scheduling problem according to the corresponding relationship;

[0008] Constructing policy networks and value networks for deep reinforcement learning;

[0009] Training a deep reinforcement learning model according to the refinery crude oil scheduling simulation environment data, the corresponding relationship, and the reward;

[0010] Intelligent scheduling decisions for crude oil in refineries are made based on the trained deep reinforcement learning model.

[0011] Furthermore, the refinery crude oil scheduling simulation environment intelligent body includes oil tankers, terminal tanks, commercial storage tanks, long-distance pipelines, in-plant tanks, onshore imports and atmospheric and vacuum devices.

[0012] Furthermore, Markovization of the refinery crude oil scheduling problem includes:

[0013] The refinery production system initializes the simulation environment, provides state information of the intelligent agent through the simulation environment, and inputs the state information to the intelligent agent;

[0014] The intelligent agent makes a decision action and inputs the decision action into the simulation environment;

[0015] The simulation environment updates the node status in the decision action according to the decision action and determines the reward obtained by executing the decision action in the current state.

[0016] Further, determining the correspondence between the Markov transformation and the refinery crude oil scheduling simulation environment data includes:

[0017] Determine the set of state space, action space and agent rewards for each agent in the scheduling task.

[0018] Furthermore, the reward for the refinery scheduling problem is determined according to the corresponding relationship, including:

[0019] The rewards for refinery scheduling problems are determined based on the early warning over-limit rewards and penalties, planned over-limit rewards and penalties, fluctuation rewards and penalties, and costs.

[0020] Furthermore, the warning over-limit reward and punishment are determined by the periodic liquid level, the safety liquid level and the tank quantity limit;

[0021] The planned over-limit reward and penalty is determined by the difference between the sum of the periodic oil intake of the atmospheric and vacuum distillation device and the planned total processing volume;

[0022] The fluctuation reward and penalty is determined by the total number of pipeline speed switching, the pipeline speed switching cost, the total number of switching of the processing scheme of the atmospheric and vacuum distillation device, and the scheme switching cost;

[0023] The cost is determined by the total delay time for tanker unloading, the cost of delayed unloading, the total inventory of terminal tanks, the cost of terminal tank inventory, the total inventory of in-plant tanks, the cost of in-plant tank inventory, the total volume of long-distance pipeline transportation and the transportation cost.

[0024] Furthermore, the strategy network and value network of deep reinforcement learning are constructed through the SAC framework network.

[0025] Further, training a deep reinforcement learning model according to the refinery crude oil scheduling simulation environment data, the corresponding relationship and the reward includes:

[0026] Setting an initial state of the agent and generating a decision action according to the initial state;

[0027] Obtain the initial state, next state and reward according to Markov decision making;

[0028] Training the agent according to the SAC algorithm;

[0029] Determine whether the current number of iterations is equal to the preset number of iterations;

[0030] If the current number of iterations is equal to the preset number of iterations, new path data is generated according to the deep reinforcement learning model, and the path data is stored in the experience pool until the new segment is set as the initial state and the training is repeated;

[0031] If the current number of iterations is not equal to the preset number of iterations, the decision action is regenerated according to the initial state until the current number of iterations is equal to the preset number of iterations.

[0032] The present invention also provides a refinery crude oil intelligent scheduling system, the system comprising: a first construction unit, a Markov unit, a corresponding unit, a reward unit, a second construction unit, a reinforcement unit and a decision unit.

[0033] The first construction unit is used to construct a refinery crude oil scheduling simulation environment;

[0034] A Markov unit, in communication connection with the first building unit, for Markovizing the refinery crude oil scheduling problem;

[0035] A corresponding unit, which is in communication connection with the first construction unit and the Markov unit, and is used to determine the corresponding relationship between the Markov transformation and the refinery crude oil scheduling simulation environment data;

[0036] a reward unit, in communication with the corresponding unit, for determining a reward for the refinery scheduling problem according to the corresponding relationship;

[0037] The second building block is used to build the policy network and value network of deep reinforcement learning;

[0038] A reinforcement unit, which is in communication connection with the first construction unit, the corresponding unit and the reward unit, and is used to train a deep reinforcement learning model according to the refinery crude oil scheduling simulation environment data, the corresponding relationship and the reward;

[0039] The decision-making unit is in communication with the reinforcement unit and is used to make intelligent scheduling decisions for refinery crude oil based on the trained deep reinforcement learning model.

[0040] Furthermore, the refinery crude oil scheduling simulation environment intelligent body includes oil tankers, terminal tanks, commercial storage tanks, long-distance pipelines, in-plant tanks, onshore imports and atmospheric and vacuum devices.

[0041] Further, the Markov unit includes a state determination module, an action determination module and an update module;

[0042] A state determination module is used to control the refinery production system to initialize the simulation environment, provide state information of the intelligent agent through the simulation environment, and input the state information to the intelligent agent;

[0043] An action determination module, used to control the agent to make a decision action and input the decision action into the simulation environment;

[0044] The updating module is used to control the simulation environment to update the node status in the decision action according to the decision action and determine the reward obtained by executing the decision action in the current state.

[0045] Further, determining the correspondence between the Markov transformation and the refinery crude oil scheduling simulation environment data includes:

[0046] Determine the set of state space, action space and agent rewards for each agent in the scheduling task.

[0047] Furthermore, the reward for the refinery scheduling problem is determined according to the corresponding relationship, including:

[0048] The rewards for refinery scheduling problems are determined based on the early warning over-limit rewards and penalties, planned over-limit rewards and penalties, fluctuation rewards and penalties, and costs.

[0049] Furthermore, the warning over-limit reward and punishment are determined by the periodic liquid level, the safety liquid level and the tank quantity limit;

[0050] The planned over-limit reward and penalty is determined by the difference between the sum of the periodic oil intake of the atmospheric and vacuum distillation device and the planned total processing volume;

[0051] The fluctuation reward and penalty is determined by the total number of pipeline speed switching, the pipeline speed switching cost, the total number of switching of the processing scheme of the atmospheric and vacuum distillation device, and the scheme switching cost;

[0052] The cost is determined by the total delay time for tanker unloading, the cost of delayed unloading, the total inventory of terminal tanks, the cost of terminal tank inventory, the total inventory of in-plant tanks, the cost of in-plant tank inventory, the total volume of long-distance pipeline transportation and the transportation cost.

[0053] Furthermore, the strategy network and value network of deep reinforcement learning are constructed through the SAC framework network.

[0054] The refinery crude oil intelligent scheduling method and system of the present invention converts the refinery crude oil scheduling task into Markov decision for modeling, and adopts SAC algorithm network for deep reinforcement learning training, which greatly improves the efficiency of solving refinery crude oil scheduling decisions.

[0055] Other features and advantages of the present invention will be described in the following description, and partly become apparent from the description, or understood by practicing the present invention. The purpose and other advantages of the present invention can be realized and obtained by the structures pointed out in the description, claims and drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0057] Figure 1 A schematic diagram of the process flow of the refinery crude oil intelligent scheduling method in an embodiment of the present invention is shown;

[0058] Figure 2 A schematic diagram of the algorithm flow of training a deep reinforcement learning model in an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0059] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0060] The embodiment of the present invention provides a refinery crude oil intelligent scheduling method. Figure 1 The figure shows a schematic diagram of the process of the intelligent scheduling method for crude oil in a refinery in an embodiment of the present invention. Figure 1 The method includes: constructing a refinery crude oil scheduling simulation environment; Markovizing the refinery crude oil scheduling problem and determining the correspondence between the Markovization and the refinery crude oil scheduling simulation environment data; determining the reward of the refinery scheduling problem according to the correspondence; constructing a strategy network and a value network of deep reinforcement learning; training a deep reinforcement learning model according to the refinery crude oil scheduling simulation environment data, the correspondence and the reward; and making intelligent refinery crude oil scheduling decisions according to the trained deep reinforcement learning model.

[0061] Specifically, the intelligent entities in the refinery crude oil scheduling simulation environment include oil tankers, terminal tanks, commercial storage tanks, long-distance pipelines, in-plant tanks, onshore imports and atmospheric and vacuum devices.

[0062] Specifically, Markovization of the refinery crude oil scheduling problem includes:

[0063] The refinery production system initializes the simulation environment, provides state information of the intelligent agent through the simulation environment, and inputs the state information to the intelligent agent;

[0064] The intelligent agent makes a decision action and inputs the decision action into the simulation environment;

[0065] The simulation environment updates the node status in the decision action according to the decision action and determines the reward obtained by executing the decision action in the current state.

[0066] Specifically, determining the correspondence between the Markov transformation and the refinery crude oil scheduling simulation environment data includes:

[0067] Determine the set of state space, action space and agent rewards for each agent in the scheduling task.

[0068] Specifically, the rewards for the refinery scheduling problem determined according to the corresponding relationship include:

[0069] The rewards for refinery scheduling problems are determined based on the early warning over-limit rewards and penalties, planned over-limit rewards and penalties, fluctuation rewards and penalties, and costs.

[0070] Specifically, the warning over-limit reward and punishment are determined by the periodic liquid level, safety liquid level and tank quantity limit;

[0071] The planned over-limit reward and penalty is determined by the difference between the sum of the periodic oil intake of the atmospheric and vacuum distillation device and the planned total processing volume;

[0072] The fluctuation reward and penalty is determined by the total number of pipeline speed switching, the pipeline speed switching cost, the total number of switching of the processing scheme of the atmospheric and vacuum distillation device, and the scheme switching cost;

[0073] The cost is determined by the total delay time for tanker unloading, the cost of delayed unloading, the total inventory of terminal tanks, the cost of terminal tank inventory, the total inventory of in-plant tanks, the cost of in-plant tank inventory, the total volume of long-distance pipeline transportation and the transportation cost.

[0074] Specifically, the strategy network and value network of deep reinforcement learning are constructed through the SAC framework network.

[0075] Specifically, training a deep reinforcement learning model according to the refinery crude oil scheduling simulation environment data, the corresponding relationship and the reward includes:

[0076] Setting an initial state of the agent and generating a decision action according to the initial state;

[0077] Obtain the initial state, next state and reward according to Markov decision making;

[0078] Training the agent according to the SAC algorithm;

[0079] Determine whether the current number of iterations is equal to the preset number of iterations;

[0080] If the current number of iterations is equal to the preset number of iterations, new path data is generated according to the deep reinforcement learning model, and the path data is stored in the experience pool until the new segment is set as the initial state and the training is repeated;

[0081] If the current number of iterations is not equal to the preset number of iterations, the decision action is regenerated according to the initial state until the current number of iterations is equal to the preset number of iterations.

[0082] In the embodiment of the present invention, the process of the refinery crude oil scheduling solution method based on deep reinforcement learning is also specifically described:

[0083] first step:

[0084] A refinery crude oil scheduling simulation environment is constructed. The simulation environment adopts a multi-agent simulation modeling method. The simulation model includes: tankers, terminal tanks, commercial storage tanks, land imports, long-distance pipelines, plant tanks, atmospheric and vacuum devices and other types of agents and their network relationships; the properties, actions and network relationships of each agent are also specifically described in the embodiment of the present invention:

[0085] 1) Tanker Agent

[0086] Attributes: Estimated arrival time, estimated departure time, oil type name, oil type density, oil type sulfur content, remaining unloading volume, unloading speed, docking terminal, delayed unloading cost;

[0087] Action: Pay oil;

[0088] 2) Terminal tank / commercial storage tank intelligent body

[0089] Attributes: oil type name, oil type density, oil type sulfur content, current tank status, current liquid level, safe high and low liquid levels, maximum and low tank volume, whether tank dumping is allowed, tank type, remaining static time, inventory cost, high and low overflow volume;

[0090] Actions: add oil, let stand, collect oil;

[0091] 3) Land import

[0092] Attributes: oil type name, oil type density, oil type sulfur content, oil transportation speed, oil transportation cost, oil transportation destination tank;

[0093] Action: Pay oil;

[0094] 4) Intelligent tank in the factory

[0095] Attributes: oil type name, oil type density, oil type sulfur content, tank current status, current liquid level, safe high and low liquid levels, maximum and low tank volume, whether tank dumping is allowed, tank type, inventory cost, high and low overflow volume;

[0096] Action: pay oil, collect oil;

[0097] 5) Long-distance pipelines

[0098] Attributes: name of batch oil type in pipeline, density of batch oil type in pipeline, sulfur content of batch oil type in pipeline, batch oil volume in pipeline, total oil volume in pipeline, oil head identification, pipeline transportation speed, pipeline transportation speed switching cost, upper and lower limits of pipeline transportation capacity, and pipeline transportation cost;

[0099] Actions: receiving oil and delivering oil;

[0100] 6) Atmospheric and distillation device nodes

[0101] Attributes: current processing plan, planned processing volume, type and volume of oil received, and plan switching cost;

[0102] Action: Processing;

[0103] 7) Network Relationships

[0104] Including but not limited to the communication connection of tankers, terminal tanks, commercial storage tanks, long-distance pipelines, in-plant tanks, land imports and atmospheric and vacuum devices;

[0105] The wharf and the wharf tanks have a fixed connection relationship; the wharf tanks and the in-plant tanks are fixedly connected through long-distance pipelines; the connection between the in-plant tanks and the atmospheric and vacuum devices is abstracted as a point-to-point connection, which is instantaneous; the connection between the wharf tanks and the commercial storage tanks is abstracted as a point-to-point connection, which is instantaneous; the onshore crude oil is abstracted as a pipeline connected to the in-plant tanks;

[0106] Step 2:

[0107] Markovization of the refinery crude oil scheduling problem, specifically, converting the refinery scheduling task problem into a Markov decision problem: the refinery production system gives the status information of oil tankers, commercial storage tanks, terminal tanks, long-distance pipelines, in-plant tanks, onshore crude oil, and atmospheric and vacuum devices, and inputs the status information to the intelligent agent, which makes decision actions. The work includes the transportation speed and oil transportation target of each node, and inputs the action to the simulation environment. The simulation environment updates the node status according to the action and returns the reward obtained by executing the action under the current state;

[0108] Step 3:

[0109] The Markov design in the second step corresponds to the refinery scheduling simulation data to construct the state space, action space and agent reward set of the scheduling task;

[0110] 1) State Space

[0111] Including oil tankers (remaining oil volume, remaining departure time), terminal tanks (oil volume, oil type), commercial storage tanks (oil volume, oil type), long-distance pipelines (total oil volume, batch oil volume, batch oil type), in-plant tanks (oil volume, oil type), atmospheric and vacuum devices (processing plan, completed processing volume), etc.

[0112] 2) Action Space

[0113] Action information includes oil tankers (unloading speed and unloading target), terminal tanks (target tanks, transmission speed), commercial storage tanks (target tanks, transmission speed), long-distance pipelines (pipeline transmission speed), in-plant tanks (target tanks, transmission speed), atmospheric and vacuum devices (processing plans), etc.

[0114] 3) Rewards for each agent

[0115] Tanker: speed fluctuation cost + detention cost;

[0116] Terminal tanks, commercial storage tanks, in-plant tanks: overflow cost + storage cost;

[0117] Long-distance pipelines: speed fluctuation costs + overflow costs + freight;

[0118] Atmospheric and vacuum device: scheme fluctuation cost + processing cost;

[0119] Step 4:

[0120] Constructing a policy network and a value network for deep reinforcement learning. In the embodiment of the present invention, the SAC algorithm network is used for network construction. In the embodiment of the present invention, the SAC algorithm is used to introduce the concept of maximum entropy on the basis of maximizing future cumulative rewards. The purpose of adding entropy is to enhance robustness and the exploration ability of the intelligent agent.

[0121] Step 5:

[0122] Constructing rewards for the refinery scheduling problem. In the embodiment of the present invention, the following aspects are considered in constructing rewards: rewards and penalties for early warning over-limit, rewards and penalties for planned over-limit, rewards and penalties for fluctuations, and costs;

[0123] 1) The early warning over-limit reward and punishment include the sum of the over-limit penalties of all cycles and all tanks. In the embodiment of the present invention, the early warning over-limit reward and punishment are determined by the cycle liquid level, safety liquid level and tank quantity limit, specifically including:

[0124] a: If the liquid level in the current cycle > high safety liquid level && the liquid level in the current cycle < the maximum tank volume: the reward and penalty for over-limit warning = high safety liquid level - liquid level in the current cycle;

[0125] b: If the current cycle liquid level < low safety liquid level && current cycle liquid level > minimum tank volume: warning over limit reward and penalty = current cycle liquid level - low safety liquid level;

[0126] c: If the liquid level in the current cycle > the maximum tank volume: warning over-limit reward and penalty = (high safety liquid level - liquid level in the current cycle) * coefficient. In the embodiment of the present invention, the coefficient > 1;

[0127] d: If the liquid level in the current cycle < the minimum tank volume: warning over-limit reward and penalty = (low safety liquid level - liquid level in the current cycle) * coefficient. In the embodiment of the present invention, the coefficient > 1;

[0128] e: If the current cycle liquid level < high safety liquid level && current cycle liquid level > low safety liquid level: warning over limit reward and penalty = 0;

[0129] 2) Planned over-limit rewards and penalties are determined by the difference between the sum of the periodic oil intake of the atmospheric and vacuum distillation device and the planned total processing volume, and planned over-limit rewards and penalties = (the absolute value of the difference between the sum of the oil intake of all periods of all atmospheric and vacuum distillation devices and the planned total processing volume) * coefficient;

[0130] 3) Fluctuation rewards and penalties are determined by the total number of pipeline speed switching, pipeline speed switching cost, the total number of switching of the processing scheme of the constant and distillation device, and the scheme switching cost. Specifically, in the embodiment of the present invention, the pipeline switching cost and the constant processing device scheme switching cost are considered; Fluctuation rewards and penalties: total number of pipeline speed switching * pipeline speed switching cost + total number of switching of the processing scheme of the constant and distillation device * scheme switching cost;

[0131] 4) Cost, which is determined by the total delay time of oil tanker unloading, the delayed unloading cost, the total inventory of terminal tanks, the inventory cost of terminal tanks, the total inventory of in-plant tanks, the inventory cost of in-plant tanks, the total volume of long-distance pipeline transportation, and the transportation cost. In the embodiment of the present invention, the total delay time of oil tanker unloading*delayed unloading cost+the total inventory of all terminal tanks*inventory cost+the total inventory of all in-plant tanks*inventory cost+the total volume of long-distance pipeline transportation*the transportation cost;

[0132] 5) In the embodiment of the present invention, the overall reward and punishment is determined by adding the warning over-limit reward and punishment, the planned over-limit reward and punishment, the fluctuation reward and punishment and the cost;

[0133] Step 6:

[0134] According to the refinery crude oil scheduling simulation environment data, the corresponding relationship and the reward training deep reinforcement learning model, in the embodiment of the present invention, the algorithm flow of training the deep reinforcement learning model is also specifically described. Figure 2 The figure shows a schematic diagram of the algorithm flow of training a deep reinforcement learning model in an embodiment of the present invention. Figure 2In the algorithm flow, the algorithm flow includes: creating an environment and initializing the experience pool D, setting the initial state s0 of the agent, generating a decision action a0 according to the initial state s0, obtaining the next state s1 and the reward r1 according to the simulation environment, and in the embodiment of the present invention, in the Markov decision process, the decision maker (Agent) selects an action according to the current state at each moment, and transfers to the next state according to the state transition probability, trains the agent according to the SAC algorithm, determines whether the current number of iterations is equal to the preset number of iterations, and if the current number of iterations is equal to the preset number of iterations, generates new path data {s0, a0, r1} according to the deep reinforcement learning model. 0… ,s t , a t , r t}, and store the path data into the experience pool D until the new episode is set as the initial state and the training is repeated. If the current number of iterations is not equal to the preset number of iterations, return to the step of generating actions according to the state until the current number of iterations is equal to the preset number of iterations.

[0135] In an embodiment of the present invention, training the intelligent agent according to the SAC (Soft Actor-Critic) algorithm includes: obtaining sampled data from the experience pool D, and updating the V Critic (V value) network, the Q Critic (Q value) network, the strategy network and the Target V Critic (target V value) network in sequence to complete the training.

[0136] Step 7:

[0137] The trained model is deployed to the application environment for scheduling decisions. In the embodiment of the present invention, the application environment includes but is not limited to the intelligent agent described in the embodiment of the present invention. The adaptability adjustment of the type of intelligent agent is also within the protection scope of the embodiment of the present invention.

[0138] The embodiment of the present invention also provides a refinery crude oil intelligent scheduling system, the system comprising: a first construction unit, a Markov unit, a corresponding unit, a reward unit, a second construction unit, a reinforcement unit and a decision unit.

[0139] The first construction unit is used to construct a refinery crude oil scheduling simulation environment;

[0140] A Markov unit, in communication connection with the first building unit, for Markovizing the refinery crude oil scheduling problem;

[0141] A corresponding unit, which is in communication connection with the first construction unit and the Markov unit, and is used to determine the corresponding relationship between the Markov transformation and the refinery crude oil scheduling simulation environment data;

[0142] a reward unit, in communication with the corresponding unit, for determining a reward for the refinery scheduling problem according to the corresponding relationship;

[0143] The second building block is used to build the policy network and value network of deep reinforcement learning;

[0144] A reinforcement unit, which is in communication connection with the first construction unit, the corresponding unit and the reward unit, and is used to train a deep reinforcement learning model according to the refinery crude oil scheduling simulation environment data, the corresponding relationship and the reward;

[0145] The decision-making unit is in communication with the reinforcement unit and is used to make intelligent scheduling decisions for refinery crude oil based on the trained deep reinforcement learning model.

[0146] Specifically, the Markov unit includes a state determination module, an action determination module and an update module;

[0147] A state determination module is used to control the refinery production system to initialize the simulation environment, provide state information of the intelligent agent through the simulation environment, and input the state information to the intelligent agent;

[0148] An action determination module, used to control the agent to make a decision action and input the decision action into the simulation system environment;

[0149] The updating module is used to control the simulation environment to update the node status in the decision action according to the decision action and determine the reward obtained by executing the decision action in the current state.

[0150] The refinery crude oil intelligent scheduling method and system of the present invention converts the refinery crude oil scheduling task into Markov decision for modeling, and adopts SAC algorithm network for deep reinforcement learning training, which greatly improves the efficiency of solving refinery crude oil scheduling decisions without simplifying actual business problems.

[0151] Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent substitutions for some of the technical features therein; and these modifications or substitutions do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A refinery crude oil intelligent scheduling method, characterized in that: The method comprises: Build a refinery crude oil scheduling simulation environment; Markovizing the refinery crude oil scheduling problem, and determining the corresponding relationship between the Markovizing and the refinery crude oil scheduling simulation environment data; determining a reward for the refinery scheduling problem according to the corresponding relationship; Constructing policy networks and value networks for deep reinforcement learning; Training a deep reinforcement learning model according to the refinery crude oil scheduling simulation environment data, the corresponding relationship, and the reward; Intelligent scheduling decisions for crude oil in refineries are made based on the trained deep reinforcement learning model.

2. The refinery crude oil intelligent scheduling method according to claim 1, characterized in that: The refinery crude oil scheduling simulation environment intelligent body includes oil tankers, terminal tanks, commercial storage tanks, long-distance pipelines, in-plant tanks, land imports and atmospheric and vacuum devices.

3. The refinery crude oil intelligent scheduling method according to claim 1 or 2, characterized in that: Markovization of the refinery crude oil scheduling problem involves: The refinery production system initializes the simulation environment, provides state information of the intelligent agent through the simulation environment, and inputs the state information to the intelligent agent; The intelligent agent makes a decision action and inputs the decision action into the simulation environment; The simulation environment updates the node status in the decision action according to the decision action and determines the reward obtained by executing the decision action in the current state.

4. The refinery crude oil intelligent scheduling method according to claim 3, characterized in that: Determining the correspondence between the Markov transformation and the refinery crude oil scheduling simulation environment data includes: Determine the set of state space, action space and agent rewards for each agent in the scheduling task.

5. The method for intelligent scheduling of crude oil in a refinery according to claim 2, characterized in that: The rewards for the refinery scheduling problem determined according to the corresponding relationship include: The rewards for refinery scheduling problems are determined based on the early warning over-limit rewards and penalties, planned over-limit rewards and penalties, fluctuation rewards and penalties, and costs.

6. The refinery crude oil intelligent scheduling method according to claim 5, characterized in that: The warning over-limit reward and punishment are determined by the periodic liquid level, safety liquid level and tank quantity limit; The planned over-limit reward and penalty is determined by the difference between the sum of the periodic oil intake of the atmospheric and vacuum distillation device and the planned total processing volume; The fluctuation reward and penalty is determined by the total number of pipeline speed switching, the pipeline speed switching cost, the total number of switching of the processing scheme of the atmospheric and vacuum distillation device, and the scheme switching cost; The cost is determined by the total delay time for tanker unloading, the cost of delayed unloading, the total inventory of terminal tanks, the cost of terminal tank inventory, the total inventory of in-plant tanks, the cost of in-plant tank inventory, the total volume of long-distance pipeline transportation and the transportation cost.

7. The refinery crude oil intelligent scheduling method according to claim 6, characterized in that: The policy network and value network of deep reinforcement learning are constructed through the SAC framework network.

8. The refinery crude oil intelligent scheduling method according to claim 7, characterized in that: Training a deep reinforcement learning model according to the refinery crude oil scheduling simulation environment data, the corresponding relationship and the reward includes: Setting an initial state of the agent and generating a decision action according to the initial state; Obtain the initial state, next state and reward according to Markov decision making; Training the agent according to the SAC algorithm; Determine whether the current number of iterations is equal to the preset number of iterations; If the current number of iterations is equal to the preset number of iterations, new path data is generated according to the deep reinforcement learning model, and the path data is stored in the experience pool until the new segment is set as the initial state and the training is repeated; If the current number of iterations is not equal to the preset number of iterations, the decision action is regenerated according to the initial state until the current number of iterations is equal to the preset number of iterations.

9. An intelligent crude oil dispatching system for a refinery, characterized in that: The system comprises: a first building unit, a Markov unit, a corresponding unit, a reward unit, a second building unit, a reinforcement unit and a decision unit. The first construction unit is used to construct a refinery crude oil scheduling simulation environment; A Markov unit, in communication connection with the first building unit, for Markovizing the refinery crude oil scheduling problem; A corresponding unit, which is in communication connection with the first construction unit and the Markov unit, and is used to determine the corresponding relationship between the Markov transformation and the refinery crude oil scheduling simulation environment data; a reward unit, in communication with the corresponding unit, for determining a reward for the refinery scheduling problem according to the corresponding relationship; The second building block is used to build the policy network and value network of deep reinforcement learning; A reinforcement unit, which is in communication connection with the first construction unit, the corresponding unit and the reward unit, and is used to train a deep reinforcement learning model according to the refinery crude oil scheduling simulation environment data, the corresponding relationship and the reward; The decision-making unit is in communication with the reinforcement unit and is used to make intelligent scheduling decisions for refinery crude oil based on the trained deep reinforcement learning model.

10. The refinery crude oil intelligent scheduling system according to claim 9, characterized in that: The refinery crude oil scheduling simulation environment intelligent body includes oil tankers, terminal tanks, commercial storage tanks, long-distance pipelines, in-plant tanks, land imports and atmospheric and vacuum devices.

11. The refinery crude oil intelligent scheduling system according to claim 9 or 10, characterized in that: The Markov unit includes a state determination module, an action determination module and an update module; A state determination module is used to control the refinery production system to initialize the simulation environment, provide state information of the intelligent agent through the simulation environment, and input the state information to the intelligent agent; An action determination module, used to control the agent to make a decision action and input the decision action into the simulation environment; The updating module is used to control the simulation environment to update the node status in the decision action according to the decision action and determine the reward obtained by executing the decision action in the current state.

12. The refinery crude oil intelligent scheduling system according to claim 11, characterized in that: Determining the correspondence between the Markov transformation and the refinery crude oil scheduling simulation environment data includes: Determine the set of state space, action space and agent rewards for each agent in the scheduling task.

13. The refinery crude oil intelligent scheduling system according to claim 10, characterized in that: The rewards for the refinery scheduling problem determined according to the corresponding relationship include: The rewards for refinery scheduling problems are determined based on the early warning over-limit rewards and penalties, planned over-limit rewards and penalties, fluctuation rewards and penalties, and costs.

14. The refinery crude oil intelligent scheduling system according to claim 13, characterized in that: The warning over-limit reward and punishment are determined by the periodic liquid level, safety liquid level and tank quantity limit; The planned over-limit reward and penalty is determined by the difference between the sum of the periodic oil intake of the atmospheric and vacuum distillation device and the planned total processing volume; The fluctuation reward and penalty is determined by the total number of pipeline speed switching, the pipeline speed switching cost, the total number of switching of the processing scheme of the atmospheric and vacuum distillation device, and the scheme switching cost; The cost is determined by the total delay time for tanker unloading, the cost of delayed unloading, the total inventory of terminal tanks, the cost of terminal tank inventory, the total inventory of in-plant tanks, the cost of in-plant tank inventory, the total volume of long-distance pipeline transportation and the transportation cost.

15. The refinery crude oil intelligent scheduling system according to claim 14, characterized in that: The policy network and value network of deep reinforcement learning are constructed through the SAC framework network.

Images