Crude oil supply chain scheduling optimization method, system, device, medium and product

Abstract

This application provides a method, system, equipment, medium, and product for optimizing crude oil supply chain scheduling, relating to the field of petroleum industry chain optimization technology. The method involves: determining a target operations operator based on current state information; determining the target inventory level and target inventory penalty function for each node in the crude oil supply chain based on current inventory levels and the target operations operator; determining the objective function of a mathematical programming model using a preset oil transportation cost function and the target inventory penalty function; determining a production scheduling plan for the day based on current inventory levels and the objective function; executing the production scheduling plan for the day to obtain daily cost information; determining a reward function for the day based on the daily cost information; obtaining the daily reward function for each day within a preset time period; accumulating the daily reward functions within the preset time period to obtain the accumulated reward function; if the accumulated reward function value is not the maximum value, using a backpropagation algorithm to adjust the parameters to obtain an adjusted reinforcement learning layer.

Description

Technical Field

[0001] This application relates to the field of petroleum industry chain optimization technology, and in particular to a crude oil supply chain scheduling optimization method, system, equipment, medium and product. Background Technology

[0002] In the oil industry chain, crude oil supply chain scheduling is an extremely complex and crucial link, involving multiple stages such as crude oil procurement, transportation, storage, and processing. It requires comprehensive consideration of various resource constraints, market changes, logistics costs, and other factors. Therefore, optimizing crude oil supply chain scheduling is a critical issue. The core of this optimization problem lies in how to rationally arrange crude oil processing and transportation to minimize transportation and storage costs, and ensure that inventory levels do not exceed prescribed limits, thereby avoiding alarms triggered by excessive inventory.

[0003] Currently, crude oil supply chain scheduling optimization is mainly achieved through traditional optimization algorithms. However, traditional optimization algorithms, such as operations research and mathematical programming algorithms, are inadequate when dealing with the large scale, dynamism, and uncertainty of the crude oil supply chain. Especially when optimizing the overall returns over a future period, traditional algorithms typically only provide locally optimal solutions for each day. This strategy is essentially a greedy strategy and cannot guarantee achieving a global optimum over the entire optimization cycle. Summary of the Invention

[0004] This application provides a crude oil supply chain scheduling optimization method, system, equipment, medium, and product to solve the problem that the current stage cannot guarantee the achievement of global optimum throughout the entire optimization cycle.

[0005] In a first aspect, this application provides a crude oil supply chain scheduling optimization method, which is applied to a crude oil supply chain scheduling optimization system, and the method includes:

[0006] The following operations are performed repeatedly until the accumulated reward function value reaches its maximum value, and the production scheduling schemes for each day, sorted by time within a preset time period, are output as the target production scheduling scheme. The following operations include:

[0007] The current state information of the crude oil supply chain is obtained, and a reinforcement learning layer is used to determine the target operation operator based on the current state information. The target operation operator includes the target inventory adjustment strategy and the target inventory penalty function construction method. The current state information of the crude oil supply chain includes the current inventory of oil products at each node in the crude oil supply chain.

[0008] Based on the current inventory level and the target operations operator, the target inventory level and target inventory penalty function of oil products at each node in the crude oil supply chain are determined.

[0009] The objective function of the mathematical programming model is determined by using a preset oil transportation cost function and the target inventory penalty function;

[0010] A solver is used to determine the daily production scheduling scheme that minimizes the objective function when the oil products at each node in the crude oil supply chain reach the target inventory level, based on the current inventory level and the objective function.

[0011] The production scheduling plan for the day is executed using a simulator to obtain the cost information for the day.

[0012] The reinforcement learning layer is employed, and the daily reward function is determined based on the daily cost information;

[0013] Obtain the daily reward function corresponding to each day within a preset time period, and use the reinforcement learning layer to accumulate and calculate the daily reward functions within the preset time period to obtain the accumulated reward function;

[0014] If the accumulated reward function value is not the maximum value, the reinforcement learning layer uses the backpropagation algorithm to adjust the parameters to obtain the adjusted reinforcement learning layer.

[0015] In one possible design, the crude oil supply chain scheduling optimization system includes a preset operations operator layer, which pre-stores multiple preset operations operators. Each preset operations operator includes multiple preset inventory adjustment strategies and a preset inventory penalty function construction method.

[0016] The multiple preset inventory adjustment strategies include refinery refined product adjustment strategy, refinery crude oil adjustment strategy, and pipeline crude oil storage adjustment strategy;

[0017] The refinery refined product adjustment strategy includes any one of the following: increasing refinery refined product inventory to the corresponding expected target, decreasing refinery refined product inventory to the corresponding expected target, or maintaining refinery refined product inventory.

[0018] The refinery crude oil adjustment strategy includes any one of the following: increasing refinery crude oil inventory to the corresponding expected target, or meeting crude oil demand within a preset time period.

[0019] The pipeline crude oil storage adjustment strategy includes adjusting the pipeline crude oil inventory to a preset safety line range;

[0020] The preset inventory penalty function can be constructed in the following ways: piecewise linear construction or multi-objective sequence construction.

[0021] In one possible design, if the target inventory penalty function in the target operations operator is determined to be constructed using a piecewise linear construction method, then the target inventory penalty function for each node in the crude oil supply chain is determined based on the current inventory level and the target operations operator, including:

[0022] Obtain the upper limit, lower limit, upper limit, and lower limit of safe inventory for oil products at each node in the crude oil supply chain;

[0023] Based on the designed lower limit of inventory, the safety stock lower limit, the safety stock upper limit, and the designed upper limit of inventory, multiple inventory ranges are constructed, and each of the inventory ranges has a corresponding preset inventory penalty function;

[0024] The target inventory penalty function is determined from the preset inventory penalty function based on the inventory range in which the current inventory level is located;

[0025] The objective function of the mathematical programming model is determined by using a preset oil transportation cost function and the target inventory penalty function, including:

[0026] The preset oil transportation cost function and the target inventory penalty function are weighted and summed to obtain the objective function of the mathematical programming model.

[0027] In one possible design, the expression for the preset inventory penalty function corresponding to each of the inventory ranges is as follows:

[0028]

[0029] in, This represents the target inventory penalty function. This represents the current inventory of oil products at the corresponding node, where v represents the node (including refinery nodes and pipeline crude oil storage nodes), t represents the current day t, and c represents the oil product (including crude oil and refined oil products); Ub v,c Indicates the upper limit of the designed inventory, Sub v,c This indicates the upper limit of safety stock, Slb. v,c Lb represents the lower limit of safety stock. v,c β1 and β2 represent the design inventory lower limit, respectively, and represent the first penalty coefficient and the second penalty coefficient.

[0030] The objective function of the mathematical programming model is expressed as follows:

[0031]

[0032] in, c represents the pipeline transport volume of the e-th segment in the crude oil supply chain. eThis represents the transportation cost of the e-th pipe segment, and β represents the preset equilibrium coefficient. This is a preset oil transportation cost function.

[0033] In one possible design, if the target inventory penalty function in the target operations operator is determined to be constructed using a multi-objective sequence construction method, then the target inventory penalty function for each node in the crude oil supply chain is determined based on the current inventory level and the target operations operator, including:

[0034] Obtain the upper and lower limits of safe inventory for oil products at each node in the crude oil supply chain; the nodes in the crude oil supply chain include: refineries and crude oil pipeline storage facilities, and the oil products include crude oil and refined oil products;

[0035] A first inventory penalty function is constructed based on the current inventory level, the upper limit and lower limit of the safe inventory of refined oil products in the refinery, a second inventory penalty function is constructed based on the current inventory level, the upper limit and lower limit of the safe inventory of crude oil in the refinery, and a third inventory penalty function is constructed based on the current inventory level, the upper limit and lower limit of the safe inventory of crude oil in the crude oil pipeline storage.

[0036] The first inventory penalty function, the second inventory penalty function, and the third inventory penalty function are determined as the target inventory penalty functions for oil products at each node;

[0037] The objective function of the mathematical programming model is determined by using a preset oil transportation cost function and the target inventory penalty function, including:

[0038] The preset oil transportation cost function and the target inventory penalty functions are determined as the objective functions.

[0039] In one possible design, the expression for the first inventory penalty function is as follows:

[0040]

[0041] The expression for the second inventory penalty function is as follows:

[0042]

[0043] The expression for the third inventory penalty function is as follows:

[0044]

[0045] The expression for the oil transportation cost function is as follows:

[0046]

[0047] in, Sub represents the current inventory level of the corresponding oil product at the specified node. v,c This indicates the upper limit of safety stock, Slb. v,c This indicates the lower limit of safety stock. c represents the pipeline transport volume of the e-th segment in the crude oil supply chain. e This represents the transportation cost of the e-th pipe segment.

[0048] In one possible design, if a solver is used and the target inventory level of oil products at each node in the crude oil supply chain is determined to be unattainable based on the objective function, the method further includes:

[0049] Return infeasible action information to the reinforcement learning layer and output an approximately feasible current production scheduling scheme;

[0050] The reinforcement learning layer is used, and the current reward function is determined based on the daily cost information and the information on actions that cannot be taken.

[0051] In one possible design, the daily cost information includes daily transportation cost information, daily inventory warning information, and daily inventory disposal information;

[0052] The formula for calculating the reward function is as follows:

[0053]

[0054] Where, r t This represents the reward function for that day. This indicates the penalties corresponding to the daily inventory warning information. This indicates the loss items corresponding to the inventory disposal information for that day. This indicates the penalty items corresponding to the oil transportation cost information for that day. The penalty item corresponding to the information of inaction is indicated. k1, k2, and k3 represent the corresponding preset weight values. α takes the value 0 or 1. When the value is 1, it means that the target inventory level is unattainable. When the value is 0, it means that the target inventory level is attainable.

[0055] Secondly, this application provides a crude oil supply chain scheduling optimization system, comprising:

[0056] A reinforcement learning layer is used to acquire the current state information of the crude oil supply chain and determine the target operations operator based on the current state information;

[0057] The operations operator layer is used to determine the target inventory level and target inventory penalty function of oil products at each node in the crude oil supply chain based on the current inventory level and the target operations operator.

[0058] The mathematical programming layer is used to determine the objective function of the mathematical programming model using a preset oil transportation cost function and the target inventory penalty function;

[0059] The mathematical programming layer is also used to use a solver and based on the objective function to determine the daily production scheduling scheme corresponding to minimizing the objective function when the oil products at each node in the crude oil supply chain reach the target inventory level;

[0060] The simulator is used to execute the production scheduling plan for the day in order to obtain the cost information for the day;

[0061] The reinforcement learning layer is also used to determine the daily reward function based on the daily cost information;

[0062] The reinforcement learning layer is also used to obtain the daily reward function corresponding to each day within a preset time period, and to accumulate the daily reward functions within the preset time period to obtain the accumulated reward function.

[0063] The reinforcement learning layer is also used to adjust the parameters using a backpropagation algorithm if the accumulated reward function value is not the maximum value, so as to obtain an adjusted reinforcement learning layer.

[0064] Thirdly, this application provides a crude oil supply chain scheduling optimization device, including: a processor, a memory communicatively connected to the processor, and a transceiver;

[0065] The memory stores the instructions executed by the computer; the transceiver is used to receive and send instructions.

[0066] The processor executes computer-executable instructions stored in memory to implement the method as described in any of the first aspects.

[0067] Fourthly, this application provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement the method as described in any of the first aspects.

[0068] Fifthly, this application provides a computer program product, including a computer program that, when executed by a processor, implements the method as described in the first aspect.

[0069] The crude oil supply chain scheduling optimization method, system, equipment, medium, and product provided in this application perform the following operations cyclically until the accumulated reward function value reaches its maximum value, and output the daily production scheduling schemes sorted by time within a preset time period as the target production scheduling scheme. The following operations include: obtaining the current state information of the crude oil supply chain; employing a reinforcement learning layer and determining a target operations operator based on the current state information; the target operations operator includes a target inventory adjustment strategy and a target inventory penalty function construction method; the current state information of the crude oil supply chain includes the current inventory of oil products at each node in the crude oil supply chain; determining the target inventory of oil products at each node in the crude oil supply chain and the target inventory penalty function based on the current inventory and the target operations operator; and employing a preset oil product... The transportation cost function and the target inventory penalty function determine the objective function of the mathematical programming model. A solver is used, and based on the objective function, a daily production scheduling scheme is determined that minimizes the objective function when the oil products at each node in the crude oil supply chain reach the target inventory level. A simulator is used to execute the daily production scheduling scheme to obtain daily cost information. The reinforcement learning layer is used, and based on the daily cost information, a daily reward function is determined. The daily reward function for each day within a preset time period is obtained, and the reinforcement learning layer accumulates these daily reward functions within the preset time period to obtain the accumulated reward function. If the accumulated reward function value is not the maximum value, the reinforcement learning layer uses a backpropagation algorithm to adjust the parameters to obtain an adjusted reinforcement learning layer. Since the goal is to optimize the overall revenue of the crude oil supply chain over a future period and find the global optimum, the current state information of the crude oil supply chain is obtained. A reinforcement learning layer is used, and based on the current state information, a target operations operator is determined. This target operations operator includes a target inventory adjustment strategy and a method for constructing the target inventory penalty function.Therefore, by using the target operations operator and the current inventory level, the target inventory level and target inventory penalty function for each node in the crude oil supply chain can be determined. Then, using a preset oil transportation cost function and the target inventory penalty function, the objective function of the mathematical programming model can be determined. This allows the solver to use the current inventory level and the objective function to solve for the daily production scheduling scheme corresponding to the minimization of the objective function when the oil at each node in the crude oil supply chain reaches the target inventory level. The daily production scheduling scheme is then simulated using a simulator to obtain the daily cost information. Further, a reinforcement learning layer is used to determine the daily reward function based on the daily cost information. Since this is a supply chain scheduling optimization within a preset time period, the above steps are repeated to obtain the daily cost for each day within the preset time period. The algorithm calculates the daily reward function and uses a reinforcement learning layer to accumulate the daily reward functions within a preset time period. To ensure a globally optimal solution, the accumulated reward function value is evaluated. If the accumulated reward function value is not the maximum value, the reinforcement learning layer adjusts its parameters using a backpropagation algorithm to obtain an adjusted reinforcement learning layer. This process is repeated until the accumulated reward function value reaches its maximum value. Finally, the production scheduling schemes for each day within the preset time period, sorted by time, are output as the target production scheduling scheme. By combining traditional optimization algorithms with reinforcement learning, this approach can adapt to the dynamic uncertainties of the crude oil supply chain every day, and the optimal production scheduling scheme obtained is the globally optimal solution. Attached Figure Description

[0070] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0071] Figure 1 This is an application scenario diagram of the crude oil supply chain scheduling optimization method provided in one embodiment of this application;

[0072] Figure 2 A flowchart of a crude oil supply chain scheduling optimization method provided in an embodiment of this application;

[0073] Figure 3 A schematic diagram illustrating the value of the penalty coefficient in a linear inventory penalty function provided in an embodiment of this application;

[0074] Figure 4 A schematic diagram illustrating the value of the penalty coefficient in another linear inventory penalty function provided in an embodiment of this application;

[0075] Figure 5 A schematic diagram of the training process of a crude oil supply chain scheduling optimization system provided in another embodiment of this application;

[0076] Figure 6A schematic diagram of the structure of a crude oil supply chain scheduling optimization system provided in an embodiment of this application;

[0077] Figure 7 A schematic diagram of the structure of a crude oil supply chain scheduling optimization device provided in an embodiment of this application.

[0078] The accompanying drawings illustrate specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through reference to particular embodiments. Detailed Implementation

[0079] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0080] To clearly understand the technical solution of this application, the solutions of the prior art will be described in detail first.

[0081] In actual production practice, the transportation and supply of oil are dynamic and involve many uncertainties. Crude oil supply chain scheduling optimization is a time-dependent high-dimensional sparse linear programming problem. Therefore, research and application of crude oil resource scheduling optimization are mainly at the long-term strategic level, typically on a monthly or yearly basis. Currently, for crude oil supply chain scheduling optimization within a preset time period, traditional optimization algorithms such as operations research and mathematical programming algorithms are commonly used to calculate the optimal strategy for each day's objective. However, this strategy is essentially a greedy strategy, considering only the optimal solution for the current day without considering the impact of future time points, which may lead to a decline in overall efficiency throughout the entire optimization cycle.

[0082] Therefore, when facing technical problems in existing technologies, the transportation and supply of oil in actual production practice are dynamic and involve many uncertainties. Thus, traditional mathematical programming alone cannot obtain the globally optimal solution. Reinforcement learning is a very suitable tool for dealing with dynamic optimization problems over finite or infinite time. By combining traditional optimization algorithms with reinforcement learning, a three-layer optimization framework is proposed, including a reinforcement learning layer, an operations research operator layer, and a mathematical programming layer. The highest layer is the reinforcement learning layer, the middle layer is the operations research operator layer, and the lowest layer is the mathematical programming layer. The operations research operator layer provides multiple preset operations research operators, and the reinforcement learning layer obtains the current state information of the crude oil supply chain, using operations research operator selection as an action to determine the target operations research operator. The operations research operator layer determines the target inventory level and target inventory penalty function for each node in the crude oil supply chain based on the current inventory level and the target operations research operator. The mathematical programming layer determines the objective function of the mathematical programming model using a preset oil transportation cost function and the target inventory penalty function. A solver then determines the daily production scheduling plan corresponding to minimizing the objective function when the target inventory level is reached at each node in the crude oil supply chain, based on the current inventory level and the objective function. A simulator is then used to execute the daily production scheduling plan to obtain the daily cost information. The reinforcement learning layer can then determine the daily reward function based on this cost information. Since this solution addresses the crude oil supply chain scheduling over a period of time... To address the optimization problem, the above steps are repeated within a preset time period to obtain the daily reward function for each day within that period. A reinforcement learning layer is then used to accumulate the daily reward function within the preset time period. The accumulated reward function is then checked to determine if it is the maximum value. If it is not the maximum value, the reinforcement learning layer uses a backpropagation algorithm to adjust its parameters to obtain an adjusted reinforcement learning layer. The above steps are repeated until the accumulated reward function reaches its maximum value. Finally, the production scheduling schemes for each day within the preset time period, sorted by time, are output as the target production scheduling scheme, thus ensuring global optimization throughout the entire optimization cycle.

[0083] Figure 1 This is an application scenario diagram of the crude oil supply chain scheduling optimization method provided in one embodiment of this application, such as... Figure 1As shown in the embodiments of this application, the application scenario corresponding to the crude oil supply chain scheduling optimization method includes: a client device 101 and a crude oil supply chain scheduling optimization system 102. The crude oil supply chain scheduling optimization system 102 includes a reinforcement learning layer 103, an operations research operator layer 104, a mathematical programming layer 105, and a simulator 106. Users can select a preset time period and trigger a production scheduling plan generation request on the client device 101 and send it to the crude oil supply chain scheduling optimization system 102. After receiving the preset time period and the production scheduling plan generation request, the reinforcement learning layer 103 obtains the current state information of the crude oil supply chain from the simulator 106 and determines the target operations research operator based on the current state information. The current state information of the crude oil supply chain includes the current inventory and current supply and demand of oil products at each node in the crude oil supply chain. The target operations research operator includes a target inventory adjustment strategy and a target inventory penalty function construction method. The reinforcement learning layer 103 sends the determined target operations operator to the operations operator layer 104. The operations operator layer 104 determines the target inventory level and target inventory penalty function for each node in the crude oil supply chain based on the current supply and demand and the target operations operator, and sends this information to the mathematical programming layer 105. The mathematical programming layer 105 obtains the current inventory level of each node in the crude oil supply chain from the simulator 106, and determines the objective function of the mathematical programming model using a preset oil transportation cost function and target inventory penalty function. It then uses a solver and, based on the objective function, determines the daily production scheduling plan corresponding to minimizing the objective function when each node in the crude oil supply chain reaches the target inventory level. Finally, it sends the determined daily production scheduling plan to the simulator 106, which executes the daily production scheduling plan to obtain the daily cost information. The system receives the daily cost information and sends it to the reinforcement learning layer 103. The reinforcement learning layer 103 determines the daily reward function based on the daily cost information. By repeatedly executing the above steps, it obtains the daily reward function corresponding to each day within the preset time period. The reinforcement learning layer 103 accumulates the daily reward functions within the preset time period to obtain the accumulated reward function. It then determines whether the accumulated reward function is the maximum value. If the accumulated reward function is not the maximum value, the reinforcement learning layer 103 uses the backpropagation algorithm to adjust the parameters to obtain the adjusted reinforcement learning layer 103. If the accumulated reward function reaches the maximum value, the mathematical programming layer 105 outputs the daily production scheduling schemes sorted by time within the preset time period as the target production scheduling scheme and sends the target production scheduling scheme to the client device 101.

[0084] The technical solution of this application and how the technical solution of this application solves the above-mentioned technical problems are described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will now be described with reference to the accompanying drawings.

[0085] Figure 2 A flowchart of a crude oil supply chain scheduling optimization method provided in an embodiment of this application is shown below. Figure 2 As shown, the execution entity in this embodiment is a crude oil supply chain scheduling optimization system. This system can be implemented through a computer program, or through a medium storing the relevant computer program, such as a USB flash drive and / or optical disc; alternatively, it can be implemented through a physical device integrating or installing the relevant computer program, such as a chip or electronic device. The crude oil supply chain scheduling optimization method provided in this embodiment includes the following steps:

[0086] S201: Obtain the current state information of the crude oil supply chain, adopt a reinforcement learning layer, and determine the target operations operator based on the current state information.

[0087] The target operations operator includes the target inventory adjustment strategy and the method for constructing the target inventory penalty function. The current state information of the crude oil supply chain refers to the current state information of oil products at each node in the supply chain, including the current inventory level of oil products at each node. The current inventory level refers to the current quantity of oil products in stock at each node. The current state information of the crude oil supply chain also includes the current supply of crude oil and the current demand for refined oil products.

[0088] The nodes may include refineries and pipeline crude oil storage facilities, and the oil products include crude oil and refined oil products.

[0089] Optionally, the nodes in the crude oil supply chain include oilfield nodes, procurement nodes, crude oil pipeline storage nodes, refinery nodes, and pipeline sections connecting the nodes. The nodes can be selected and used according to actual needs, and no limitation is made in this embodiment.

[0090] Understandably, a refinery, also known as an oil refinery, is a major production and operating entity in the petroleum refining industry, used to produce petroleum products and petrochemical raw materials. Pipeline crude oil storage facilities are important facilities for storing and transporting crude oil. They connect oil fields, refineries, and other related facilities via pipeline systems to achieve the storage and transfer of crude oil. Crude oil is unprocessed petroleum. Refined petroleum products are produced through the processing of crude oil, and can include gasoline, diesel, etc.

[0091] The target operations operator is an operations operator determined by the reinforcement learning layer from a set of preset operations operators provided by the operations operator layer based on the current supply and demand. The target inventory adjustment strategy is the inventory adjustment strategy included in the target operations operator. The method for constructing the target inventory penalty function is the method for constructing the inventory penalty function included in the target operations operator.

[0092] The preset operations operators include preset inventory adjustment strategies and preset inventory penalty function construction methods. The preset inventory adjustment strategies include refinery refined product adjustment strategies, refinery crude oil adjustment strategies, and pipeline crude oil storage adjustment strategies.

[0093] The refinery refined product adjustment strategies may include increasing refinery refined product inventories to corresponding expected targets, decreasing refinery refined product inventories to corresponding expected targets, or maintaining refinery refined product inventories. Refinery crude oil adjustment strategies may include increasing refinery crude oil inventories to corresponding expected targets, decreasing refinery crude oil inventories to corresponding expected targets, or meeting crude oil demand within a preset time period. Pipeline crude oil storage adjustment strategies may include adjusting pipeline crude oil inventories to a preset safety line range. The inventory penalty function can be constructed using piecewise linear methods, multi-objective sequence methods, or stepped loss functions, etc.

[0094] Optionally, the refinery's strategies for increasing and decreasing refined oil product inventory can be further subdivided according to different expected targets, such as increasing refined oil product inventory by 1%, 2%, 5%, 10% or increasing it to the safety stock level, and decreasing refined oil product inventory by 1%, 2%, 5%, 10% or decreasing it to the safety stock level, etc. This embodiment does not impose any limitations.

[0095] Optionally, the refinery's strategy for increasing crude oil inventory can be further subdivided according to different expected targets, such as increasing crude oil inventory by 1%, 2%, 5%, 10%, or increasing it to a safe inventory level, etc., which is not limited in this embodiment.

[0096] Optionally, the inventory penalty function can be constructed in a piecewise linear manner, a multi-objective sequence manner, a step loss function, or other manners, which are not limited in this embodiment.

[0097] It is understandable that the number of operations operators can increase by further subdividing any one of the inventory adjustment strategies, by increasing the adjustment actions of any one of the inventory adjustment strategies, or by increasing the way the inventory penalty function is constructed. This embodiment does not impose any limitations.

[0098] Specifically, each operations operator includes any one of the refinery refined product adjustment strategies, any one of the refinery crude oil adjustment strategies, any one of the crude oil storage adjustment strategies, and any one of the inventory penalty function construction methods.

[0099] Specifically, for the reinforcement learning layer, a deep reinforcement learning framework consisting of reinforcement learning and neural networks includes a reinforcement learning agent. When using reinforcement learning, the production scheduling problem in the crude oil supply chain is modeled as a Markov Decision Process (MDP) problem. An MDP problem includes a state space, action space, state transition function, and reward function. Reinforcement learning is a machine learning method that allows an agent to interact with its environment and learn how to take actions to maximize the reward function. The environment is the world the agent interacts with, providing state information and immediate rewards. The agent is the subject of learning and decision-making, aiming to maximize cumulative rewards by taking a series of actions. Neural networks are used to learn the mapping from state to action or from state to value.

[0100] Optionally, the nodes in the crude oil supply chain include oil field nodes, procurement nodes, crude oil pipeline storage nodes, refinery nodes, and pipeline segments connecting the nodes, etc., which are not limited in this embodiment.

[0101] In this process, oilfield nodes produce crude oil, procurement nodes purchase crude oil, and oilfield nodes and procurement nodes transport crude oil directly or through crude oil pipeline storage nodes to refinery nodes. Refinery nodes process crude oil into refined oil products. The daily supply of crude oil and demand for refined oil products are dynamic and change, forming a random environment.

[0102] Markov Decision Processes (MDPs) are an important concept in reinforcement learning, a mathematical model used to describe the process of making decisions in uncertain environments. The state space defines the set of all possible states an agent can be in. The action space contains the set of all possible actions the agent can take in each state. The state transition function describes the probability distribution of transitioning to the next state after taking an action from the current state.

[0103] For the production scheduling problem in the crude oil supply chain, its state space corresponds to the state at the end of each time period in the crude oil supply chain, such as the state at the end of each day. This is a continuous space, including the upper and lower limits of inventory at oil fields, pipeline crude oil storage facilities, and refineries, the remaining processing capacity of refineries, and the demand for refined oil products. The action space consists of operations operators, with each node corresponding to multiple operations operators. These operators represent the methods for constructing inventory adjustment strategies and inventory penalty functions for each node in the crude oil supply chain. The state transition function is the probability of the system transitioning to the next state given the current state and the actions performed. Since the demand at the end of the refinery node changes with the market, the state transition probability function is a random variable related to the daily inventory level and the output of the reinforcement learning actions. The reward function is determined by calculating the daily inventory warning penalty, inventory discard loss, and transportation cost penalty based on the daily state information of the crude oil supply chain.

[0104] The inventory warning penalty item reflects the penalty cost when inventory levels approach or exceed the warning range. The inventory disposal loss item reflects the cost of the amount of inventory that needs to be disposed of due to excess inventory. The transportation cost penalty item reflects the additional costs incurred during transportation.

[0105] Specifically, in this embodiment, the reinforcement learning layer obtains the current state information of the crude oil supply chain from the simulator, and the agent in the reinforcement learning layer makes corresponding decisions based on the current state information, thereby determining the target operation operator, and sending the determined target operation operator to the operation operator layer.

[0106] S202 determines the target inventory level and target inventory penalty function for each node in the crude oil supply chain based on the current inventory level and the target operations operator.

[0107] The target inventory level is calculated based on the target inventory adjustment strategy included in the target operations operator and the current inventory levels of oil products at each node in the crude oil supply chain. The target inventory penalty function is an inventory penalty function constructed according to the target inventory penalty function construction method in the target operations operator.

[0108] Specifically, in this embodiment, the operations research operator layer calculates the target inventory levels of oil products at each node in the crude oil supply chain according to the target inventory adjustment strategy in the target operations research operator and based on the current inventory levels of the corresponding oil products at each node in the crude oil supply chain. Examples of target inventory levels include the refined oil product target inventory level of the refinery, the crude oil target inventory level of the refinery, and the crude oil target inventory level of the pipeline crude oil depot. The operations research operator layer constructs the target inventory penalty function according to the target inventory penalty function construction method in the target operations research operator, and sends the calculated target inventory levels of oil products at each node in the crude oil supply chain and the determined target inventory penalty function to the mathematical programming layer.

[0109] S203 uses a preset oil transportation cost function and a target inventory penalty function to determine the objective function of the mathematical programming model.

[0110] The preset oil transportation cost function is a pre-defined function for calculating oil transportation costs. The objective function is a function based on the preset oil transportation cost function and the target inventory penalty function.

[0111] Understandably, mathematical programming models are a mathematical expression for optimization problems, which find the optimal solution by defining an objective function and constraints.

[0112] The expression for the preset oil transportation cost function can be as follows:

[0113]

[0114] in, c represents the pipeline transport volume of the e-th segment in the crude oil supply chain. e This represents the transportation cost of the e-th pipe segment.

[0115] Specifically, in this embodiment, the mathematical programming layer combines the obtained preset oil transportation cost function and target inventory penalty function in a preset manner to obtain the objective function of the mathematical programming model.

[0116] Optionally, the preset method can be addition, etc., and this embodiment is not limited to this.

[0117] S204 uses a solver and is based on the objective function to determine the daily production scheduling scheme corresponding to minimizing the objective function when the oil products at each node in the crude oil supply chain reach the target inventory level.

[0118] Understandably, the solver is used to solve the crude oil supply chain scheduling optimization problem based on a mathematical programming model. The daily production scheduling plan is the production scheduling plan for each node in the crude oil supply chain corresponding to the current state information.

[0119] Specifically, in this embodiment, the mathematical programming model includes not only the objective function but also constraints. These constraints include constraints on the transportation capacity of different modes of transport at the crude oil pipeline and refinery, as well as constraints on the inventory levels of oil products at the end of the day at each node, as shown below:

[0120] (1) Oilfield node constraints

[0121] For each oilfield node v∈V c The daily crude oil inventory (T=1,2,…) cannot exceed the upper or lower limits:

[0122]

[0123] in, Let V be the production of oil field during cycle T. This represents the pipeline transport volume of the e-th segment in the crude oil supply chain, where e =<v,v′> This refers to the starting and ending points of the pipeline, E c Let p be the collection of pipelines e. Average daily crude oil production (10,000 tons) v Crude oil design inventory lower limit lc v and upper limit uc v Initial crude oil inventory V c It is a set of oilfield nodes.

[0124] (2) Procurement Node Constraints

[0125] For each procurement node v∈V s The daily crude oil inventory (T=1,2,…) cannot exceed the upper or lower limits:

[0126]

[0127] The daily crude oil arrival volume at procurement node v (T=1,2,…) is (ten thousand tons). Procurement node design inventory lower limit lc v and upper limit uc v Initial crude oil inventory V s This is a set of procurement nodes.

[0128] (3) Crude oil pipeline storage node constraints

[0129] For each crude oil pipeline storage node v∈V b The daily crude oil inventory (T=1,2,…) does not exceed the upper or lower limits:

[0130]

[0131] Among them, the lower limit of the design inventory of crude oil pipeline storage nodes lc v and upper limit uc v Initial crude oil inventory t e This represents the transit time of inventory due to cross-day shipping. (Tt) e This can be understood as the cycle after considering transit time. vi refers to each crude oil pipeline storage node. V b It refers to the set of nodes in the crude oil pipeline storage area.

[0132] (4) Refinery node constraints

[0133] For each refinery node v∈V r The daily processing volume (T = 1, 2, ...) cannot exceed the maximum production capacity.

[0134]

[0135] The inventory levels of crude oil, gasoline, and diesel should all remain within the upper and lower limits.

[0136]

[0137] Among them, pg vi For gasoline grade i, production volume (in ten thousand tons), pd vi For diesel fuel grade i, the production volume (in ten thousand tons) is given, and the designed lower limit of crude oil inventory is given. v and upper limit uc v Initial crude oil inventory ug v The design limit for gasoline inventory, initial inventory of grade i gasoline ud v Designed upper limit for diesel fuel inventory, initial inventory of grade i diesel fuel. Let N be the total crude oil processing volume at refinery node v, where N is the total crude oil processing volume at refinery node v. r This refers to the number of refineries. v = 1, ..., N r ,i=1…N PGLE For the production volume of various types of gasoline at each refinery node, where N PGLE This refers to the quantity of different types of gasoline. For each type of crude oil, the volume transported per cycle from the oilfield node to the pipeline crude oil depot, from the procurement node to the pipeline crude oil depot, and from the pipeline crude oil depot to the refinery node is N. e Number of edges. V r p is the set of refinery nodes. r This indicates the node's production capacity. This indicates the daily processing volume of a refinery node.

[0138] (5) Path constraints

[0139]

[0140] For each edge (transport path) e∈E, it includes: oil s e ∈{c crude oil, i grade gasoline, g} i Diesel brand (i) i The daily (T=1,2,…) maximum outbound volume (in ten thousand tons) per transport edge. T is a positive integer.

[0141] Specifically, in this embodiment, the mathematical programming layer uses a solver to calculate the production scheduling plan for the day when the objective function of each node in the crude oil supply chain reaches the target inventory level, based on the objective function of the mathematical programming model and the preset constraints. The production scheduling plan for the day is then sent to the simulator.

[0142] Optionally, the preset solution method may vary depending on the solver, and this embodiment does not impose any limitations.

[0143] S205 uses a simulator to execute the daily production scheduling plan in order to obtain the daily cost information.

[0144] The daily cost information refers to the cost information generated after executing the daily production scheduling plan. This includes current transportation cost information, current inventory warning information, and current inventory disposal information. Current transportation cost information refers to the cost of transporting oil products from one node to another within the current time period. Current inventory warning information is a warning issued when the current inventory level exceeds or falls below a preset safety stock level. Current inventory disposal information refers to the portion of inventory that needs to be disposed of or processed within the current time period due to certain reasons (such as expiration, damage, changes in market demand, etc.).

[0145] The simulator is used to simulate the operation of the crude oil supply chain.

[0146] Specifically, in this embodiment, a simulator is used to calculate inventory changes based on the operating volume information given by the production scheduling plan for the day, thereby obtaining the cost information for the day, and sending it to the reinforcement learning layer.

[0147] Furthermore, after executing the production scheduling plan for the day, the simulator calculates the current inventory of oil products at each node in the crude oil supply chain for the next day according to the transportation logic corresponding to the production scheduling plan for the day.

[0148] The simulator 104 contains the following node types and pipes:

[0149] (1) Oil field nodes, for each oil field node v∈V c This includes: average daily crude oil production (10,000 tons) p v Crude oil design inventory lower limit lc v and upper limit uc v Initial crude oil inventory

[0150] Among them, V c It is a set of oilfield nodes.

[0151] (2) Procurement nodes, for each procurement node v∈V sThis includes: the daily crude oil arrival volume (in ten thousand tons) at procurement node v (T=1,2,…). Procurement node design inventory lower limit lc v and upper limit uc v Initial crude oil inventory

[0152] Among them, V s This is a set of procurement nodes.

[0153] (3) Crude oil pipeline storage nodes, for each crude oil pipeline storage node v∈V b This includes: the design inventory lower limit for crude oil pipeline storage nodes (lc) v and upper limit uc v Initial crude oil inventory

[0154] Among them, V b It is a set of pipeline nodes.

[0155] (4) Refinery nodes, for each refinery node v∈V r This includes: gasoline grade i, production volume (in ten thousand tons), and output per gram (pg). vi Diesel grade i, production volume (in ten thousand tons) pd vi Crude oil processing volume (10,000 tons) dp v Gasoline yield rg v diesel yield v Crude oil design inventory lower limit lc v and upper limit uc v Initial crude oil inventory Gasoline design inventory limit ug v Initial inventory of gasoline of brand name i Diesel design inventory limit ud v Initial inventory of grade i diesel

[0156] Among them, V r This is a set of refinery nodes.

[0157] (5) Edges, for each edge (transport path) e∈E, including: oil s e ∈{c crude oil, i grade gasoline, g} i Diesel brand (i) i The daily (T=1,2,…) maximum outbound volume (in ten thousand tons) per transport edge. Daily (T=1,2,…) transportation cost per transport edge (yuan / 10,000 tons) c e .

[0158] S206 employs a reinforcement learning layer and determines the daily reward function based on the daily cost information.

[0159] In reinforcement learning, the reward function is a key component guiding the agent's learning. The reward function defines the immediate reward the agent receives after taking an action at each time step; these rewards are used to guide the agent in learning the optimal policy. The daily reward function is the reward the agent receives after executing the daily production scheduling plan.

[0160] Specifically, in this embodiment, the reinforcement learning layer inputs the cost information of the day into the formula corresponding to the preset reward function of the day for calculation, thereby deriving the reward function of the day.

[0161] S207: Obtain the daily reward function for each day within a preset time period, and use a reinforcement learning layer to accumulate the daily reward functions within the preset time period to obtain the accumulated reward function.

[0162] Optionally, the duration of the preset time period can be one week, one month, etc., and this embodiment does not limit this.

[0163] Specifically, in this embodiment, S201 to S207 are executed cyclically within a preset time period to obtain the daily reward function. The daily reward functions within the preset time period are accumulated and added together by the reinforcement learning layer to obtain the accumulated reward function.

[0164] S208, determine whether the accumulated reward function value is the maximum value.

[0165] S209. If the accumulated reward function value is not the maximum value, the reinforcement learning layer uses the backpropagation algorithm to adjust the parameters to obtain the adjusted reinforcement learning layer.

[0166] Understandably, the backpropagation algorithm is used to adjust the parameters of a neural network to maximize the cumulative reward. When the accumulated reward function value is not at its maximum, adjusting the parameters through backpropagation can help the agent learn the optimal policy more effectively.

[0167] Specifically, in this embodiment, the reinforcement learning layer uses the backpropagation algorithm to calculate the parameters in the neural network of the reinforcement learning layer by obtaining the accumulated reward function value, thereby updating the parameters through a preset optimization algorithm to obtain the adjusted reinforcement learning layer.

[0168] Optionally, the parameter can be a gradient or a weight gradient, etc., which is not limited in this embodiment.

[0169] Optionally, the preset optimization algorithm may be gradient descent, etc., and this embodiment is not limited to it.

[0170] S210, if the accumulated reward function value reaches the maximum value, then output the production scheduling schemes of each day sorted by time within the preset time period as the target production scheduling scheme.

[0171] The crude oil supply chain scheduling optimization method provided in this embodiment performs the following operations cyclically until the accumulated reward function value reaches its maximum value, and outputs the daily production scheduling schemes sorted by time within a preset time period as the target production scheduling scheme. The following operations include: obtaining the current state information of the crude oil supply chain; using a reinforcement learning layer and determining the target operations operator based on the current state information; the target operations operator includes a target inventory adjustment strategy and a target inventory penalty function construction method; the current state information of the crude oil supply chain includes the current inventory of oil products at each node in the crude oil supply chain; determining the target inventory of oil products at each node in the crude oil supply chain and the target inventory penalty function based on the current inventory and the target operations operator; and using a preset oil product transportation... The objective function of the mathematical programming model is determined by the cost function and the target inventory penalty function. A solver is used, and based on the objective function, a daily production scheduling scheme is determined that minimizes the objective function when the oil products at each node of the crude oil supply chain reach the target inventory level. A simulator is used to execute the daily production scheduling scheme to obtain daily cost information. A reinforcement learning layer is used, and based on the daily cost information, a daily reward function is determined. The daily reward function for each day within a preset time period is obtained, and the reinforcement learning layer accumulates these daily reward functions to obtain the accumulated reward function. If the accumulated reward function value is not the maximum value, the reinforcement learning layer uses a backpropagation algorithm to adjust the parameters to obtain an adjusted reinforcement learning layer. Since the goal is to optimize the overall revenue of the crude oil supply chain over a future period and find the global optimum, the current state information of the crude oil supply chain is obtained. A reinforcement learning layer is used, and based on the current state information, a target operations operator is determined. This target operations operator includes a target inventory adjustment strategy and a method for constructing the target inventory penalty function.Therefore, by using the target operations operator and the current inventory level, the target inventory level and target inventory penalty function for each node in the crude oil supply chain can be determined. Then, by using a preset oil transportation cost function and the target inventory penalty function, the objective function of the mathematical programming model can be determined. This allows the solver to use the current inventory level and the objective function to solve for the daily production scheduling scheme corresponding to the achievement of the target inventory level and the minimization of the objective function for each node in the crude oil supply chain. The daily production scheduling scheme is then simulated using a simulator to obtain the daily cost information. Further, a reinforcement learning layer is used to determine the daily reward function based on the daily cost information. Since this is a supply chain scheduling optimization within a preset time period, the above steps are repeated to obtain the daily cost for each day within the preset time period. The reward function is calculated, and a reinforcement learning layer is used to accumulate the reward functions of each day within a preset time period to obtain the accumulated reward function. To ensure that the global optimal solution is obtained, the accumulated reward function value is judged. If the accumulated reward function value is not the maximum value, the reinforcement learning layer is used to adjust the parameters through the backpropagation algorithm to obtain the adjusted reinforcement learning layer. The above steps are repeated until the accumulated reward function value reaches the maximum value. Then, the production scheduling schemes of each day within the preset time period are output in time order as the target production scheduling scheme. By combining traditional optimization algorithms and reinforcement learning, it can adapt to the dynamic uncertainty factors of the crude oil supply chain every day, and the optimal production scheduling scheme obtained is the global optimal solution.

[0172] As an optional implementation, based on the above embodiments, the crude oil supply chain scheduling optimization system includes an operations management operator layer. The operations management operator layer pre-stores multiple preset operations management operators, and each preset operations management operator includes multiple preset inventory adjustment strategies and a preset inventory penalty function construction method.

[0173] Multiple preset inventory adjustment strategies include refinery refined product adjustment strategy, refinery crude oil adjustment strategy, and pipeline crude oil storage adjustment strategy;

[0174] Refinery refined product inventory adjustment strategies include any of the following: increasing refinery refined product inventory to the corresponding expected target, decreasing refinery refined product inventory to the corresponding expected target, or maintaining refinery refined product inventory.

[0175] Refinery crude oil adjustment strategies include any of the following: increasing refinery crude oil inventories to the corresponding expected target, or meeting crude oil demand within a preset time period;

[0176] Pipeline crude oil storage adjustment strategies include adjusting pipeline crude oil inventory to a preset safety level.

[0177] The preset inventory penalty function can be constructed in either a piecewise linear manner or a multi-objective sequence manner.

[0178] Piecewise linear construction is a modeling method used to capture nonlinear relationships between variables. Multi-objective sequence construction is a modeling method used to handle the simultaneous optimization of multiple objectives.

[0179] Specifically, each preset operations operator consists of a set of four actions comprised of preset inventory adjustment strategies and a preset inventory penalty function construction method. This includes multiple preset inventory adjustment strategies and one preset inventory penalty function construction method. The preset inventory adjustment strategies include refinery refined product adjustment strategies, refinery crude oil adjustment strategies, and pipeline crude oil storage adjustment strategies. Refinery refined product adjustment strategies may include increasing refinery refined product inventory to a corresponding expected target, decreasing refinery refined product inventory to a corresponding expected target, and maintaining refined product inventory. Refinery crude oil adjustment strategies may include increasing refinery crude oil inventory to a corresponding expected target and meeting crude oil demand within a preset time period. Pipeline crude oil storage adjustment strategies may include adjusting pipeline crude oil inventory to a preset safety line range. The preset inventory penalty function construction method may include a piecewise linear construction method and a multi-objective sequence construction method.

[0180] Optionally, the expected target can be set independently according to the needs, and this embodiment does not impose any restrictions.

[0181] Optionally, the preset time period can be set independently according to the requirements, and this embodiment does not impose any limitations.

[0182] For example, based on the inventory adjustment strategy and the inventory penalty function construction method, 3x2x1x2 = 12 kinds of operation operators can be combined, as shown in Table 1. Table 1 is an example table of operation operators.

[0183] It is understandable that the operations operator layer has multiple preset operations operators stored in advance, which can be selected by the reinforcement learning layer. The reinforcement learning layer can determine the target operations operator from the multiple preset operations operators based on the current supply and demand.

[0184] The crude oil supply chain scheduling optimization method provided in this embodiment includes an operations research operator layer. This layer pre-stores multiple preset operations research operators, each including multiple preset inventory adjustment strategies and a preset inventory penalty function construction method. The preset inventory adjustment strategies include refinery refined product adjustment strategies, refinery crude oil adjustment strategies, and pipeline crude oil storage adjustment strategies. The refinery refined product adjustment strategies include any one of the following: increasing refinery refined product inventory to a corresponding expected target, decreasing refinery refined product inventory to a corresponding expected target, or maintaining refinery refined product inventory. The refinery crude oil adjustment strategies include any one of the following: increasing refinery crude oil inventory to a corresponding expected target or meeting crude oil demand within a preset time period. The pipeline crude oil storage adjustment strategy includes adjusting pipeline crude oil inventory to a preset safety line range. The preset inventory penalty function construction method includes either a piecewise linear construction method or a multi-objective sequence construction method. Because multiple preset inventory adjustment strategies and multiple preset inventory penalty function construction methods are available, multiple preset operations research operators can be constructed based on these strategies and methods in different combinations, thus providing a rich selection of preset operations research operators for the reinforcement learning layer.

[0185] Table 1 Examples of Operations Research Operators

[0186]

[0187] As an optional implementation, based on any of the above embodiments, if the target inventory penalty function in the target operations operator is determined to be constructed using a piecewise linear construction method, then the target inventory penalty function for each node in the crude oil supply chain is determined based on the current inventory level and the target operations operator, including:

[0188] Obtain the upper limit, lower limit, upper limit, and lower limit of safe inventory for oil products at each node in the crude oil supply chain;

[0189] Multiple inventory ranges are constructed based on the design inventory lower limit, safety inventory lower limit, safety inventory upper limit, and design inventory upper limit. Each inventory range has a corresponding preset inventory penalty function.

[0190] The target inventory penalty function is determined from the preset inventory penalty function based on the current inventory level and its corresponding inventory range.

[0191] The objective function of the mathematical programming model is determined by using a pre-defined oil transportation cost function and a target inventory penalty function, including:

[0192] The objective function of the mathematical programming model is obtained by weighted summation of the preset oil transportation cost function and the target inventory penalty function.

[0193] The safety stock upper limit, safety stock lower limit, design stock upper limit, and design stock lower limit are four pre-set thresholds. The design stock upper limit is higher than the safety stock upper limit, and the design stock lower limit is lower than the safety stock lower limit.

[0194] Understandably, the upper limit of the designed inventory can be the maximum amount of oil that the container storing the oil can hold. The lower limit of the designed inventory can be the minimum amount of oil that the container storing the oil can hold.

[0195] Optionally, the upper limit of safety stock, the lower limit of safety stock, the upper limit of design stock, and the lower limit of design stock can be set independently according to needs, and are not limited in this embodiment.

[0196] Specifically, in this embodiment, when the target inventory penalty function in the target operations operator is constructed using a piecewise linear construction method, the mathematical programming layer obtains the upper limit, lower limit, upper limit, and lower limit of the safe inventory of oil products at each node in the crude oil supply chain from the simulator. It then divides the inventory into multiple inventory ranges based on these values, with each range corresponding to a different preset inventory penalty function. Furthermore, it obtains the current inventory level of oil products at each node in the crude oil supply chain and determines the target inventory penalty function from the preset inventory penalty functions based on the current inventory range. This allows for a weighted summation of the preset oil transportation cost function and the determined target inventory penalty function, which is then used as the objective function of the mathematical programming model.

[0197] Optionally, multiple inventory ranges can be defined based on the designed lower inventory limit, safety lower inventory limit, safety upper inventory limit, and designed upper inventory limit of oil products at each node in the crude oil supply chain. These ranges could include: a first range below the designed lower inventory limit; a second range above the designed lower inventory limit but below the safety lower inventory limit; a third range above the safety lower inventory limit but below the safety upper inventory limit; a fourth range above the safety upper inventory limit but below the designed upper inventory limit; and a fifth range above the designed upper inventory limit. When the current inventory levels of oil products at each node in the crude oil supply chain fall within different ranges, the penalty coefficient of the corresponding linear inventory penalty function will be different.

[0198] Specifically, for the business needs of the oil supply chain, the ideal situation is when the current oil inventory is below the upper limit of the safe inventory and above the lower limit of the safe inventory; when the current oil inventory exceeds the upper limit of the safe inventory but is below the upper limit of the designed inventory, an inventory warning is generated; when the current oil inventory is below the lower limit of the safe inventory but above the lower limit of the designed inventory, an inventory warning is generated; when the oil inventory is below the lower limit of the designed inventory or above the upper limit of the designed inventory, it is absolutely not allowed.

[0199] When the target inventory penalty function is constructed using a piecewise linear method, different penalty coefficients can be applied to different ranges of current inventory levels.

[0200] Specifically, for example, such as Figure 3 As shown, Figure 3 This diagram illustrates the values ​​of the penalty coefficient in a linear inventory penalty function provided in one embodiment of this application. As can be seen from the diagram, under ideal conditions (i.e., when the current inventory level of oil is below the upper limit of the safety stock and above the lower limit of the safety stock), the penalty coefficient is 0. Under inventory warning conditions (i.e., when the current inventory level of oil exceeds the upper limit of the safety stock but is below the upper limit of the designed inventory or when the current inventory level of oil is below the lower limit of the safety stock but above the lower limit of the designed inventory), the penalty coefficient is a larger value. Under absolutely unacceptable conditions (i.e., when the inventory level of oil is below the lower limit of the designed inventory or above the upper limit of the designed inventory), the penalty coefficient is the largest.

[0201] Optionally, exemplarily, such as Figure 4 As shown, Figure 4 This diagram illustrates the value of the penalty coefficient in another linear inventory penalty function provided in an embodiment of this application. Specifically, the penalty coefficient is set to its maximum value when the oil inventory is absolutely not allowed to be below the designed lower inventory limit or above the designed upper inventory limit. During inventory warnings, when the current oil inventory exceeds the safety stock limit but is below the designed upper inventory limit, the penalty coefficient is relatively large. When the oil inventory is below the safety stock limit and above the safety stock lower limit, but still some distance from the target inventory level, the penalty coefficient can be greater than 0 for inventory warnings. When the oil inventory is the target inventory level, the penalty coefficient is 0.

[0202] Among them, refinery nodes can adopt Figure 4 The linear inventory penalty function shown can be used for crude oil pipeline storage nodes. Figure 3 The linear inventory penalty function is shown.

[0203] The crude oil supply chain scheduling optimization method provided in this embodiment, if the target inventory penalty function in the target operations operator is determined to be constructed using a piecewise linear construction method, then the target inventory penalty function for each node in the crude oil supply chain is determined based on the current inventory level and the target operations operator. This includes: obtaining the upper limit, lower limit, upper limit, and lower limit of the safety inventory for each node in the crude oil supply chain; constructing multiple inventory ranges based on the lower limit, lower limit, upper limit, and upper limit of the safety inventory, each inventory range having a corresponding preset inventory penalty function; determining the target inventory penalty function from the preset inventory penalty functions based on the inventory range in which the current inventory level is located; and determining the objective function of the mathematical programming model using the preset oil transportation cost function and the target inventory penalty function, including: performing a weighted summation of the preset oil transportation cost function and the target inventory penalty function to obtain the objective function of the mathematical programming model. Because the target inventory penalty function in the target operations operator is constructed using a piecewise linear approach, it allows for more precise control of oil inventory levels at each node in the crude oil supply chain. This avoids the risks of excessively high or low inventory levels. Furthermore, incorporating a pre-defined oil transportation cost function into the objective function optimizes transportation routes and volumes, reducing overall transportation costs. The piecewise linear construction method also makes the target inventory penalty function flexible, allowing for adjustments to the penalty coefficient based on actual conditions, thus better responding to changes in market demand.

[0204] As an optional implementation, based on any of the above embodiments, the expression for the preset inventory penalty function corresponding to each inventory range is as follows:

[0205]

[0206] in, This represents the target inventory penalty function. Ub represents the current inventory of oil products at each node in the crude oil supply chain, where v represents the node (including refinery nodes and pipeline crude oil storage nodes), t represents the current day t, and c represents the oil product (including crude oil and refined oil products); v,c Indicates the upper limit of the designed inventory, Sub v,c This indicates the upper limit of safety stock, Slb. v,c Lb represents the lower limit of safety stock. v,c β1 and β2 represent the design inventory lower limit, respectively, and represent the first penalty coefficient and the second penalty coefficient.

[0207] The objective function of the mathematical programming model is expressed as follows:

[0208]

[0209] in, c represents the pipeline transport volume of the e-th segment in the crude oil supply chain. e This represents the transportation cost of the e-th pipe segment, and β represents the preset equilibrium coefficient. This is a preset oil transportation cost function.

[0210] The second penalty coefficient β2 is greater than the first penalty coefficient β1. β represents a preset equilibrium coefficient, used to balance the importance of the two indicators: oil transportation cost and inventory penalty.

[0211] Optionally, the preset equalization coefficient can be set independently according to preset requirements, and this embodiment does not impose any limitations.

[0212] Specifically, in this embodiment, the target inventory penalty function is constructed using a piecewise linear construction method.

[0213] Understandably, the objective function of a mathematical programming model aims to minimize operating costs and penalties.

[0214] As an optional implementation, based on any of the above embodiments, if the target inventory penalty function in the target operations operator is determined to be constructed using a multi-objective sequence construction method, then the target inventory penalty function for each node in the crude oil supply chain is determined based on the current inventory level and the target operations operator, including:

[0215] Obtain the upper and lower limits of safe inventory for oil products at each node in the crude oil supply chain; each node in the crude oil supply chain includes: refineries and crude oil pipeline storage facilities, and oil products include crude oil and refined oil products;

[0216] A first inventory penalty function is constructed based on the current inventory level, the upper limit and lower limit of the safe inventory of refined oil products in the refinery, a second inventory penalty function is constructed based on the current inventory level, the upper limit and lower limit of the safe inventory of crude oil in the refinery, and a third inventory penalty function is constructed based on the current inventory level, the upper limit and lower limit of the safe inventory of crude oil in the crude oil pipeline storage.

[0217] The first inventory penalty function, the second inventory penalty function, and the third inventory penalty function are determined as the target inventory penalty functions for oil products at each node;

[0218] The objective function of the mathematical programming model is determined by using a pre-defined oil transportation cost function and a target inventory penalty function, including:

[0219] The preset oil transportation cost function and the target inventory penalty function are defined as the objective functions.

[0220] The first inventory penalty function is constructed from the current inventory level, the upper and lower limits of the refinery's safe inventory of refined oil products. The second inventory penalty function is constructed from the current inventory level, the upper and lower limits of the refinery's safe inventory of crude oil. The third inventory penalty function is constructed from the current inventory level, the upper and lower limits of the safe inventory of crude oil in the pipeline storage.

[0221] Specifically, in this embodiment, if the target inventory penalty function in the target operations operator is determined to be constructed using a multi-objective sequence construction method, the mathematical programming layer obtains the upper and lower safety stock limits for crude oil and refined oil products in the crude oil supply chain, as well as the upper and lower safety stock limits for crude oil in the crude oil pipeline storage, from the simulator, and constructs the inventory penalty function. When the current inventory of refined oil products in the refinery is greater than the upper safety stock limit, the sum of the difference between the current inventory and the upper safety stock limit is calculated. When the current inventory of refined oil products in the refinery is less than the lower safety stock limit, the sum of the difference between the lower safety stock limit and the current inventory is calculated, thus constituting the first inventory penalty function. When the current crude oil inventory at the refinery exceeds the upper limit of the refinery's safe inventory, the sum of the difference between the current inventory and the upper limit is calculated. When the current inventory is less than the lower limit of the refinery's safe inventory, the sum of the difference between the lower limit and the current inventory is calculated, thus forming the second inventory penalty function. Similarly, when the current crude oil inventory at the pipeline storage exceeds the upper limit of the pipeline storage's safe inventory, the sum of the difference is calculated. When the current inventory is less than the lower limit of the pipeline storage's safe inventory, the sum of the difference is calculated, thus forming the third inventory penalty function. Furthermore, the first, second, and third inventory penalty functions are defined as the target inventory penalty functions for each node of the oil product, thereby defining the preset oil product transportation cost function and the target inventory penalty function as the objective function.

[0222] Specifically, the objective function expression constructed using the multi-objective sequence construction method is shown below:

[0223]

[0224] Among them, F iF(x) represents the target inventory penalty function and the objective function, i = 1, 2, ..., K. K represents the number of optimization objectives in the multi-objective sequence. In this embodiment, K = 4, where when i = 1, F1(x) is the transportation cost corresponding to the preset oil transportation cost function; when i = 2, F2(x) is the third inventory penalty function; when i = 3, F3(x) is the second inventory penalty function; when i = 4, F4(x) is the first inventory penalty function; and X is the set of variables.

[0225] X can include variables such as transportation volume and inventory volume.

[0226] Furthermore, when optimizing the multi-objective function consisting of the first inventory penalty function, the second inventory penalty function, the third inventory penalty function, and the preset oil transportation cost function, the mathematical programming layer can first optimize the demand side, that is, optimize the refinery's refined oil inventory, then optimize the refinery's crude oil inventory, then optimize the crude oil pipeline inventory, and finally consider the oil transportation cost. That is, the optimization priority relationship is: refinery refined oil inventory > refinery crude oil inventory > pipeline crude oil inventory > oil transportation cost.

[0227] Optionally, when optimizing a multi-objective function, the optimization order can be set independently according to requirements, and this embodiment does not impose any restrictions.

[0228] The crude oil supply chain scheduling optimization method provided in this embodiment, if the target inventory penalty function in the target operations operator is determined to be constructed using a multi-objective sequence construction method, then the target inventory penalty function for each node in the crude oil supply chain is determined based on the current inventory level and the target operations operator. This includes: obtaining the upper and lower safety inventory limits for each node in the crude oil supply chain; each node in the crude oil supply chain includes refineries and crude oil pipeline storage facilities, and the oil products include crude oil and refined oil products; constructing a first inventory penalty function based on the current inventory level, the upper and lower safety inventory limits for refined oil products at refineries, constructing a second inventory penalty function based on the current inventory level, the upper and lower safety inventory limits for crude oil at refineries, and constructing a third inventory penalty function based on the current inventory level, the upper and lower safety inventory limits for crude oil at crude oil pipeline storage facilities; determining the first, second, and third inventory penalty functions as the target inventory penalty functions for each node's oil products; and determining the objective function of the mathematical programming model using a preset oil product transportation cost function and the target inventory penalty functions, including: determining the preset oil product transportation cost function and each target inventory penalty function as the objective function. Since the target inventory penalty function in the target operations operator is constructed using a multi-objective sequence construction method, it can simultaneously consider the inventory penalties of each node in the crude oil supply chain and various oil products. By constructing a corresponding inventory penalty function for each oil product node, the inventory level can be controlled more precisely, avoiding the risk of excessively high or low inventory. Furthermore, the preset oil transportation cost function is also used as the objective function, thus taking into account the transportation costs in the crude oil supply chain and achieving more comprehensive optimization.

[0229] As an optional implementation, based on the above embodiments, the expression of the first inventory penalty function is as follows:

[0230]

[0231] The expression for the second inventory penalty function is as follows:

[0232]

[0233] The expression for the third inventory penalty function is as follows:

[0234]

[0235] The expression for the preset oil transportation cost function is as follows:

[0236]

[0237] in, Sub represents the current inventory level of the corresponding oil product at the specified node. v,c This indicates the upper limit of safety stock, Slb. v,cThis indicates the lower limit of safety stock. c represents the pipeline transport volume of the e-th segment in the crude oil supply chain. e This represents the transportation cost of the e-th pipe segment.

[0238] Specifically, the first inventory penalty function is the inventory penalty function for refined product oil products, which uses the current inventory level of refined product oil products, the upper limit of the safe inventory level, and the lower limit of the safe inventory level. The second inventory penalty function is the inventory penalty function for crude oil products, which uses the current inventory level of crude oil products, the upper limit of the safe inventory level, and the lower limit of the safe inventory level. The third inventory penalty function is the inventory penalty function for crude oil storage facilities in pipelines, which uses the current inventory level of crude oil storage facilities in pipelines, the upper limit of the safe inventory level, and the lower limit of the safe inventory level.

[0239] As an optional implementation, based on the above embodiments, if a solver is used and the target inventory level of oil products at each node in the crude oil supply chain is determined to be unattainable based on the objective function, the method further includes:

[0240] Return information on actions that cannot be taken to the reinforcement learning layer, and output an approximately feasible current production scheduling plan;

[0241] A reinforcement learning layer is used to determine the current reward function based on the cost information and the information on actions that cannot be taken that day.

[0242] The "Inactive Action" information indicates whether to output the production scheduling plan for the day corresponding to achieving the target inventory level and minimizing the objective function. When the "Inactive Action" information is 1, it means that the target inventory level is unattainable; when it is 0, it means that the target inventory level is attainable.

[0243] Specifically, in this embodiment, if the mathematical programming layer determines that the target inventory level of oil products at each node in the crude oil supply chain is unattainable after solving the mathematical programming model using a solver, it sends an inability-to-act information message (indicating an inability to act) to the reinforcement learning layer. The inability-to-act information message is set to 1. The solver then uses relaxed preset constraints and the current inventory level and objective function to solve the problem, outputting an approximately feasible current production scheduling scheme. This scheme is then sent to the simulator, where the operational volume information from the approximately feasible current production scheduling scheme is used to calculate inventory changes, thereby obtaining the daily cost information, which is then sent to the reinforcement learning layer. The reinforcement learning layer adds the current transportation cost information, current inventory warning information, current inventory discard information, and inability-to-act information to form the current reward function.

[0244] The crude oil supply chain scheduling optimization method provided in this embodiment further includes the following steps if, when a solver is used and the target inventory level of oil products at each node in the crude oil supply chain is determined to be unattainable based on the objective function: returning unachievable action information to the reinforcement learning layer and outputting an approximately feasible current production scheduling scheme; and using the reinforcement learning layer to determine the current reward function based on the daily cost information and the unachievable action information. Since the mathematical programming layer returns unachievable action information to the reinforcement learning layer and generates an approximately feasible current production scheduling scheme when it determines that the target inventory level of oil products at each node in the crude oil supply chain is unattainable, the simulator can calculate the daily cost information corresponding to the approximately feasible current production scheduling scheme and send it to the reinforcement learning layer. This allows the reinforcement learning layer to determine the current reward function, enabling the reinforcement learning agent to perform analysis and learning, which helps in parameter optimization.

[0245] As an optional implementation, based on the above embodiments, the daily cost information includes daily transportation cost information, daily inventory warning information, and daily inventory disposal information;

[0246] The formula for calculating the reward function is as follows:

[0247]

[0248] Where, r t This represents the reward function for that day. This indicates the penalties corresponding to the daily inventory warning information. This indicates the loss items corresponding to the inventory disposal information for that day. This indicates the penalty items corresponding to the oil transportation cost information for that day. The penalty item corresponding to the information of inaction is indicated. k1, k2, and k3 represent the corresponding preset weight values. α takes the value 0 or 1. When the value is 1, it means that the target inventory level is unattainable. When the value is 0, it means that the target inventory level is attainable.

[0249] Specifically, in this embodiment, α represents the inaction information. When it is possible to generate a production scheduling plan for the day that minimizes the objective function when the oil products at each node in the crude oil supply chain reach the target inventory level, the mathematical programming layer feeds back the inaction information α = 0 to the reinforcement learning layer. If it is not possible to generate a production scheduling plan for the day that minimizes the objective function when the oil products at each node in the crude oil supply chain reach the target inventory level, the mathematical programming layer feeds back the penalty coefficient α = 1 to the reinforcement learning layer and returns an approximately feasible solution by relaxing the preset constraints.

[0250] Figure 5 This is a schematic diagram of the training process of a crude oil supply chain scheduling optimization system provided in another embodiment of this application, as shown below. Figure 5As shown. When it is necessary to predict the production scheduling plan of the crude oil supply chain within a preset time period, the reinforcement learning layer obtains the current state information of the crude oil supply chain from the simulator, determines the target operation operator based on the current state information, and sends the target operation operator to the operation operator layer. There are multiple operation operators. The operation operator layer determines the target inventory of oil products at each node in the crude oil supply chain based on the current supply and demand and the target operation operator, and constructs the target inventory penalty function according to the construction method of the target inventory penalty function. The mathematical programming layer obtains the current inventory of oil products at each node in the crude oil supply chain from the simulator, and determines the objective function of the mathematical programming model using the preset oil product transportation cost function and the target inventory penalty function. It then uses a solver to generate the daily production scheduling plan corresponding to minimizing the objective function when the oil products at each node in the crude oil supply chain reach the target inventory, with the goal of reducing the preset oil product transportation cost function value and the inventory penalty function value corresponding to the target inventory penalty function. The daily production scheduling plan is then sent to the simulator. The simulator executes the daily production scheduling plan and obtains the corresponding daily transportation cost information, daily inventory warning information, daily inventory discard information, and inactive action information. The reinforcement learning layer determines the daily reward function based on the day's transportation cost information, daily inventory warning information, daily inventory discard information, and information on actions that cannot be taken. By repeatedly executing the above steps, the daily reward function for each day within a preset time period can be obtained. The daily reward functions for each day within the preset time period are then accumulated. When the accumulated reward function reaches its maximum value, the mathematical programming layer outputs the daily production scheduling schemes sorted by time within the preset time period as the target production scheduling scheme.

[0251] In the crude oil supply chain scheduling optimization system, the operations research operator layer and mathematical programming layer employ pure mathematical optimization methods and require no training. The reinforcement learning layer, however, is a deep reinforcement learning framework composed of reinforcement learning and neural networks, containing a reinforcement learning agent. Only the agent within the reinforcement learning layer needs to be trained. When the mathematical programming layer sends inoperable action information to the reinforcement learning layer and generates an approximately feasible current production scheduling plan, a simulator is used to simulate this plan. The system sends the day's transportation cost information, day's inventory warning information, day's inventory discard information, and inoperable action information to the reinforcement learning layer. Upon receiving the day's cost information and inoperable action information, the reinforcement learning layer calculates the corresponding reward function and updates its parameters using the backpropagation algorithm by maximizing the accumulated reward function.

[0252] Figure 6 A schematic diagram of the structure of a crude oil supply chain scheduling optimization system provided in an embodiment of this application is shown below. Figure 6As shown, the crude oil supply chain scheduling optimization system provided in this embodiment is located in the crude oil supply chain scheduling optimization equipment. The crude oil supply chain scheduling optimization system 40 provided in this embodiment includes: a reinforcement learning layer 41, an operations research operator layer 42, a mathematical programming layer 43, and a simulator 44.

[0253] The system comprises the following layers: Reinforcement learning layer 41, which acquires the current state information of the crude oil supply chain and determines the target operations operator based on this information; Operations operator layer 42, which determines the target inventory level and target inventory penalty function for each node in the crude oil supply chain based on the current supply and demand and the target operations operator; Mathematical programming layer 43, which uses a preset oil transportation cost function and target inventory penalty function to determine the objective function of the mathematical programming model; Mathematical programming layer 43, which also uses a solver and the objective function to determine the daily production scheduling scheme corresponding to minimizing the objective function when the oil products at each node in the crude oil supply chain reach the target inventory level; Simulator 44, which executes the daily production scheduling scheme to obtain daily cost information; Reinforcement learning layer 41, which also determines the daily reward function based on the daily cost information; Reinforcement learning layer 41, which also acquires the daily reward function for each day within a preset time period and accumulates the daily reward functions within the preset time period to obtain the accumulated reward function; and Reinforcement learning layer 41, which adjusts the parameters using a backpropagation algorithm if the accumulated reward function value is not the maximum value, to obtain the adjusted reinforcement learning layer.

[0254] The crude oil supply chain scheduling and optimization system provided in this embodiment can execute... Figure 2 The implementation principles and technical effects of the methods shown are similar, and will not be repeated here.

[0255] Optionally, the crude oil supply chain scheduling optimization system includes an operations operation layer. This layer pre-stores multiple preset operations operation operators. Each preset operations operation operator includes multiple preset inventory adjustment strategies and a preset inventory penalty function construction method. The multiple preset inventory adjustment strategies include refinery refined product adjustment strategies, refinery crude oil adjustment strategies, and pipeline crude oil storage adjustment strategies. The refinery refined product adjustment strategy includes any one of the following: increasing refinery refined product inventory to a corresponding expected target, decreasing refinery refined product inventory to a corresponding expected target, or maintaining refinery refined product inventory. The refinery crude oil adjustment strategy includes any one of the following: increasing refinery crude oil inventory to a corresponding expected target or meeting crude oil demand within a preset time period. The pipeline crude oil storage adjustment strategy includes adjusting pipeline crude oil inventory to a preset safety line range. The preset inventory penalty function construction method includes: a piecewise linear construction method or a multi-objective sequence construction method.

[0256] Optionally, in mathematical programming layer 43, if the target inventory penalty function in the target operations operator is constructed using a piecewise linear construction method, then when determining the target inventory penalty function for each node in the crude oil supply chain based on the current inventory level and the target operations operator, it is specifically used for: obtaining the upper limit, lower limit, upper limit, and lower limit of the safety inventory for each node in the crude oil supply chain; constructing multiple inventory ranges based on the lower limit, lower limit, upper limit, and upper limit of the safety inventory, each inventory range having a corresponding preset inventory penalty function; and determining the target inventory penalty function from the preset inventory penalty functions based on the inventory range in which the current inventory level is located. In mathematical programming layer 43, when determining the objective function of the mathematical programming model using the preset oil transportation cost function and the target inventory penalty function, it is specifically used for: performing a weighted summation of the preset oil transportation cost function and the target inventory penalty function to obtain the objective function of the mathematical programming model.

[0257] Optionally, in mathematical programming layer 43, if the target inventory penalty function in the target operations operator is determined to be constructed using a multi-objective sequence construction method, then when determining the target inventory penalty function for each node in the crude oil supply chain based on the current inventory level and the target operations operator, it is specifically used to: obtain the upper and lower safety inventory limits for each node in the crude oil supply chain; each node in the crude oil supply chain includes: refineries and crude oil pipeline storage facilities, and the oil products include crude oil and refined oil products; construct a first inventory penalty function based on the current inventory level, the upper and lower safety inventory limits for refined oil products in refineries, construct a second inventory penalty function based on the current inventory level, the upper and lower safety inventory limits for crude oil in refineries, and construct a third inventory penalty function based on the current inventory level, the upper and lower safety inventory limits for crude oil in crude oil pipeline storage facilities; and determine the first, second, and third inventory penalty functions as the target inventory penalty functions for each node's oil products. Mathematical programming layer 43, when determining the objective function of the mathematical programming model using the preset oil transportation cost function and the target inventory penalty function, is specifically used to: determine the preset oil transportation cost function and each target inventory penalty function as the objective function.

[0258] Optionally, mathematical programming layer 43, if a solver is used and the target inventory level of oil products at each node in the crude oil supply chain is determined to be unattainable based on the objective function, is specifically used to: return information on unachievable actions to the reinforcement learning layer and output an approximately feasible current production scheduling scheme. Reinforcement learning layer 41 is also used to: determine the current reward function based on the daily cost information and information on unachievable actions.

[0259] Figure 7 A schematic diagram of the structure of a crude oil supply chain scheduling optimization device provided in an embodiment of this application is shown below. Figure 7As shown, the crude oil supply chain scheduling optimization device 50 provided in this embodiment includes: a processor 51, a memory 52 communicatively connected to the processor, and a transceiver 53.

[0260] The system includes a memory 52 that stores computer-executed instructions; a transceiver 53 for receiving and sending instructions; and a processor 51 that executes the computer-executed instructions stored in the memory 52 to implement the crude oil supply chain scheduling optimization method provided in any of the above embodiments. Related explanations can be understood by referring to the descriptions and effects corresponding to the steps in the accompanying drawings, and will not be elaborated upon here.

[0261] The program may include program code, which includes computer-executable instructions. Memory 52 may include high-speed RAM, and may also include non-volatile memory, such as at least one disk storage device.

[0262] In this embodiment, the memory 52, processor 51, and transceiver 53 are connected via a bus. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. The bus can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 7 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0263] This application also provides a computer-readable storage medium storing computer-executable instructions. When executed by a processor, these instructions are used to implement the crude oil supply chain scheduling optimization method provided in any of the above embodiments. For example, the computer-readable storage medium may be a ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, or optical data storage device.

[0264] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the crude oil supply chain scheduling optimization method provided in any of the above embodiments.

[0265] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are all optional embodiments, and the actions and modules involved are not necessarily essential to this application.

[0266] It should be further noted that although the steps in the flowchart are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowchart may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the sub-steps or stages of other steps.

[0267] It should be understood that the above-described device embodiments are merely illustrative, and the device of this application can also be implemented in other ways. For example, the division of units / modules in the above embodiments is only a logical functional division, and there may be other division methods in actual implementation. For example, multiple units, modules, or components may be combined, or integrated into another system, or some features may be ignored or not executed.

[0268] Furthermore, unless otherwise specified, the functional units / modules in the various embodiments of this application can be integrated into one unit / module, or each unit / module can exist physically separately, or two or more units / modules can be integrated together. The integrated units / modules described above can be implemented in hardware or in the form of software program modules.

[0269] When integrated units / modules are implemented in hardware, the hardware can be digital circuits, analog circuits, etc. The physical implementation of the hardware structure includes, but is not limited to, transistors, memristors, etc. Unless otherwise specified, the processor can be any suitable hardware processor, such as a CPU, GPU, FPGA, DSP, and ASIC, etc. Unless otherwise specified, the storage unit can be any suitable magnetic or magneto-optical storage medium, such as Resistive Random Access Memory (RRAM), Dynamic Random Access Memory (DRAM), Static Random Access Memory (SRAM), Enhanced Dynamic Random Access Memory (EDRAM), High-Bandwidth Memory (HBM), Hybrid Memory Cube (HMC), etc.

[0270] If the integrated unit / module is implemented as a software program module and sold or used as an independent product, it can be stored in a computer-readable storage device (CMD). Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a memory and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned memory includes various media capable of storing program code, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard drive, magnetic disk, or optical disk.

[0271] In the above embodiments, the descriptions of each embodiment have their own emphasis. Parts not described in detail in a particular embodiment can be referred to in the relevant descriptions of other embodiments. The technical features of the above embodiments can be combined arbitrarily. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as these combinations of technical features do not contradict each other, they should be considered within the scope of this specification. Those skilled in the art, upon considering the specification and practicing the invention disclosed herein, will readily conceive of other embodiments of this application. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary technical means in the art not disclosed in this application. The specification and embodiments are considered exemplary only, and the true scope and spirit of this application are indicated by the following claims.

[0272] It should be understood that this application is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this application is limited only by the appended claims.

Claims

1. A crude oil supply chain scheduling optimization method, characterized in that, The method is applied to a crude oil supply chain scheduling optimization system, and the method includes: The following operations are performed repeatedly until the accumulated reward function value reaches its maximum value, and the production scheduling schemes for each day, sorted by time within a preset time period, are output as the target production scheduling scheme. The following operations include: The current state information of the crude oil supply chain is obtained, and a reinforcement learning layer is used to determine the target operation operator based on the current state information. The target operation operator includes the target inventory adjustment strategy and the target inventory penalty function construction method. The current state information of the crude oil supply chain includes the current inventory of oil products at each node in the crude oil supply chain. Based on the current inventory level and the target operations operator, the target inventory level and target inventory penalty function of oil products at each node in the crude oil supply chain are determined. The objective function of the mathematical programming model is determined by using a preset oil transportation cost function and the target inventory penalty function; A solver is used to determine the daily production scheduling scheme that minimizes the objective function when the oil products at each node in the crude oil supply chain reach the target inventory level, based on the current inventory level and the objective function. The production scheduling plan for the day is executed using a simulator to obtain the cost information for the day. The reinforcement learning layer is employed, and the daily reward function is determined based on the daily cost information; Obtain the daily reward function corresponding to each day within a preset time period, and use the reinforcement learning layer to accumulate and calculate the daily reward functions within the preset time period to obtain the accumulated reward function; If the accumulated reward function value is not the maximum value, the reinforcement learning layer uses the backpropagation algorithm to adjust the parameters to obtain the adjusted reinforcement learning layer.

2. The method according to claim 1, characterized in that, The crude oil supply chain scheduling optimization system includes an operations management operator layer, which pre-stores multiple preset operations management operators. Each preset operations management operator includes multiple preset inventory adjustment strategies and a preset inventory penalty function construction method. The multiple preset inventory adjustment strategies include refinery refined product adjustment strategy, refinery crude oil adjustment strategy, and pipeline crude oil storage adjustment strategy; The refinery refined product adjustment strategy includes any one of the following: increasing refinery refined product inventory to the corresponding expected target, decreasing refinery refined product inventory to the corresponding expected target, or maintaining refinery refined product inventory. The refinery crude oil adjustment strategy includes any one of the following: increasing refinery crude oil inventory to the corresponding expected target, or meeting crude oil demand within a preset time period. The pipeline crude oil storage adjustment strategy includes adjusting the pipeline crude oil inventory to a preset safety line range; The preset inventory penalty function can be constructed in the following ways: piecewise linear construction or multi-objective sequence construction.

3. The method according to claim 1, characterized in that, If the target inventory penalty function in the target operations operator is determined to be constructed using a piecewise linear construction method, then the target inventory penalty function for each node in the crude oil supply chain is determined based on the current inventory level and the target operations operator, including: Obtain the upper limit, lower limit, upper limit, and lower limit of safe inventory for oil products at each node in the crude oil supply chain; Based on the designed lower limit of inventory, the safety stock lower limit, the safety stock upper limit, and the designed upper limit of inventory, multiple inventory ranges are constructed, and each of the inventory ranges has a corresponding preset inventory penalty function; The target inventory penalty function is determined from the preset inventory penalty function based on the inventory range in which the current inventory level is located; The objective function of the mathematical programming model is determined by using a preset oil transportation cost function and the target inventory penalty function, including: The preset oil transportation cost function and the target inventory penalty function are weighted and summed to obtain the objective function of the mathematical programming model.

4. The method according to claim 3, characterized in that, The expression for the preset inventory penalty function corresponding to each of the inventory ranges is as follows: in, This represents the target inventory penalty function. This indicates the current inventory level of the corresponding oil product at the specified node, in Ub. v,c Indicates the upper limit of the designed inventory, Sub v,c This indicates the upper limit of safety stock, Slb. v,c Lb represents the lower limit of safety stock. v,c β1 and β2 represent the first and second penalty coefficients, respectively; v represents a node, including refinery nodes and pipeline crude oil storage nodes; t represents the current day t; and c represents oil products, including crude oil and refined oil products. The objective function of the mathematical programming model is expressed as follows: in, c represents the pipeline transport volume of the e-th segment in the crude oil supply chain. e This represents the transportation cost of the e-th pipe segment, and β represents the preset equilibrium coefficient. This is a preset oil transportation cost function.

5. The method according to claim 1, characterized in that, If the target inventory penalty function in the target operations operator is determined to be constructed using a multi-objective sequence construction method, then the target inventory penalty function for each node in the crude oil supply chain is determined based on the current inventory level and the target operations operator, including: Obtain the upper and lower limits of safe inventory for oil products at each node in the crude oil supply chain; the nodes in the crude oil supply chain include: refineries and crude oil pipeline storage facilities, and the oil products include crude oil and refined oil products; A first inventory penalty function is constructed based on the current inventory level, the upper limit and lower limit of the safe inventory of refined oil products in the refinery, a second inventory penalty function is constructed based on the current inventory level, the upper limit and lower limit of the safe inventory of crude oil in the refinery, and a third inventory penalty function is constructed based on the current inventory level, the upper limit and lower limit of the safe inventory of crude oil in the crude oil pipeline storage. The first inventory penalty function, the second inventory penalty function, and the third inventory penalty function are determined as the target inventory penalty functions for oil products at each node; The objective function of the mathematical programming model is determined by using a preset oil transportation cost function and the target inventory penalty function, including: The preset oil transportation cost function and each of the target inventory penalty functions are determined as the objective functions.

6. The method according to claim 5, characterized in that, The expression for the first inventory penalty function is as follows: The expression for the second inventory penalty function is as follows: The expression for the third inventory penalty function is as follows: The expression for the oil transportation cost function is as follows: in, Sub represents the current inventory level of the corresponding oil product at the specified node. v,c This indicates the upper limit of safety stock, Slb. v,c This indicates the lower limit of safety stock. c represents the pipeline transport volume of the e-th segment in the crude oil supply chain. e This represents the transportation cost of the e-th pipe segment.

7. The method according to any one of claims 1-6, characterized in that, If a solver is used and the target inventory levels of oil products at each node in the crude oil supply chain are determined to be unattainable based on the objective function, the method further includes: Return infeasible action information to the reinforcement learning layer and output an approximately feasible current production scheduling scheme; The reinforcement learning layer is used, and the current reward function is determined based on the daily cost information and the information on actions that cannot be taken.

8. The method according to claim 7, characterized in that, The daily cost information includes daily transportation cost information, daily inventory warning information, and daily inventory disposal information; The formula for calculating the reward function is as follows: Where, r t This represents the reward function for that day. This indicates the penalties corresponding to the daily inventory warning information. This indicates the loss items corresponding to the inventory disposal information for that day. This indicates the penalty items corresponding to the oil transportation cost information for that day. The penalty item corresponding to the information of inaction is indicated. k1, k2, and k3 represent the corresponding preset weight values. α takes the value 0 or 1. When the value is 1, it means that the target inventory level is unattainable. When the value is 0, it means that the target inventory level is attainable.

9. A crude oil supply chain scheduling optimization system, characterized in that, include: A reinforcement learning layer is used to acquire the current state information of the crude oil supply chain and determine the target operations operator based on the current state information; The operations operator layer is used to determine the target inventory level and target inventory penalty function of oil products at each node in the crude oil supply chain based on the current supply and demand and the target operations operator. The mathematical programming layer is used to determine the objective function of the mathematical programming model using a preset oil transportation cost function and the target inventory penalty function; The mathematical programming layer is also used to use a solver and based on the objective function to determine the daily production scheduling scheme corresponding to minimizing the objective function when the oil products at each node in the crude oil supply chain reach the target inventory level; The simulator is used to execute the production scheduling plan for the day in order to obtain the cost information for the day; The reinforcement learning layer is also used to determine the daily reward function based on the daily cost information; The reinforcement learning layer is also used to obtain the daily reward function corresponding to each day within a preset time period, and to accumulate the daily reward functions within the preset time period to obtain the accumulated reward function. The reinforcement learning layer is also used to adjust the parameters using a backpropagation algorithm if the accumulated reward function value is not the maximum value, so as to obtain an adjusted reinforcement learning layer.

10. A crude oil supply chain scheduling optimization device, characterized in that, include: A processor, a memory communicatively connected to the processor, and a transceiver; The memory stores computer-executed instructions; the transceiver is used to receive and send instructions. The processor executes computer execution instructions stored in the memory to implement the method as described in any one of claims 1 to 8.

11. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the method as described in any one of claims 1 to 8.

12. A computer program product, characterized in that, Includes a computer program that, when executed by a processor, implements the method described in any one of claims 1 to 8.

Images