Molten iron transportation scheduling method, system, equipment and medium

Abstract

The invention provides a molten iron transportation scheduling method, system and device and a medium, and the method comprises the steps: obtaining molten iron transportation data, building a molten iron transportation road network topology according to the molten iron transportation data, the molten iron transportation road network topology comprises a plurality of nodes, and each node comprises a node state and an available action set; converting the node state of each node into a state feature vector by using a GCN network; constructing a scheduling decision function, and then solving the scheduling decision function by using a PPO algorithm based on the state feature vector and the available action feature set to obtain a decision vector of each node; and converting the decision vector into a scheduling scheme, and scheduling each node according to the scheduling scheme. The problems that in the prior art, a dispatcher dispatches molten iron transportation by means of personal experience and mastering of complex dispatching information and adopting a dispatching mode of watching while walking, molten iron dispatching operation conflicts are likely to be caused, and efficiency is low are solved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of hot metal transportation, and in particular to a hot metal transportation scheduling method, system, device and medium. BACKGROUND

[0002] The steel industry is the foundation of all industries and the cornerstone of national economic development, and is also a representative of the industrialization of a country. In the process of steel production, the transportation process of hot metal occupies a pivotal position, and the production capacity of ironmaking blast furnace and steelmaking converter is limited by the smoothness of the hot metal transportation process, which requires the use of a reasonable hot metal transportation organization method to maximize the production capacity of each device as much as possible. Therefore, based on the hot metal transportation shunting operation plan, all hot metal transportation shunting operations within the stage are considered, an optimized scheduling algorithm is used in combination with computer technology to develop a route scheduling plan for the shunting operation process, potential conflicts and interference between operations are resolved, intelligentization of hot metal transportation scheduling is achieved, and green low-carbonization and deep intelligentization of steel production are achieved, which has important academic significance and application value.

[0003] However, in the traditional hot metal scheduling process, the dispatcher relies on personal experience and mastery of complex scheduling information to adopt a walk-and-look scheduling mode, which not only wastes time and effort and is inefficient, but also increases the work burden of the dispatcher, and due to the large number of scheduling operations and lack of efficiency, it is difficult to consider all scheduling operations within the stage. In addition, due to the uneven level of the dispatcher himself, there is a lack of predictability for the cross interference between potential hot metal scheduling operations, so it is easy to conflict with each other, affecting the efficiency and quality of the hot metal scheduling operation, and further affecting the efficiency of the whole production task of the iron area, causing a great impact on the stable and orderly production of the steel enterprise. SUMMARY

[0004] In view of the deficiencies in the prior art, the present application provides a hot metal transportation scheduling method, system, device and medium, which solves the problem that the dispatcher relies on personal experience and mastery of complex scheduling information to adopt a walk-and-look scheduling mode to schedule hot metal transportation, which easily leads to conflicts in hot metal scheduling operations and low efficiency.

[0005] According to an embodiment of the present application, a hot metal transportation scheduling method comprises: Obtaining hot metal transportation data and constructing a hot metal transportation road network topology according to the hot metal transportation data, the hot metal transportation road network topology comprising a plurality of nodes, each node comprising a node state and a set of available actions; Using a GCN network to convert the node state of each node into a state feature vector; A scheduling decision function is constructed, and then the scheduling decision function is solved using the PPO algorithm based on the state feature vector and the available action feature set to obtain the decision vector of each node; The decision vector is converted into a scheduling scheme, and each node is scheduled according to the scheduling scheme.

[0006] Preferably, the nodes include blast furnace nodes, locomotive nodes, tank car nodes, and converter nodes; The node state of the blast furnace node includes tapping time, tapping amount, and tapping temperature, the node state of the converter node includes steelmaking time and required iron amount, and the blast furnace node and the converter node have no available action set; The node state of the locomotive node includes current position, speed, and running state, and the node state of the tank car node includes current position, loading state, and target converter.

[0007] Preferably, the method for converting the node state of each node into a state feature vector using a GCN network includes: An adjacency matrix and a degree matrix are constructed according to the topology of the molten iron transportation network; Based on the adjacency matrix and the degree matrix, a 2-layer graph convolution layer is used to convert the node state of each node into semantic features; The semantic features are normalized to obtain the state feature vector.

[0008] Preferably, the calculation formula of each graph convolution layer is: wherein, is the semantic feature of the node of the lth layer, is the weight matrix, is the adjacency matrix, is the degree matrix, is the activation function (ReLU).

[0009] Preferably, the scheduling decision function is as follows: wherein, is the action taken by node i at time t, is the state feature vector of the node at time t, is the policy parameter, is the available action set of node i.

[0010] Preferably, the method for solving the scheduling decision function using the PPO algorithm to obtain the decision vector of each node includes: S1: Initialize the policy parameter, then calculate the policy trajectory using the scheduling policy function, and calculate the approximate policy gradient of the scheduling policy function at this time; S2: adjust the policy parameters according to the approximate policy gradient, and then repeat steps S1-S2 using the new policy parameters until the policy parameters no longer change; S3: substitute the latest policy parameters into the scheduling decision function, calculate the action probability of each policy trajectory, and take the policy trajectory with the maximum action probability as the decision vector of the node.

[0011] Preferably, the policy parameters are initialized using random parameters.

[0012] In another aspect, according to an embodiment of the present application, there is also provided a molten iron transportation scheduling system, which uses the above-mentioned molten iron transportation scheduling method, comprising: a data acquisition module, configured to acquire molten iron transportation data; a topology module, configured to construct a molten iron transportation road network topology according to the molten iron transportation data; a conversion module, configured to convert the node state of each node in the molten iron transportation road network topology into a state feature vector using a GNN network; a decision module, configured to construct a scheduling decision function, and solve the scheduling decision function using a PPO algorithm to obtain a decision vector of each node; a control module, configured to convert the decision vector into a scheduling scheme, and schedule each node according to the scheduling scheme.

[0013] In another aspect, according to an embodiment of the present application, there is also provided a computer comprising at least one processor and a memory, wherein the memory stores a computer program configured to be executed by the processor to implement the above-mentioned bridge health monitoring data recovery method.

[0014] In another aspect, according to an embodiment of the present application, there is also provided a storage medium, which is a computer-readable storage medium, and has stored thereon a computer program executable by one or more processors to implement the above-mentioned bridge health monitoring data recovery method.

[0015] Compared with the prior art, the present application has the following beneficial effects: The present application avoids the scheduling operation conflicts caused by manual scheduling and improves the efficiency of the scheduling operation by constructing the molten iron transportation network topology in the form of a topology graph from the data of the entire molten iron transportation, converting the node state of each node in the molten iron transportation network topology into a state feature vector convenient for calculation and processing, simplifying all scheduling information, quantifying the entire molten iron transportation process, using a scheduling decision function and using a PPO algorithm to solve, calculating the optimal decision vector, and then converting the decision vector into a scheduling scheme and scheduling each node according to the scheduling scheme. BRIEF DESCRIPTION OF DRAWINGS

[0016] Figure 1 The flow chart of the molten iron transportation scheduling method of the embodiment of the present application.

[0017] Figure 2 The molten iron transportation network topology of the embodiment of the present application. DETAILED DESCRIPTION

[0018] The technical solutions in the present application will be further described below with reference to the drawings and embodiments.

[0019] As shown in Figure 1 , the embodiment of the present application proposes a molten iron transportation scheduling method, which comprises: obtaining molten iron transportation data and constructing a molten iron transportation network topology according to the molten iron transportation data, wherein the molten iron transportation network topology comprises a plurality of nodes, and each node comprises a node state and an available action set; After obtaining the molten iron transportation data, the present application constructs the molten iron transportation network topology as shown in Figure 2 , which comprises two blast furnaces (BF1 and BF2), one steelmaking plant (SP1, three converters 1, 2 and 3), four locomotives (L1-L4) and six tank cars (T1-T6). The two blast furnaces supply molten iron to the steelmaking plant together, the locomotives pull the tank cars to the destination, N101-N111 are the locomotive stop nodes, N201-N208 are the tank car stop nodes, and the edge connection relationship is that N101 is connected with N103, the length is 50m, and the traffic state is normal.

[0020] At this time, the node state is as follows: (1) Collect the tapping data of the blast furnace BF1 at t=0: {time: 0, volume: 150t, temperature: 1500℃}, and the tapping data of BF2 at t=10min: {time: 10, volume: 150t, temperature: 1480℃}.

[0021] (2) Locomotive state: L1 is located at node N101, speed 0, state idle; L2 is located at N102, speed 0, state idle, and other locomotive states are the same.

[0022] (3) Tank car initial state: T1 is at N201, empty tank, no target spot; T2 is at N202, full tank, target spot SP1. Other tank car states are the same.

[0023] (4) Steelmaking demand: SP1: one converter needs one 150t tank every 38 minutes, so three tanks are needed, a total of 450t.

[0024] The road network topology is converted into a graph structure G=(V,E), wherein the node V represents the position of the spot, turnout and the like, the edge E represents the connection path, and each edge is additionally provided with the length, passing state and the like. At this time, the node state expression of the locomotive is [position_x, position_y, speed, status_onehot], and the node state expression of the tank car is [position_x, position_y, load_status_onehot, target_spot].

[0025] Wherein status_onehot represents the encoding (One-Hot Encoding) vector of the current running state of the locomotive, which is divided into two categories of "idle" and "in transit", the idle state: [1, 0], in transit: [0, 1]; load_status_onehot represents the encoding vector of the loading state of the tank car, which is divided into two categories of "empty" and "full", empty tank: [1, 0], full tank: [0, 1]; target_spot represents the unique identification number of the target spot to which the tank car needs to go, for example, target_spot=2 indicates that the tank car needs to go to the No. 2 converter spot.

[0026] In addition, the blast furnace node and the converter node continuously perform only the same action, so the present application defines them as having no available action set, and the locomotive node and the tank car node take the driving target position as the action space.

[0027] The node state of each node is converted into a state feature vector using a GCN network; Since a lot of states are additionally provided on each node (node) and edge (edge), such as the locomotive state, the tank car state, the target spot, the driving distance and the like, the road network state is a structured data with a high-dimensional vector. In order to enhance the efficiency of the learning model, the present application intends to adopt a graph convolutional neural network (GCN) to express the vectorized features of the molten iron transportation road network environment state.

[0028] A two-layer graph convolution layer (GCN) is adopted, and the calculation formula of each layer is: wherein, is the semantic feature of the node of the lth layer, is the weight matrix, is the adjacency matrix with self-loop, is the degree matrix; is the activation function (ReLU).

[0029] The input layer node feature is the preprocessed entity feature, neighbor node information is aggregated through graph convolution operation to generate semantic features, and then the semantic features are normalized to obtain the corresponding, for example: Locomotive L1 node state: [x = 10, y = 20, speed = 0, status = [1, 0]] (idle state).

[0030] Tank car T2 node state: [x = 30, y = 40, load_status = [0, 1], target_spot = SP1_id] (full tank, 1 seat steel plant).

[0031] The state feature vector h_L1 of the locomotive L1 obtained is [0.8, 0.6, 0.3, 0.9], and the state feature vector h_T2 of the tank car T2 is [0.2, 0.7, 0.1, 0.5].

[0032] h_L1 = [0.8, 0.6, 0.3, 0.9] represents the mapped result, 0.8, 0.6 is the mapped result of the original coordinates x = 10, y = 20, 0.3 represents that there is no task at present and the priority is low, and 0.9 represents a 0% probability of assigning a task.

[0033] Global state vector: S = [h_L1, h_L2, h_L3, h_L4, h_T1, h_T2...h_T6, h_N101, h_N102...h_N111, h_SP1].

[0034] The present application constructs the molten iron transportation data into a molten iron transportation network topology in the form of a topological graph, then converts the node state of each node in the molten iron transportation network topology into a state feature vector convenient for calculation and processing, simplifies all scheduling information, and quantizes the entire molten iron transportation process.

[0035] A scheduling decision function is constructed, and then based on the state feature vector and the available action feature set, the PPO algorithm is used to solve the scheduling decision function to obtain a decision vector of each node; The molten iron transportation scheduling can be represented as a four-tuple (S, A, Pa, Ra), wherein: ①S represents a state space set, st represents a state at time t, and the state feature vector learned by the GCN is used as a representation of the molten iron transportation environment state.

[0036] ②A represents an action space set, and at represents an action to be taken at time t. The present application considers two types of action representing molten iron transportation scheduling behavior, i.e., tank car distribution rules (such as earliest idle priority, shortest distance priority, and usage balance) and locomotive obstacle avoidance rules (shortest path priority, conflict minimum priority, and parking waiting).

[0037] ③Pa represents the probability of migrating from state st to st+1 under the action of a, which is determined by historical data statistics and environmental dynamic simulation.

[0038] ④Ra represents the reward obtained by migrating from state st to st+1.

[0039] For the tank car matching node: if the tank car is successfully matched, a reward of +5 is obtained; otherwise, -5 is obtained. For the locomotive running node: if the shortest path is selected, a reward of +10 is obtained; if a conflict-free path is obtained, +0 is obtained; if there is a path conflict, -20 is obtained.

[0040] According to the output state feature vector (st) of the GCN, the present application defines the following scheduling strategy function: wherein, is the action taken by node i at time t, is the state feature vector of the node at time t, is the policy parameter, is the available action set of node i.

[0041] In order to obtain a satisfactory scheduling strategy, the present application uses a RL scheduling framework based on Proximal Policy Optimization (PPO) to solve the scheduling strategy function, and the basic process is as follows: S1: use random parameters to initialize the policy parameters , the number of iterations k=0, then use the scheduling strategy function to guide the interaction between the agent and the environment, obtain the policy trajectory , and calculate the approximate policy gradient of the scheduling strategy function at this time , and calculate the value of each action in the policy . S2: adjust the policy parameters according to the approximate policy gradient , the objective function for measuring the pros and cons of the policy in the kth iteration, is with the policy parameters The "slope" of the change, find the maximum Only need to update the parameters in the direction of the gradient, that is, the state, action, reward and other formulas in each trajectory are substituted into the formula to calculate the gradient value, and then guide Adjust in that direction Then repeat steps S1-S2 using the new policy parameters until the policy parameters no longer change, at which point the maximum policy parameters are obtained ; S3: Substitute the latest policy parameters into the scheduling decision function to calculate the action probability of each policy trajectory, and take the policy trajectory with the maximum action probability as the decision vector of the node.

[0042] Taking locomotive node L1 as an example, if the policy function calculates the probabilities of each action: assign_tank(L1, T2, SP1) is 0.7, and plan_route(L1, [N101→N103→SP1]) is 0.6.

[0043] assign_tank(L1, T2, SP1) represents the probability of assigning the locomotive numbered L1 to the tank car numbered T2 to go to the SP1 work station is 70%.

[0044] plan_route(L1, [N101→N103→SP1]) represents planning a driving path for the locomotive L1 to start from node N101, pass through N103, and arrive at the SP1 work station.

[0045] Select the action combination with the highest probability: assign L1 to pull T2 to SP1, and the path planning is N101→N103→SP1, with a total reward of +30 points After that, the decision vector (L1, [N101→N103→SP1]) is converted into a scheduling scheme, and each node can be scheduled according to the scheduling scheme.

[0046] Using the scheduling decision function and using the PPO algorithm to solve, the optimal decision vector is calculated, and then the decision vector is converted into a scheduling scheme, and each node is scheduled according to the scheduling scheme, so as to avoid the conflict between scheduling operations caused by manual scheduling and improve the efficiency of scheduling operations.

[0047] On the other hand, the embodiment of the present application also provides a molten iron transportation scheduling system, which uses the above-mentioned molten iron transportation scheduling method, comprising: A data acquisition module, the data acquisition module is used for acquiring molten iron transportation data; A topology module, the topology module is used for constructing a molten iron transportation road network topology according to the molten iron transportation data; A conversion module is configured to convert the node state of each node in the molten iron transportation network topology into a state feature vector using a GNN network; A decision module is configured to construct a scheduling decision function and solve the scheduling decision function using a PPO algorithm to obtain a decision vector of each node. A control module is configured to convert the decision vector into a scheduling scheme and schedule each node according to the scheduling scheme.

[0048] In another aspect, the embodiments of the present application further provide a computer, comprising at least one processor and a memory, wherein the memory stores a computer program configured to be executed by the processor to implement the bridge health monitoring data recovery method.

[0049] In another aspect, the embodiments of the present application further provide a storage medium, which is a computer readable storage medium, and stores a computer program, wherein the computer program can be executed by one or more processors to implement the bridge health monitoring data recovery method.

[0050] Finally, it should be pointed out that the above embodiments are only used to illustrate the technical solutions of the present application but not limit the present application. Although the present application has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present application can be modified or replaced equivalently without departing from the spirit and scope of the present application, and all of them should be covered in the scope of the claims of the present application.

Claims

1. A method for dispatching molten iron transportation, characterized by: include: Obtaining molten iron transportation data, and constructing a molten iron transportation network topology based on the molten iron transportation data, wherein the molten iron transportation network topology includes a plurality of nodes, each node including a node state and an available action set; Use the GCN network to convert the node state of each node into a state feature vector; Construct a scheduling decision function, and then use the PPO algorithm to solve the scheduling decision function based on the state feature vector and the available action feature set to obtain the decision vector of each node; The decision vector is converted into a scheduling plan, and each node is scheduled according to the scheduling plan.

2. A molten iron transportation scheduling method according to claim 1, characterized in that: The nodes include blast furnace nodes, locomotive nodes, tank car nodes and converter nodes; The node status of the blast furnace node includes the tapping time, tapping amount, and tapping temperature. The node status of the converter node includes the steelmaking time and required iron amount. There are no available action sets for the blast furnace node and the converter node. The node status of the locomotive node includes the current position, speed and running status, and the node status of the tank car node includes the current position, loading status and target converter.

3. A molten iron transportation scheduling method according to claim 1, characterized in that: Methods for using the GCN network to convert the node state of each node into a state feature vector include: Construct the adjacency matrix and degree matrix based on the topology of the molten iron transportation network; Based on the adjacency matrix and degree matrix, a two-layer graph convolution layer is used to convert the node state of each node into semantic features; Normalize the semantic features to obtain the state feature vector.

4. A molten iron transportation scheduling method according to claim 3, characterized in that: The calculation formula of each graph convolution layer is: in, is the semantic feature of the node in the lth layer, is the weight matrix, is the adjacency matrix, is the degree matrix, is the activation function (ReLU).

5. The method for molten iron transportation scheduling according to claim 1, wherein: The scheduling decision function is as follows: in, is the action taken by node i at time t, is the state feature vector of the node at time t, is the strategy parameter, is the set of available actions for node i.

6. The method for molten iron transportation scheduling according to claim 1, wherein: The method of using the PPO algorithm to solve the scheduling decision function and obtain the decision vector of each node includes: S1: Initialize the policy parameters, then use the scheduling policy function to calculate the policy trajectory and calculate the approximate policy gradient of the scheduling policy function at this time; S2: Adjust the policy parameters according to the approximate policy gradient, and then repeat steps S1-S2 with the new policy parameters until the policy parameters no longer change; S3: Substitute the latest policy parameters into the scheduling decision function, calculate the action probability of each policy trajectory, and use the policy trajectory with the largest action probability as the decision vector of the node.

7. A molten iron transportation scheduling method according to claim 6, characterized in that: include: Use random parameters to initialize the strategy parameters.

8. A molten iron transportation scheduling system, characterized in that: The system uses a molten iron transportation scheduling method according to any one of claims 1 to 7, comprising: A data acquisition module, wherein the data acquisition module is used to collect molten iron transportation data; A topology module, configured to construct a molten iron transportation network topology based on molten iron transportation data; A conversion module, the conversion module is used to convert the node state of each node in the molten iron transportation network topology into a state feature vector using a GNN network; A decision module is used to construct a scheduling decision function and solve the scheduling decision function using a PPO algorithm to obtain a decision vector for each node; The control module is used to convert the decision vector into a scheduling plan and schedule each node according to the scheduling plan.

9. A computer, characterized in that: The system comprises at least one processor and a memory, wherein the memory stores a computer program, and the computer program is configured to be executed by the processor to implement the bridge health monitoring data recovery method according to any one of claims 1 to 7.

10. A storage medium, characterized in that: The storage medium is a computer-readable storage medium, on which a computer program is stored. The computer program can be executed by one or more processors to implement the bridge health monitoring data recovery method according to any one of claims 1 to 7.

Images