Metallurgy reservoir area unmanned crown block intelligent scheduling method

By combining global and local optimization models with solvers and the DQN algorithm to optimize crane scheduling, the problem of low efficiency in the collaborative work of multiple cranes in the metallurgical warehouse area was solved, achieving efficient crane utilization and task processing, and adapting to dynamic disturbances.

CN118966708AActive Publication Date: 2024-11-15ANHUI MA STEEL AUTOMATION INFORMATION TECH +1

Patent Information

Application Number
CN202411144072.3
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-20
Publication Date
2024-11-15
Estimated Expiration
2044-08-20

AI Technical Summary

Technical Problem

Existing technologies have low efficiency when multiple overhead cranes work in parallel in metallurgical storage areas, making it difficult to cope with equipment malfunctions and order changes, resulting in insufficient crane utilization and task processing capacity.

Method used

A combination of global and local optimization models is adopted, and the Gurobi, CPLEX, SCIP, and CBC solvers are used to optimize crane scheduling. The DQN algorithm is combined with a convolutional neural network to fit the Q value. An empirical replay strategy is used to train the neural network, select the optimal crane allocation rule, and establish crane-to-crane avoidance rules to achieve rapid response under dynamic disturbances.

Benefits of technology

The optimal solution for crane scheduling was achieved within seconds, improving crane utilization and task processing capacity, and significantly enhancing scheduling efficiency under dynamic disturbances.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118966708A_ABST
    Figure CN118966708A_ABST
Patent Text Reader

Abstract

The invention discloses an intelligent scheduling method for an unmanned crown block in a metallurgical reservoir area, relates to the technical field of unmanned crown block scheduling, establishes a multi-task multi-crown block cluster global optimization scheduling model, provides an efficient parallel solving method, and establishes a crown block scheduling local optimization model for equipment abnormity and order change disturbance conditions. A solution method based on a scheduling rule is provided, the crown block global optimization model and the crown block local optimization model can achieve optimal solution on the second level, the solution efficiency of a Gurobi tool box and the solution efficiency of a CPLEX tool box are the highest, and in addition, compared with a single scheduling rule algorithm, the decision tree algorithm has the advantages that the crown block utilization rate and the task handling capacity are remarkably improved, and the decision tree algorithm is suitable for large-scale popularization and application. The DQN algorithm proposed for dynamic disturbance can determine to select a certain optimal crown block in a certain state, the crown block utilization rate and the task handling capacity are better than those of a decision tree and a single scheduling rule, and the problem that in the prior art, the working efficiency is low when cooperative work is carried out is solved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of unmanned overhead crane intelligent dispatching, and in particular to an unmanned overhead crane intelligent dispatching method in a metallurgical storage area. Background Art

[0002] Overhead crane dispatching refers to the process of rationally allocating overhead cranes to complete production logistics lifting operations in order to make the overall transportation efficiency of the warehouse area consistent with production goals and ensure the orderly and compact progress of production tasks, while meeting the resource constraints and space constraints during overhead crane operation.

[0003] However, in the prior art, there is still a lack of intelligent scheduling function for unmanned overhead cranes when multiple overhead cranes work in parallel. For example, Chinese patent CN111123868B discloses a system and method for realizing intelligent scheduling of unmanned overhead cranes in hot-rolled slab warehouses. The system includes an online management module for steel loading plan, an intelligent scheduling management module, an offline automatic stacking module, an online automatic outbound module, a warehouse area warehouse map management module, an overhead crane basic data management module, a warehouse area basic setting management module, and an interface management module; the method includes configuring the overhead crane and warehouse area database, creating a steel loading plan and generating a detailed plan, and generating and executing work order instruction steps.

[0004] However, in the metallurgical storage area, there are multiple overhead cranes working in parallel and in collaboration with other logistics equipment. The existing traditional unmanned overhead crane intelligent scheduling method has low working efficiency when facing collaborative work. Therefore, there is an urgent need for an unmanned overhead crane intelligent scheduling method for metallurgical storage areas with high overhead crane utilization rate, which can adjust the parallel scheme according to equipment abnormalities and order changes to solve such problems. Summary of the invention

[0005] 1. Technical issues to be resolved

[0006] In view of the deficiencies of the prior art, the present invention provides an intelligent dispatching method for unmanned overhead cranes in a metallurgical storage area, which solves the problem of low work efficiency in the face of collaborative work in the prior art.

[0007] (II) Technical solution

[0008] To achieve the above objectives, the present invention is implemented through the following technical solutions. The present invention provides an intelligent dispatching method for an unmanned overhead crane in a metallurgical storage area, comprising:

[0009] The crane operation is divided into two lifting states: transportation operation and stacking operation. The model is established to set constraints and set the priority of the crane operation task type.

[0010] A global optimization model and a local optimization model are established to divide the overhead crane scheduling problem into two categories: global optimization and local optimization scheduling:

[0011] The global optimization model is to obtain the hoisting time and the crane that performs the hoisting operation for each slab, given the order of entry and exit of all slabs entering the warehouse.

[0012] The local optimization model is to achieve real-time allocation of overhead cranes under dynamic disturbances. On the basis of global optimization, the optimal overhead crane allocation rule is selected according to the current state of the storage area to complete the rapid real-time allocation of lifting tasks.

[0013] Gurobi, CPLEX, SCIP, and CBC solvers were used to conduct comparative tests on the constructed overhead crane dispatching model;

[0014] Conduct local optimization model and solution of overhead crane;

[0015] Define the parameters and rules of the local optimization model of the overhead crane, select the assignment rules of the overhead crane according to the actual situation of the depot area, set three basic dispatching rules based on previous dispatching rule experience, and establish the overhead crane-vehicle avoidance rules;

[0016] The local optimization algorithm of the overhead crane is implemented based on the DQN algorithm. The state action attributes are fitted through a convolutional neural network to approximate the Q value, and the experience replay strategy is used to train the neural network parameters. The ε-greedy strategy is used to select the action. Two independent and identical neural networks are used to train it. One neural network is used to obtain the latest parameters, which is called the training network, and the other neural network parameters are updated by passing the previous neural network parameters to the neural network after a period of time, which is called the target network.

[0017] (III) Beneficial effects

[0018] The present invention provides an intelligent dispatching method for unmanned overhead cranes in metallurgical storage areas. It has the following beneficial effects:

[0019] (1) The global optimization and local optimization models of the overhead crane established in the present invention can both achieve optimal solutions within seconds, among which Gurobi and CPLEX toolboxes have the highest solution efficiency.

[0020] (2) The decision tree algorithm adopted by the present invention has significantly improved the utilization rate of the overhead crane and the task processing volume compared with the single scheduling rule algorithm. The DQN algorithm proposed for dynamic disturbances can better determine the selection of an optimal overhead crane under a certain state. Its overhead crane utilization rate and task processing volume are better than those of the decision tree and the single scheduling rule, which solves the problem of low work efficiency in the face of collaborative work in the prior art. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 This is the layout diagram of the slab workshop in the present invention;

[0022] Figure 2It is a box diagram of speed ratio distribution in the present invention;

[0023] Figure 3 This is a multiple comparison result diagram in the present invention;

[0024] Figure 4 is the optimal driving path diagram in the present invention; wherein, Figure 4 (a) is the optimal driving path diagram when the safety constraint distance δ is set to 1 unit length; Figure 4 (b) is the optimal driving path diagram when the safety constraint distance δ is set to 0;

[0025] Figure 5 is the optimal scheduling solution result diagram of the overhead travelling crane in the present invention; wherein, Figure 5 (a) in the figure is the optimal scheduling result diagram of case 1#; Figure 5 (b) is the optimal scheduling result diagram of case 2#; Figure 5 (c) in the figure is the optimal scheduling result diagram of case 11#; Figure 5 (d) in the figure is the optimal scheduling result diagram of case 12#;

[0026] Figure 6 This is a diagram showing the impact of production scheduling constraints on model complexity in the present invention;

[0027] Figure 7 A rule model diagram of the scheduling mode in the present invention;

[0028] Figure 8 It is the state attribute diagram of the scheduling workshop in the present invention;

[0029] Fig. 9 This is a structural diagram of the DQN algorithm in the present invention. DETAILED DESCRIPTION

[0030] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0031] Example

[0032] Figure 1 The plan view of a span in the buffer storage area of ​​the continuous casting-hot rolling link. The slabs produced by the continuous casting machine are transported to the slab storage and the heating furnace by rollers. For slabs that the heating furnace cannot receive in time, they need to be transported to the designated area of ​​the slab storage by the overhead crane deployed between the spans. The present invention provides an intelligent dispatching method for an unmanned overhead crane in a metallurgical storage area, including:

[0033] The crane operation is divided into two lifting states: transportation operation and stacking operation. The model is established to set constraints and set the priority of the crane operation task type. Among them:

[0034] The model constraints include:

[0035] When the overhead crane performs an operation task, it only moves one slab at a time;

[0036] The movement of each slab to its final target position is performed only once by one overhead crane;

[0037] Before the target slab is moved, the number of obstructing slabs of the target slab is zero;

[0038] The obstruction slab must be moved to one position by the overhead crane and cannot be moved to other slabs;

[0039] Only when all stack positions have slabs, can the obstructing slabs be stacked at other slabs;

[0040] When stacking another obstructing slab, it must be placed on the previously hoisted obstructing slab first;

[0041] Ensure that there is enough time for idling operation between the two transportation or stacking operations;

[0042] To avoid collision between adjacent overhead cranes, a safe distance is set between the cranes and they cannot cross each other;

[0043] Ensure that the processing sequence of the front and rear processes in the warehouse area is consistent to avoid overlap of front and rear operations.

[0044] The priorities of overhead crane operation task types are as follows:

[0045] The slabs produced by the continuous casting process must be stored in the warehouse in a timely manner.

[0046] The priority of outbound delivery is second, and the supply of slabs required for the subsequent hot rolling process production must be followed up in a timely manner;

[0047] Stacking, transportation and idling have the lowest priority and are sorted according to the specific lifting plan.

[0048] A global optimization model and a local optimization model are established to divide the overhead crane scheduling problem into two categories: global optimization scheduling and local optimization scheduling:

[0049] The global optimization scheduling model is to obtain the hoisting time and the crane that performs the hoisting operation for each slab, given the order of entry and exit of all slabs entering the warehouse.

[0050] The local optimization scheduling model is designed to achieve real-time allocation of overhead cranes under dynamic disturbances. On the basis of global optimization, the optimal overhead crane allocation rule is selected according to the current status of the storage area to complete the rapid real-time allocation of lifting tasks.

[0051] The specific steps for establishing the global optimization dispatching model of the overhead crane include:

[0052] After simplifying the reservoir area plan, the layout of one of the spans is as follows Figure 1 As shown, suppose:

[0053] There are K overhead cranes in the working span, and the overhead cranes can only run along the track to perform working tasks in the same span;

[0054] The span area is divided into J rows along the direction of the working span, and the distance between adjacent rows is L;

[0055] Each row will be divided into multiple slab stacks evenly according to the width of the span, and the stacks are generally kept parallel to facilitate operation;

[0056] During operation, the overhead crane can move horizontally along the working span, and can also move vertically along the working span.

[0057] The overhead crane dispatching model in the metallurgical storage area is simplified to the intra-span operation, specifically:

[0058] Given a production order plan, which contains N tasks, lifting is achieved through K overhead cranes, and N will increase as production progresses, forming a static schedule that is connected to the production plan;

[0059] The global optimization scheduling problem can be modeled as a mixed integer programming model

[0060] Define model parameters and decision variables;

[0061] Model construction: In the process of slab storage, in order to make the overhead crane dispatching goal pursue the shortest overhead crane heavy-load travel distance, the objective function is defined as:

[0062]

[0063] The specific model parameter definitions include:

[0064] represents the set of slabs to be stored, i=1, 2, ..., I, where the slab sets corresponding to the No. 1 and No. 2 continuous casting machines are I1 and I2 respectively, I=I1+I2,

[0065] represents the set of slab warehouse areas, j = 1, 2, ..., J;

[0066] represents a set of overhead cranes, which are numbered k=1, 2, ..., K in sequence, where K is the number of overhead cranes in the span, and the speed of the overhead cranes transporting slabs is v;

[0067] represents a set of scheduling periods, t = 1, 2, ..., T;

[0068] H j represents the maximum slab capacity of region j;

[0069] and They represent the distances that crane k travels from continuous casting machines No. 1 and No. 2 to area j respectively;

[0070] r1 and r2 represent the areas where the roller tables of the No. 1 and No. 2 continuous casting machines are located, respectively;

[0071] δ represents the safe distance between adjacent overhead cranes. The minimum safe distance means that there is no risk of collision between adjacent overhead cranes during operation. M represents the coefficient of the constraint expression, which is usually set to 10000. During hoisting, the trolley on the overhead crane will automatically run synchronously with the crane along the cross-zone track by default, that is, the running time of the trolley will be incorporated into the running time of the overhead crane.

[0072] The decision variables are defined as:

[0073] x j,k,t , if at time t, crane k is located in region j, x j,k,t =1; otherwise, x j,k,t =0;

[0074] y i,j,k,t , if slab i is lifted to area j by crane k within time period t, then y i,j,k,t =1; otherwise y i,j,k,t =0;

[0075] The model constraints include:

[0076] This means that each overhead crane must be assigned to a certain area in each time period;

[0077] Indicates that the number of overhead cranes allocated to each area in each time period cannot exceed one;

[0078]

[0079] It means that if overhead crane k is assigned to column j, then the overhead cranes on its left and right must maintain a distance of δ columns;

[0080]

[0081] Indicates that the location of the overhead crane must be consistent with the overhead crane serial number;

[0082]

[0083] It means that the overhead crane with heavy load task must be the one in motion;

[0084] This means that each slab must be lifted to a certain location;

[0085] It means that the number of slabs in the jth column cannot exceed its maximum limit;

[0086]

[0087] It means that the order of slab lifting should be consistent with the output order of the continuous casting machine.

[0088] The model contains a large number of binary variables and constraints. The number of decision variables (Number of Variables, NoV) in the model is:

[0089] NoV=J×K×T+I×J×K×T=I×(J+1)×K×T

[0090] The number of constraints (NoC) is:

[0091] NoC=K×T+J×T+2(J-1)×K×(K-1)×T+2J×(K-1)×T+I×J×K×T+K×T+I+J+(I-2)J 2 ×K 2 ×T 2 =(I-2)J 2 K 2 T 2 +2(JK-K+1)(K-1)T+IJKT+2KT+I+J;

[0092] It can be seen from this that as the scale of the problem increases, the number of decision variables and constraints of the scheduling problem increases dramatically. Because the order scale of slab warehouse scheduling is large, the solution of the model has high-dimensional characteristics. In order to solve this type of NP-Hard problem, it is necessary to seek an algorithm with strong search capabilities to solve the model.

[0093] Because the overhead crane scheduling model constructed in this application belongs to a MIP (Mixed Integer Programming) problem, this application selects the following four solvers to perform comparative tests on it.

[0094] Gurobi: A multi-platform commercial mathematical optimization software developed by the American company GurobiOptimization. It is used to solve integer programming, quadratic programming, and nonlinear constrained mathematical optimization problems. It has achieved relatively excellent results in major professional optimizer competitions.

[0095] CPLEX: Optimization software developed by IBM, originally developed and commercialized by Robert E. Bixby in 1988, is the most mature and widely used commercial solver.

[0096] SCIP (Solving Constraint Integer Programs): An open source solver developed by the Free University of Berlin and the German Business University. It combines the branch and bound of MIP, cutting plane theory and search technology in constraint programming (CP) to improve the speed of solving mathematical optimization problems.

[0097] CBC (Coin-or branch and cut): A MIP solver developed by the Coin-OR open source organization. It supports multiple branching strategies and variable selection strategies, and can customize branching and selection rules. Powerful cutting plane algorithms, including non-standard cutting planes and Gomers cutting.

[0098] Gurobi, CPLEX, SCIP and CBC solvers were used to conduct comparative tests on the constructed overhead crane scheduling model.

[0099] Except for the open source solver CBC, which cannot solve large-scale problems, other solvers can solve all cases. The speed ratio distribution box plot of each solver is shown in the figure below: Figure 2 As shown:

[0100] CBC has the lowest solution efficiency and the largest fluctuation;

[0101] Gurobi and CPLEX have the highest solving efficiency and the lowest fluctuation range.

[0102] The multiple comparison results based on TukeyTest are as follows Figure 3 As shown:

[0103] The efficiency of CBC is significantly different from that of the other three solvers (their confidence intervals do not contain zero values).

[0104] The program codes involved in the numerical experiments of this application are all implemented in Python 3.7 language, and the overhead crane scheduling model is programmatically described using Pyomo modeling language; the operating environment is Intel Core i7 3.60 GHz, RAM 16 GB; the operating system is Windows 10 (64 bit).

[0105] In order to analyze the effectiveness of the overhead crane scheduling model proposed in this application, 12 groups of scheduling cases are randomly generated to test the model. According to the set cross-zone parameters, the number of billets (I), the number of cross-zone areas (J), the number of overhead cranes (K) and the number of time periods (T) are shown in Table 1.

[0106] Table 1 Group scheduling case data

[0107]

[0108] In addition, order:

[0109] Slab output from 1# and 2# continuous casting machines:

[0110] The area where 1# and 2# continuous casting machines are located: r1=2, r2=J-1.

[0111] Maximum slab capacity in zone j:

[0112] Target region set for slab i: Randomly selected from the set Extract two elements from .

[0113] Safety distance between adjacent workshops: δ=1.

[0114] Driving distance:

[0115] Consider safety distance:

[0116] The overall experiment is divided into two schemes. Considering the safety distance, the optimal driving path is as follows: Figure 4 As shown in (a) in the figure, the safety constraint distance δ is set to 1 unit length. It can be seen that the three overhead cranes always maintain a safety distance of one unit during operation. When the safety distance is not considered, the optimal driving path is as follows Figure 4 As shown in (b) in the figure, δ is set to 0, and crane-1 is in the 4th column at the 8th iteration. This situation will not occur in actual application.

[0117] Comparison of results of different test cases:

[0118] For the randomly generated test cases, Table 2 records the optimal objective value (f) of each case and the computation time (CPU, unit: seconds) of each solver, where the fastest results are indicated in bold.

[0119] From the results in Table 2, it can be seen that the increase in the problem size will lead to a corresponding increase in the model calculation time. In order to quantify the difference in solution speed, the speed ratio is calculated according to the following formula:

[0120]

[0121] Where CPU(s) is the computation time of solver p.

[0122] Table 2 Comparison test results of test cases

[0123]

[0124] According to the optimal dispatching solutions of cases 1#, 6#, 11# and 12#, Figure 5 (a), (b), (c), and (d) in the figure plot the running trajectory of each overhead crane in different time periods.

[0125] Model complexity analysis

[0126] By increasing the data scale, the complexity of the model can be analyzed. The complexity is mainly reflected in the growth of the model's variables and constraints. On the basis of setting the number of overhead cranes and single-span columns unchanged, and the safety distance to remain unchanged at 1 unit length, the amount of slabs is gradually increased, and the time range is increased synchronously. Specifically, the number of slabs is increased by 3 each time, and the time interval is increased by 1 unit time. The detailed experimental data is recorded in Table 3, where constraints-p and constraints-np respectively represent considering sorting constraints and not considering sorting constraints.

[0127] Table 3 Comparison of different algorithms

[0128]

[0129] Figure 6 The impact of production scheduling constraints on model complexity can be seen intuitively. The increase in the number of slabs will cause the variables and constraints of the model to grow exponentially. However, whether or not to consider production scheduling constraints will lead to a huge difference in the complexity of the problem, which is approximately 100-1000 times.

[0130] It can be seen from the above experimental results that when the number of incoming slab sets is 10 or 15 and the number of overhead cranes is less than 3, the overhead crane static scheduling model established in this application can achieve the optimal solution in seconds, among which Gurobi and CPLEX toolboxes have the highest solution efficiency.

[0131] Overhead crane local optimization scheduling model and solution;

[0132] In the actual production environment of metallurgical storage areas, many dynamic uncertain disturbances further increase the difficulty of overhead crane scheduling optimization, including time disturbances, equipment disturbances, and task disturbances. Among them, time disturbances are the most common type of disturbances in the steel production process, such as the uncertainty of continuous casting processing time; equipment disturbances mainly refer to the failure of overhead crane equipment; task disturbances include new work order insertion, emergency work orders, and work order cancellation. Due to the tight logistics rhythm and busy overhead crane operations in some production line storage areas, the redundancy of the scheduling plan is relatively small, so the occurrence of dynamic disturbances will not only have a direct impact on production efficiency, but also cause economic losses in serious cases. Therefore, how to quickly respond to dynamic disturbances in the storage area to ensure the efficient and stable execution of the scheduling plan has become one of the key issues that need to be solved in unmanned overhead crane scheduling.

[0133] In data-based dynamic scheduling, rule-based scheduling methods are very common, and hundreds of rules have been developed so far. As the storage environment becomes more complex, scheduling rules applicable to a specific scenario are difficult to cope with dynamic changes. Therefore, it is necessary to select appropriate scheduling rules from alternative scheduling rules in different storage environments. For this multi-overhead crane scheduling problem, each task will be served by multiple overhead cranes. In this case, it is necessary to select an optimal overhead crane for service based on the optimization goal requirements, and determine which overhead crane to provide service based on the real-time scheduling environment information in the storage area. The optimized scheduling mode can be regarded as a classification problem in machine learning, such as Figure 7 As shown, the relevant parameter definitions of the model are shown in the following table:

[0134] Table 4 Reservoir area environmental attributes

[0135]

[0136] The overhead crane local optimization scheduling parameters and rule definitions are performed, the overhead crane assignment rules are selected according to the actual situation of the storage area, and three basic scheduling rules are set based on previous scheduling rule experience. The basic scheduling rules include: assigning the overhead crane at will, assigning the overhead crane with the shortest remaining time, and assigning the overhead crane closest to the task point.

[0137] Establish rules for avoiding traffic between vehicles, including:

[0138] Assign the earliest idle overhead crane to avoid, assign an empty overhead crane to avoid, assign the overhead crane closest to the opponent's target point to avoid, assign the overhead crane close to your own target point to avoid, assign the overhead crane with a low priority to avoid.

[0139] In the mining of dispatching rules, the reasonable description of training sample data has an important impact on rule mining. When selecting sample data, it not only includes the functional structure of the storage area, but also includes the necessary storage area environment information. The input attributes are needed to describe the training instances used. These attributes need to effectively reflect the relevant dispatching environment and overhead crane operation information, and construct a set of optimal attribute sets to improve the accuracy of the model. The description of the high-speed line storage area is mainly divided into two parts: overhead crane status and task status. The overhead crane status includes the location of the overhead crane, the remaining service time of the overhead crane, and the current service time of the overhead crane. The task status mainly includes the location of the task point, the arrival time of the task point, and the type of the task point, such as Figure 8 shown.

[0140] Scheduling rules are divided into simple scheduling rules, combined scheduling rules, weighted scheduling rules and heuristic scheduling rules according to their complexity. Simple rules are rules that consider one parameter for solution, while combined rules are scheduling rules that make multiple combinations of simple scheduling rules after considering multiple factors. Weighted scheduling rules are scheduling rules that introduce weighted values ​​after considering multiple factors to consider the different effects of different rules on scheduling. Heuristic rules are rules obtained through experience.

[0141] According to the actual situation of the warehouse area, the assignment rules of the overhead crane are selected. Three basic scheduling rules are set according to the previous scheduling rule experience. In the actual production scheduling, the overhead crane is assigned according to the rules shown in Table 5.

[0142] Table 5 Rules for the assignment of overhead cranes

[0143]

[0144] In the process of intelligent production, it is not enough to have only scheduling rules. Since multiple overhead cranes are running in one span, there is a situation where the spans of the loading and unloading positions are relatively large. Therefore, there is an intersection of the running paths of the overhead cranes when performing tasks, which may lead to collisions between overhead cranes. Therefore, it is necessary to consider the rules for avoiding overhead cranes, as shown in Table 6.

[0145] Table 6 Overhead Crane Avoidance Rules

[0146]

[0147] In the dynamic scheduling problem of overhead cranes in metallurgical storage areas considering equipment failures, the scheduling system needs to select the most appropriate allocation rule based on the current storage area status. The local optimization scheduling of overhead cranes is converted into a sequence of action sequences for solving the task selection of each overhead crane, which can be modeled as a Markov decision model. Reinforcement learning is an effective method for solving such sequential decision problems. In Markov decision making, decision refers to action, and the pros and cons of strategies are evaluated in a certain evaluation method. In reinforcement learning, Q-learning algorithm and DQN algorithm are two commonly used algorithms. In general, the Q-learning algorithm can observe the reward function under a certain state in the form of a Q table, but for continuous problems or high-dimensional inputs and outputs, the Q table is difficult to store all states and actions. Replacing the Q table with a neural network can directly solve combinatorial optimization problems. Therefore, this application adopts the DQN algorithm as the learning algorithm.

[0148] The dynamic scheduling algorithm of the overhead crane is implemented based on the DQN algorithm. The Q value is approximated by fitting the state action attributes through the convolutional neural network, and the experience replay strategy is used to train the neural network parameters. The ε-greedy strategy is used to select the action. Two independent and identical neural networks are used to train it. One neural network is used to obtain the latest parameters, which is called the training network, and the other neural network parameters are updated by passing the previous neural network parameters to the neural network after a period of time, which is called the target network.

[0149] DQN is mainly reflected in the following aspects:

[0150] Experience replay: The samples of the neural network are independent of each other. Each time DQN updates its parameters, it selects a part of the experience for update and selects some past states for learning to make the neural network more efficient.

[0151] Fixed Q value: When the neural network is updated, the non-fixed parameters will make the selection strategy unstable, so the target network value is fixed;

[0152] The selection of scheduling rules adopts a three-layer deep neural network consisting of an input layer, an output layer, and an intermediate layer. The number of nodes belonging to the input layer and the output layer is the same as the number of state characteristics and available actions. The number of nodes in the intermediate layer is set to 50, and the Relu function is used as the activation function. The DQN algorithm structure is as follows: Fig. 9 shown.

[0153] After determining the rule selection strategy for the DQN algorithm, in order to make the DQN algorithm have a higher accuracy and learning rate, it is necessary to determine the three important elements of state, action and reward in reinforcement learning.

[0154] When DQN algorithm modeling is performed on the reservoir area:

[0155] First, the S in Markov decision making is described, that is, the state of the reservoir area needs to be defined and described;

[0156] It can accurately describe the current state of the warehouse area. According to the above description of the state attributes of the warehouse area, we still divide the state into two parts: the crane state and the task state, in order to specifically study the emergencies of the crane and the task;

[0157] The crane status is divided into five states: the crane's real-time position, crane's operating time, crane's utilization rate, crane's waiting queue (i.e., crane's remaining working time), and crane's health status. The task status is based on the factors considered in the actual operation of the warehouse area, and is expressed in terms of the target location of the task entering the warehouse, the task's arrival time, the task's waiting time, and the task's priority status.

[0158] In the DQN algorithm, the reward function is to evaluate the quality of the action (rule) under a certain state (depot environment), so as to encourage the depot to learn to achieve the optimal rule allocation. The reward function is set based on the performance indicators mentioned above. When the overhead crane utilization rate E is higher than the overhead crane utilization rate at the previous moment, a positive reward function is given to the behavior action, otherwise a negative reward function is returned. When the task waiting time C is lower than the estimated waiting time C i When , a positive reward function is given, and vice versa;

[0159] Since this DQN algorithm is designed to determine the selection of rules under certain conditions and then select the optimal overhead crane, different combination rules are designed based on the on-site experience, the real-time status of the storage area and the above analysis, considering the two factors of the shortest remaining time and the closest distance to the task point, so as to maximize the utilization rate of the overhead crane. Because the failure of equipment may affect the selection of the overhead crane for the current task in dynamic scheduling, this algorithm further considers the health status and utilization rate of the overhead crane. The scheduling rules for overhead crane allocation are shown in Table 7.

[0160] Table 7 Skytrain allocation rules

[0161]

[0162] The reward function settings are shown in Table 8;

[0163] Table 8 Reward function settings:

[0164]

[0165] The DQN algorithm process includes:

[0166] Initialize the sample set D and its capacity N, initialize the Q network, and randomly generate weight values ​​θ;

[0167] Initialize the target Network, weight value θ_=θ;

[0168] Forepisode=1,Mdo;

[0169] Initialize state set S;

[0170] Fort=1,Tdo;

[0171] Choose a random action a with probability ε∈(0,1), or choose a=argmax a (Q(s, a:θ));

[0172] Execute action a, accept the reward function R and the new state s′;

[0173] Store the sample (s, a, R, s′) into the sample set D;

[0174] Randomly extract minibatch samples (s, a, R, s′) from the sample set D;

[0175] When the next step is the end point is y i = R, otherwise y i =R+γmax a ′Q(s,a′;θ_);

[0176] Yes (y i -Q(s,a:θ) 2 Update θ using gradient descent;

[0177] Update the target every C steps Network, θ_=θ.

[0178] Based on the actual production data of a steel plant's high-line storage area to reflect the actual situation of the actual storage area, the data should include the crane information in the storage area (actual location of the crane, remaining service time of the crane, utilization rate of the crane, health status of the crane) and task information (task storage location, task waiting time and task priority). This experiment was carried out on a personal notebook equipped with Intel i5-10200H, 2.40GHzCPU, 16.00GBRAM and win1064 operating system through Pycharm (PythonIDE) to conduct experimental research under different rules. The parameter settings of this algorithm model are shown in Table 9.

[0179] Table 9 Training model parameter settings

[0180]

[0181] In order to verify the effectiveness of rule scheduling and DQN algorithm for crane scheduling in the storage area, this application analyzes the crane scheduling information at different times of the day and compares it with other scheduling rules, as shown in Table 10:

[0182] Table 10 Average utilization rate and number of tasks processed by overhead crane under different rules at different times

[0183]

[0184] By statistically analyzing the utilization rates and task processing quantities of overhead cranes under various different rules, it can be seen that the decision tree algorithm has significantly improved the utilization rate and task processing volume of overhead cranes compared with the single scheduling rule algorithm. The DQN algorithm proposed for dynamic disturbances can better determine the selection of an optimal overhead crane under a certain state, and its overhead crane utilization rate and task processing volume are better than those of the decision tree and single scheduling rule.

[0185] Obviously, the above embodiments of the present invention are merely examples for clearly illustrating the present invention, and are not intended to limit the embodiments of the present invention. For those skilled in the art, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to list all the embodiments here. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the claims of the present invention.

Claims

1. An intelligent dispatching method for unmanned overhead crane in metallurgical storage area, characterized in that: include: The crane operation is divided into two lifting states: transportation operation and stacking operation. The model is established to set constraints and set the priority of the crane operation task type. A global optimization model and a local optimization model are established to divide the overhead crane scheduling problem into two categories: global scheduling and local optimization scheduling: The global optimization model is to obtain the hoisting time and the crane that performs the hoisting operation for each slab, given the order of entry and exit of all slabs entering the warehouse. The local optimization model is to achieve real-time allocation of overhead cranes under dynamic disturbances. On the basis of global optimization scheduling, the optimal overhead crane allocation rule is selected according to the current state of the storage area to complete the rapid real-time allocation of lifting tasks. Gurobi, CPLEX, SCIP, and CBC solvers were used to conduct comparative tests on the constructed overhead crane dispatching model; Conduct local optimization model and solution of overhead crane; Define the parameters and rules of the local optimization model of the overhead crane, select the assignment rules of the overhead crane according to the actual situation of the depot area, set three basic dispatching rules based on previous dispatching rule experience, and establish the overhead crane-vehicle avoidance rules; The local optimization algorithm of the overhead crane is implemented based on the DQN algorithm. The state action attributes are fitted through a convolutional neural network to approximate the Q value, and the experience replay strategy is used to train the neural network parameters. The ε-greedy strategy is used to select the action. Two independent and identical neural networks are used to train it. One neural network is used to obtain the latest parameters, which is called the training network, and the other neural network parameters are updated by passing the previous neural network parameters to the neural network after a period of time, which is called the target network.

2. The intelligent dispatching method for unmanned overhead crane in metallurgical storage area according to claim 1 is characterized in that: The model constraints include: When the overhead crane performs an operation task, it only moves one slab at a time; The movement of each slab to its final target position is performed only once by one overhead crane; Before the target slab is moved, its number of obstructing slabs is zero; The obstruction is that the slab must be moved to one position by the overhead crane and cannot be moved to other slabs; Only when all stacking positions have slabs, can the obstructing slabs be stacked at other slabs; When stacking another obstruction slab, it must be placed on the previously lifted obstruction slab first; Ensure that there is enough time for idling operation between the two transportation or stacking operations; To avoid collision between adjacent overhead cranes, a safe distance is set between the cranes and they cannot cross each other; Ensure that the processing sequence of the front and rear processes in the warehouse is consistent to avoid overlap of front and rear operations.

3. The intelligent dispatching method for unmanned overhead crane in metallurgical storage area according to claim 1 is characterized in that: The priorities of the overhead crane operation task types are specifically as follows: The slabs produced by the continuous casting process must be stored in the warehouse in a timely manner. The priority of outbound delivery is second, and the supply of slabs required for the subsequent hot rolling process production must be followed up in a timely manner; Stacking, transportation and idling have the lowest priority and are sorted according to the specific lifting plan.

4. The intelligent dispatching method for unmanned overhead crane in metallurgical storage area according to claim 1 is characterized in that: The steps of establishing the global optimization dispatching model of the overhead crane specifically include: The layout of one span after simplifying the plan of the reservoir area is shown in Figure 1. Assume that there are K overhead cranes in the working span, and the overhead cranes can only run along the track to perform working tasks in the same span; The span area is divided into J rows along the direction of the working span, and the distance between adjacent rows is L; Each row will be divided into multiple slab stacks depending on the span width; During operation, the overhead crane can move horizontally along the working span, and can also move vertically along the working span. The overhead crane dispatching model in the metallurgical storage area is simplified to the intra-span operation, specifically: Given a production order plan, which contains N tasks, lifting is achieved through K overhead cranes, and N will increase as production progresses, forming a static schedule that is connected to the production plan; Define model parameters and decision variables; Model construction: In the process of slab storage, in order to make the overhead crane dispatching goal pursue the shortest overhead crane heavy-load travel distance, the objective function is defined as:

5. The intelligent dispatching method for unmanned overhead crane in metallurgical storage area according to claim 1 is characterized in that: The specific model parameter definition includes: I represents the set of slabs to be stored, i=1, 2, ..., I, where the slab sets corresponding to the No. 1 and No. 2 continuous casting machines are I1 and I2 respectively, I=I1+I2, J represents the slab warehouse area set, j = 1, 2, ..., J; K represents a set of overhead cranes, which are numbered k=1, 2, ..., K in sequence, where K is the number of overhead cranes in the span and the speed v of the overhead cranes in transporting the slabs; T represents a set of scheduling periods, t = 1, 2, ..., T; S i represents the target area set of slab i; H i represents the maximum slab capacity of region j; and They represent the distances that crane k travels from continuous casting machines No. 1 and No. 2 to area j respectively; r1 and r2 represent the areas where the roller tables of the No. 1 and No. 2 continuous casting machines are located, respectively; δ represents the safe distance between adjacent overhead cranes. The minimum safe distance means ensuring that there is no collision risk between adjacent overhead cranes during operation. M represents the coefficient of the constraint expression and is set to 10000. During hoisting, the trolley on the overhead crane will automatically run synchronously with the crane along the cross-zone track by default, that is, the running time of the trolley will be incorporated into the running time of the overhead crane.

6. The intelligent dispatching method for unmanned overhead crane in metallurgical storage area according to claim 1 is characterized in that: The decision variables are defined as: x j,k,t , if at time t, crane k is located in region j, x j,k,t =1; otherwise, x j,k,t =0; y i,j,k,t , if slab i is lifted to area j by crane k within time period t, then y i,j,k,t =1; otherwise y i,j,k,t =0.

7. The intelligent dispatching method for unmanned overhead crane in metallurgical storage area according to claim 1 is characterized in that: The model constraints include: This means that each overhead crane must be assigned to a certain area in each time period; It is required that the number of overhead cranes allocated to each area in each period should not exceed one; It means that if overhead crane k is assigned to column j, then the overhead cranes on its left and right must maintain a distance of δ columns; Indicates that the location of the overhead crane must be consistent with the overhead crane serial number; It means that the overhead crane with heavy load task must be the one in motion; This means that each slab must be lifted to a certain location; It means that the number of slabs in the jth column cannot exceed its maximum limit; It means that the order of slab lifting should be consistent with the output order of the continuous casting machine.

8. The intelligent dispatching method for unmanned overhead crane in metallurgical storage area according to claim 1 is characterized in that: The number of decision variables (NoV) in the model is: NoV=J×K×T+I×J×K×T =I×(J+1)×K×T; The number of constraints (NoC) is: NoC=K×T+J×T+2(J-1)×K×(K-1)×T+2J×(K-1)×T+I×J×K×T+K×T+I+J+(I-2)J 2 ×K 2 ×T 2 =(I-2)J 2 K 2 T 2 +2(JK-K+1)(K-1)T+IJKT+2KT+I+J。 9. The intelligent dispatching method for unmanned overhead crane in metallurgical storage area according to claim 1 is characterized in that: The basic scheduling rules include: Assign an overhead crane at will, assign the crane with the shortest remaining time, or assign the crane closest to the task point; The sky-to-vehicle avoidance rules include: Assign the earliest idle overhead crane to avoid, assign an empty overhead crane to avoid, assign the overhead crane closest to the opponent's target point to avoid, assign the overhead crane close to your own target point to avoid, assign the overhead crane with a low priority to avoid.

10. The intelligent dispatching method for unmanned overhead crane in metallurgical storage area according to claim 1 is characterized in that: The DQN algorithm process includes: Initialize the sample set D and its capacity N, initialize the Q network, and randomly generate weight values ​​θ; Initialize the target Network, weight value θ - =θ; Forepisode=1,Mdo; Initialize state set S; Fort=1,Tdo; Choose a random action a with probability ε∈(0,1), or choose a=argmax a (Q(s, a:θ)); Execute action a, accept the reward function R and the new state s′; Store the sample (s, a, R, s′) into the sample set D; Randomly extract minibatch samples (s, a, R, s′) from the sample set D; When the next step is the end point is y i = R, otherwise y i =R+γmax a′ Q(s, a′; θ - ); Yes (y i -Q(s,a:θ) 2 Update θ using gradient descent; Update the target every C steps Network, θ=θ.

Citation Information

Patent Citations

  • Multi-agent-based steelmaking workshop crane scheduling simulation method

    CN101944201A

  • Metallurgical industry workshop crown block scheduling method based on deep reinforcement learning

    CN113837628A

  • Cooperative scheduling method for multiple crown blocks in flexible manufacturing system

    CN116822884A

  • Crown block double-layer planning intelligent scheduling optimization method, device, equipment and medium

    CN117952389A

  • Intelligent overhead crane dispatching integrated management system

    WO2023065682A1

Cited By

  • Method and system for intelligent scheduling of multiple rows of vehicles in unmanned metallurgical plant area

    CN122864942A