Intelligent scheduling method for unmanned overhead cranes in metallurgical storage areas

Abstract

This invention discloses an intelligent scheduling method for unmanned overhead cranes in metallurgical warehouses, relating to the field of unmanned overhead crane scheduling technology. This invention establishes a global optimization scheduling model for a multi-task, multi-crane cluster and proposes an efficient parallel solution method. Simultaneously, for equipment anomalies and order changes causing disturbances, a local optimization model for crane scheduling is established, and a solution method based on scheduling rules is proposed. Both the global and local optimization models can achieve optimal solutions within seconds, with the Gurobi and CPLEX toolboxes showing the highest solution efficiency. Furthermore, the decision tree algorithm significantly improves crane utilization and task processing volume compared to a single scheduling rule algorithm. The DQN algorithm, proposed to address dynamic disturbances, is better able to determine the optimal crane under certain conditions, achieving superior crane utilization and task processing volume compared to decision trees and single scheduling rules, thus solving the problem of low work efficiency in collaborative work in existing technologies.

Description

Technical Field

[0001] This invention relates to the field of intelligent scheduling technology for unmanned overhead cranes, specifically to an intelligent scheduling method for unmanned overhead cranes in metallurgical storage areas. Background Technology

[0002] Overhead crane scheduling refers to the process of rationally allocating overhead cranes to complete production logistics hoisting operations, under the conditions of meeting the resource and space constraints of overhead crane operation, in order to ensure that the overall transportation efficiency of the warehouse area is consistent with the production goals and to ensure the orderly and compact progress of production tasks.

[0003] However, existing technologies still lack intelligent scheduling functions for unmanned overhead cranes when multiple cranes are working in parallel. For example, Chinese patent CN111123868B discloses a system and method for intelligent scheduling of unmanned overhead cranes in hot-rolled slab warehouses. The system includes an online steel loading plan management module, an intelligent scheduling management module, an automatic offline stacking module, an automatic online outbound module, a warehouse area map management module, an overhead crane basic data management module, a warehouse area basic settings management module, and an interface management module. The method includes configuring the overhead crane and warehouse area database, creating a steel loading plan and generating a detailed plan, and generating and executing work order instructions.

[0004] In the context of metallurgical warehouse environments, multiple overhead cranes operate in parallel and collaborate with other logistics equipment. Existing traditional intelligent scheduling methods for unmanned overhead cranes are inefficient when dealing with collaborative work. Therefore, there is an urgent need for an intelligent scheduling method for unmanned overhead cranes in metallurgical warehouses that can adjust the parallel scheme in response to equipment malfunctions and order changes, thereby improving crane utilization and solving this problem. Summary of the Invention

[0005] (a) Technical problems to be solved

[0006] To address the shortcomings of existing technologies, this invention provides an intelligent scheduling method for unmanned overhead cranes in metallurgical warehouses, solving the problem of low work efficiency when facing collaborative work in existing technologies.

[0007] (II) Technical Solution

[0008] To achieve the above objectives, the present invention provides an intelligent scheduling method for unmanned overhead cranes in metallurgical warehouses, comprising:

[0009] The overhead crane operation is divided into two lifting states: transportation operation and stacking operation. A model is established, constraints are set, and the priority of overhead crane operation task type is set.

[0010] Establish global optimization and local optimization models to classify the overhead crane scheduling problem into two categories: global optimization and local optimization scheduling.

[0011] The global optimization model is based on the known entry and exit order of all inbound slab sets, and calculates the lifting time and the overhead crane that performs the lifting operation for each slab.

[0012] The local optimization model is designed to achieve real-time allocation of overhead cranes under dynamic disturbances. Based on global optimization, it selects the optimal overhead crane allocation rule according to the current state of the storage area to complete the rapid real-time allocation of hoisting tasks.

[0013] The constructed overhead crane scheduling model was compared and tested using Gurobi, CPLEX, SCIP, and CBC solvers.

[0014] Perform local optimization modeling and solution for the overhead crane;

[0015] Define the parameters and rules of the local optimization model for overhead cranes, select the assignment rules for overhead cranes based on the actual situation of the storage area, set three basic scheduling rules based on past scheduling rule experience, and establish the avoidance rules between overhead cranes.

[0016] The overhead crane local optimization algorithm is implemented based on the DQN algorithm. The Q-value is approximated by fitting the state-action attributes through a convolutional neural network. An empirical replay strategy is used to train the neural network parameters, and an ε-greedy strategy is used to select actions. Two independent and identical neural networks are used for training. One neural network is used to obtain the latest parameters and is called the training network. The parameters of the other neural network are updated by passing the parameters of the former neural network to the target neural network after a period of time.

[0017] (III) Beneficial Effects

[0018] This invention provides an intelligent scheduling method for unmanned overhead cranes in metallurgical storage areas. It offers the following advantages:

[0019] (1) The global and local optimization models of the overhead crane established in this invention can achieve optimal solutions in seconds, with the Gurobi and CPLEX toolboxes having the highest solution efficiency.

[0020] (2) The decision tree algorithm used in this invention has significantly improved the overhead crane utilization and task processing volume compared with the single scheduling rule algorithm. The DQN algorithm proposed for dynamic disturbances is more able to determine the optimal overhead crane in a certain state. Its overhead crane utilization and task processing volume are better than those of decision tree and single scheduling rule, solving the problem of low work efficiency when facing collaborative work in the prior art. Attached Figure Description

[0021] Figure 1 This is a layout diagram of the slab workshop in this invention;

[0022] Figure 2This is a box plot of the velocity ratio distribution in this invention;

[0023] Figure 3 This is a graph showing the results of multiple comparisons in this invention;

[0024] Figure 4 This is the optimal driving path diagram in this invention; wherein, Figure 4 (a) in the diagram represents the optimal driving path when the safety constraint distance δ is set to 1 unit length. Figure 4 (b) in the diagram represents the optimal driving path when the safety constraint distance δ is set to 0.

[0025] Figure 5 This is a diagram showing the optimal scheduling solution for overhead cranes in this invention; wherein, Figure 5 (a) in the diagram represents the optimal scheduling result for case #1. Figure 5 (b) in the diagram shows the optimal scheduling result for case #2; Figure 5 (c) in the diagram represents the optimal scheduling result for case #11. Figure 5 (d) in the diagram represents the optimal scheduling result for case #12.

[0026] Figure 6 This is a diagram illustrating the impact of production scheduling constraints on model complexity in this invention.

[0027] Figure 7 This is a rule model diagram of the scheduling mode in this invention;

[0028] Figure 8 This is a status attribute diagram of the scheduling workshop in this invention;

[0029] Figure 9 This is a structural diagram of the DQN algorithm in this invention. Detailed Implementation

[0030] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0031] Example

[0032] Figure 1 This is a plan view of one span in the buffer storage area of ​​the continuous casting-hot rolling process. Slabs produced by the continuous casting machine are conveyed to the slab storage and heating furnace via roller conveyors. Slabs that the heating furnace cannot receive in time need to be transported to a designated area in the slab storage by overhead cranes deployed between the spans. This invention provides an intelligent scheduling method for unmanned overhead cranes in metallurgical storage areas, including:

[0033] The overhead crane operations are divided into two lifting states: transportation operations and stacking operations. A model is established, constraints are set, and the priority of overhead crane operation task types is defined. Among these:

[0034] The model constraints include:

[0035] When the overhead crane performs its work, it moves only one slab at a time.

[0036] The movement of each slab to its final target position is performed only once by a single overhead crane.

[0037] Before the target slab is moved, the number of obstructing slabs is zero.

[0038] The slab must be moved to a certain position by the overhead crane and cannot be moved to other slabs;

[0039] Only when all stack positions have slabs can obstructing slabs be placed at other slab locations;

[0040] When restacking another obstructing slab, it must be placed on top of the obstructing slab that has already been hoisted.

[0041] Sufficient time should be allowed for idle operation between two consecutive transport or repacking operations;

[0042] To avoid collisions between adjacent overhead cranes, a safe distance must be set between them, and they must not operate in a cross-operation manner.

[0043] Ensure that the processing sequence of each process in the warehouse area is consistent to avoid overlap between operations.

[0044] The priority of overhead crane operation task types is as follows:

[0045] The highest priority is for warehousing; slabs produced by the continuous casting machine process must be stored in the warehouse in a timely manner.

[0046] The priority for outbound shipments is secondary; the supply of slabs required for subsequent hot rolling processes must be followed up in a timely manner.

[0047] Stacking, transportation, and idle operation have the lowest priority and are sorted according to the specific hoisting plan.

[0048] Establish global optimization models and local optimization models to classify the overhead crane scheduling problem into two categories: global optimization scheduling and local optimization scheduling.

[0049] The global optimization scheduling model is based on the known inbound and outbound order of all inbound slab sets, and calculates the lifting time and the overhead crane that performs the lifting operation for each slab.

[0050] The local optimization scheduling model is designed to achieve real-time allocation of overhead cranes under dynamic disturbances. Based on global optimization, it selects the optimal overhead crane allocation rule according to the current state of the storage area to complete the rapid real-time allocation of hoisting tasks.

[0051] The specific steps for establishing the overhead crane global optimization scheduling model include:

[0052] The simplified layout of one span of the reservoir area plan is as follows: Figure 1 As shown, let:

[0053] There are K overhead cranes in the work area. The cranes can only move along the tracks in the same span to perform the work tasks.

[0054] The area is divided into J rows along the direction of the work span, with a spacing of L between adjacent rows;

[0055] Each row will be divided into multiple slab stacks evenly according to the width of the span, and the stacks will generally be kept parallel to facilitate operation.

[0056] During operation, the overhead crane can move horizontally along the working span and vertically along the working span.

[0057] The overhead crane scheduling model for the metallurgical storage area is simplified to intra-span operations, specifically:

[0058] Given a production order plan containing N job tasks, transported by K overhead cranes, and N will continuously increase as production progresses, forming a static schedule that is linked to the production plan;

[0059] This globally optimized scheduling problem can be modeled as a mixed-integer programming model.

[0060] Define model parameters and decision variables;

[0061] Model Construction: During the slab storage process, to minimize the overhead crane's heavy-load travel distance during crane scheduling, the objective function is defined as follows:

[0062]

[0063] The specific model parameter definitions include:

[0064] Let i represent the set of slabs to be put into storage, i = 1, 2, ..., I, where the sets of slabs corresponding to continuous casting machines No. 1 and No. 2 are I1 and I2, respectively, and I = I1 + I2.

[0065] Let j represent the set of slab storage regions, j = 1, 2, ..., J;

[0066] Let K represent the set of overhead cranes, numbered sequentially as k = 1, 2, ..., K, where K is the number of overhead cranes in the span, and the speed at which the overhead cranes transport the slab is v.

[0067] Let t represent the set of scheduling periods, t = 1, 2, ..., T;

[0068] H j This indicates the maximum slab capacity in region j;

[0069] and These represent the distances traveled by the overhead crane k from continuous casting machines 1 and 2 to region j, respectively.

[0070] r1 and r2 represent the areas where the roller conveyors of continuous casting machines No. 1 and No. 2 are located, respectively;

[0071] δ represents the safe distance between adjacent overhead cranes. The minimum safe distance means ensuring that adjacent overhead cranes do not have a collision risk during operation. M represents the coefficient of the constraint expression, which is usually set to 10000. During hoisting, the trolley on the overhead crane will automatically run synchronously with the overhead crane along the cross-zone track, that is, the running time of the trolley is included in the running time of the overhead crane.

[0072] Decision variables are defined as:

[0073] x j，k，t If at time t, the crane k is located in region j, x j，k，t =1; otherwise, x j，k，t =0;

[0074] y i，j，k，t If slab i is transported to area j by overhead crane k within time period t, then y i，j，k，t =1; otherwise y i，j，k，t =0;

[0075] The model constraints include:

[0076] This means that each overhead crane must be assigned to a specific area during each time period;

[0077] This means that the number of overhead cranes allocated to each area in each time period cannot exceed one;

[0078]

[0079] This means that if crane k is assigned to column j, then the cranes to its left and right must maintain a distance of column δ.

[0080]

[0081] This indicates that the location of the overhead crane must be consistent with its serial number;

[0082]

[0083] This indicates that the overhead crane carrying a heavy load must be an overhead crane that is currently in motion.

[0084] This indicates that each slab must be hoisted to a specific location;

[0085] This indicates that the number of slabs in the j-th column cannot exceed its maximum limit;

[0086]

[0087] This indicates that the timing of slab hoisting should be consistent with the output sequence of the continuous casting machine.

[0088] The model contains a large number of binary variables and constraints. The number of decision variables (NoV) in the model is:

[0089] NoV=J×K×T+I×J×K×T=I×(J+1)×K×T

[0090] The number of constraints (NoC) is:

[0091] NoC=K×T+J×T+2(J-1)×K×(K-1)×T+2J×(K-1)×T+I×J×K×T+K×T+I+J+(I-2)J 2 ×K 2 ×T 2 =(I-2)J 2 K 2 T 2 +2(JK-K+1)(K-1)T+IJKT+2KT+I+J;

[0092] This shows that as the problem size increases, the number of decision variables and constraints in the scheduling problem increases dramatically. Because the order size of slab warehouse scheduling is very large, the solution of the model has high-dimensional characteristics. In order to solve this type of NP-hard problem, it is necessary to seek an algorithm with strong search capabilities to solve the model.

[0093] Because the overhead crane scheduling model constructed in this application belongs to a MIP (Mixed Integer Programming) problem, the following four solvers are selected for comparative testing.

[0094] Gurobi: A commercial mathematical optimization software developed by GurobiOptimization, Inc. in the United States, supporting multiple platforms. It is used to solve integer programming, quadratic programming, and nonlinear constrained mathematical optimization problems, and has achieved excellent results in various professional optimizer evaluations.

[0095] CPLEX: An optimization software developed by IBM, initially developed and commercialized by Robert E. Bixby in 1988, it is the most mature and widely used commercial solver.

[0096] SCIP (SolvingConstraintIntegerPrograms) is an open-source solver developed collaboratively by the Free University of Berlin and the German University of Commerce. It combines branch and bound theory, cutting plane theory, and search techniques from constraint programming (CP) to improve the speed of solving mathematical optimization problems.

[0097] CBC (Coin-or branchandcut): A MIP solver developed by the Coin-OR open-source organization. It supports various branching and variable selection strategies, and allows for custom branching and selection rules. It features powerful cutting plane algorithms, including invariant cutting planes and Gomers cutting.

[0098] The constructed crane scheduling model was compared and tested using Gurobi, CPLEX, SCIP, and CBC solvers.

[0099] Except for the open-source solver CBC, which cannot solve large-scale problems, all other solvers can solve all cases. The box plots of the speed ratio distribution of each solver are shown below. Figure 2 As shown:

[0100] CBC has the lowest solution efficiency and the largest fluctuation.

[0101] Gurobi and CPLEX offer the highest solution efficiency and have the lowest fluctuation range.

[0102] The results of multiple comparisons based on TukeyTest are as follows: Figure 3 As shown:

[0103] CBC differs significantly in efficiency from the other three solvers (its confidence interval does not include zero values).

[0104] The program code involved in the numerical experiments of this application is all implemented in Python 3.7 language, and the Pyomo modeling language is used to describe the crane scheduling model in a programmatic way; the running environment is Intel Core i7 3.60GHz, RAM 16GB; the operating system is Windows 10 (64bit).

[0105] To analyze the effectiveness of the overhead crane scheduling model proposed in this application, 12 sets of scheduling cases were randomly generated to test the model. Based on the set cross-regional parameters, the number of billets (I), the number of cross-regional areas (J), the number of overhead cranes (K), and the number of time periods (T) are shown in Table 1.

[0106] Table 1 shows the scheduling case data.

[0107]

[0108] Furthermore, let:

[0109] Collection of output slabs from continuous casting machines #1 and #2:

[0110] The areas where continuous casting machines #1 and #2 are located: r1 = 2, r2 = J-1.

[0111] Maximum slab capacity in region j:

[0112] The target region set of slab i: randomly selected from the set Extract two elements from the given data.

[0113] Safety distance between adjacent workstations: δ = 1.

[0114] Driving distance:

[0115] Should a safe distance be considered?

[0116] The overall experiment was divided into two schemes. Considering the safety distance, the optimal driving path was as follows: Figure 4 As shown in (a), the safety constraint distance δ is set to 1 unit length. It can be seen that the three cranes maintain a safe distance of one unit throughout their operation. Without considering the safety distance, the optimal travel path is as follows: Figure 4 As shown in (b), δ is set to 0 at this time, and the crane-1 is in the 4th column in the 8th iteration. This situation will not occur in actual applications.

[0117] Comparison of results from different test cases:

[0118] For the randomly generated test cases, Table 2 records the optimal objective value (f) for each case and the computation time (CPU, in seconds) for each solver, with the fastest result shown in bold.

[0119] As shown in Table 2, an increase in problem size leads to a corresponding increase in model computation time. To quantify the difference in solution speed, the speed ratio is calculated using the following formula:

[0120]

[0121] Where CPU(s) is the computation time of solver p.

[0122] Table 2 Comparison of Test Results for Group 2 Test Cases

[0123]

[0124] Based on the optimal scheduling solutions for cases 1#, 6#, 11#, and 12#, Figure 5 Figures (a), (b), (c), and (d) show the running trajectory of each overhead crane at different times.

[0125] Model complexity analysis

[0126] By increasing the data scale, the complexity of the model can be analyzed. This complexity is mainly reflected in the growth of the model's variables and constraints. With the overhead crane, the number of columns per span remaining constant, and the safety distance kept constant at 1 unit length, the number of slabs was gradually increased, and the time range was increased simultaneously. Specifically, the number of slabs was increased by 3 each time, with an increase time interval of 1 unit. Detailed experimental data are recorded in Table 3, where constraints-p and constraints-np represent considering and not considering sorting constraints, respectively.

[0127] Table 3 Comparison of Different Algorithms

[0128]

[0129] Figure 6 It is clear that production scheduling constraints affect the complexity of the model. An increase in the number of slabs will lead to an exponential increase in the variables and constraints of the model. However, whether or not production scheduling constraints are considered will result in a huge difference in the complexity of the problem, approximately between 100 and 1000 times.

[0130] The experimental results above show that when the number of slab sets in the warehouse is 10 or 15 and the number of overhead cranes is less than 3, the static scheduling model of overhead cranes established in this application can achieve optimal solution in the second level, among which Gurobi and CPLEX toolbox have the highest solution efficiency.

[0131] Local optimization scheduling model and solution for overhead cranes;

[0132] In the actual production environment of metallurgical warehouses, numerous dynamic and uncertain disturbances further increase the difficulty of overhead crane scheduling optimization. These include time-related disturbances, equipment-related disturbances, and task-related disturbances. Among them, time-related disturbances are the most common type of disturbance in steel production, such as the uncertainty of continuous casting processing time. Equipment-related disturbances mainly refer to overhead crane equipment failures. Task-related disturbances include the insertion of new work orders, emergency work orders, and work order cancellations. Due to the tight logistics rhythm and busy overhead crane operations in some production line warehouses, the redundancy of the scheduling plan is small. Therefore, the occurrence of dynamic disturbances not only has a direct impact on production efficiency but can also cause economic losses in severe cases. Therefore, how to respond quickly to dynamic disturbances in the warehouse area to ensure the efficient and stable execution of the scheduling plan has become one of the key issues that urgently need to be solved in unmanned overhead crane scheduling.

[0133] In data-driven dynamic scheduling, rule-based scheduling methods are prevalent, with hundreds of rules now available. However, as the storage environment becomes more complex, scheduling rules suitable for specific scenarios struggle to cope with dynamic changes. Therefore, it's necessary to select appropriate scheduling rules from a pool of options under different storage environments. For this multi-crane scheduling problem, each task is served by multiple cranes. In this case, it's crucial to select the optimal crane based on the optimization objective. Determining which crane to serve based on real-time scheduling environment information within the storage area can be viewed as a classification problem in machine learning, such as... Figure 7 As shown in the table below, the relevant parameters of the model are defined as follows:

[0134] Table 4 Representation of Environmental Attributes of the Reservoir Area

[0135]

[0136] The overhead crane local optimization scheduling parameters and rules are defined. Based on the actual situation of the storage area, the overhead crane assignment rules are selected. Based on past scheduling rule experience, three basic scheduling rules are set, including: arbitrarily assigning overhead cranes, assigning the overhead crane with the shortest remaining time, and assigning the overhead crane closest to the task point.

[0137] Establish rules for obstacle avoidance between overhead lines, including:

[0138] The system can assign the earliest available overhead crane to give way, assign an empty overhead crane to give way, assign the overhead crane closest to the other party's target point to give way, assign the overhead crane closest to its own target point to give way, or assign an overhead crane with low priority to give way.

[0139] In scheduling rule mining, a reasonable description of training sample data has a significant impact on rule mining. When selecting sample data, it's essential to consider not only the functional structure of the storage area but also necessary environmental information. Input attributes are needed to describe the training instances used. These attributes need to effectively reflect information about the scheduling environment and crane operation, constructing an optimal attribute set to improve model accuracy. The description of the high-altitude storage area mainly consists of two parts: crane status and task status. Crane status includes crane position, remaining service time, and current service time. Task status mainly includes task point location, task point arrival time, and task point type, such as... Figure 8 As shown.

[0140] Scheduling rules are classified into simple scheduling rules, combined scheduling rules, weighted scheduling rules, and heuristic scheduling rules based on their complexity. Simple rules are rules that are solved by considering one parameter, while combined rules are scheduling rules that combine simple scheduling rules in multiple ways after considering multiple factors. Weighted scheduling rules are scheduling rules that introduce weights after considering multiple factors to account for the different impacts of different rules on scheduling. Heuristic rules are rules derived from experience.

[0141] Based on the actual situation in the storage area, the overhead crane assignment rules were selected. Three basic scheduling rules were set based on past scheduling experience. In actual production scheduling, the overhead cranes were assigned according to the rules shown in Table 5.

[0142] Table 5. Rules for Assigning Overhead Cars

[0143]

[0144] In the process of intelligent manufacturing, scheduling rules alone are not enough. Since multiple overhead cranes operate in one span, there are situations where the loading and unloading positions are quite far apart. Therefore, the cranes may have intersecting paths when performing tasks, which could lead to collisions between the cranes. Thus, it is necessary to consider the rules for crane-to-crane avoidance, as shown in Table 6.

[0145] Table 6 Rules for Overhead Crane Yielding

[0146]

[0147] In the dynamic scheduling problem of overhead cranes in a metallurgical warehouse considering equipment failure, the scheduling system needs to select the most suitable allocation rule based on the current state of the warehouse. Transforming the local optimization scheduling of overhead cranes into solving the sequence of action choices for each crane can be modeled as a Markov decision model. Reinforcement learning is an effective method for solving such sequential decision problems. In Markov decision-making, a decision refers to an action, and the merits of the strategy are evaluated using a certain assessment method. In reinforcement learning, Q-learning and DQN algorithms are two commonly used algorithms. Generally, Q-learning algorithms can observe the reward function of a certain state through a Q-table. However, for continuous problems or high-dimensional inputs and outputs, Q-tables are difficult to store all states and actions. Replacing Q-tables with neural networks can directly solve combinatorial optimization problems. Therefore, this application adopts the DQN algorithm as the learning algorithm.

[0148] The overhead crane dynamic scheduling algorithm is implemented based on the DQN algorithm. The Q-value is approximated by fitting the state and action attributes through a convolutional neural network. An experience replay strategy is used to train the neural network parameters, and an ε-greedy strategy is used to select actions. Two independent and identical neural networks are used for training. One neural network is used to obtain the latest parameters and is called the training network. The parameters of the other neural network are updated by passing the parameters of the former neural network to the target neural network after a period of time.

[0149] DQN is mainly reflected in the following aspects:

[0150] Experience replay: The samples in the neural network are independent of each other. Each time DQN updates its parameters, a portion of the experience is selected for updating, and some past states are selected for learning to make the neural network more efficient.

[0151] Fixed Q-value: When the neural network is updated, if the parameters are not fixed, the selection strategy will be unstable, so the target network value is fixed.

[0152] The scheduling rule is selected using a three-layer deep neural network consisting of an input layer, an output layer, and an intermediate layer. The number of nodes in the input and output layers is the same as the number of state characteristics and available actions. The number of nodes in the intermediate layer is set to 50, and the ReLU function is used as the activation function. The DQN algorithm structure is as follows: Figure 9 As shown.

[0153] After determining the rule selection strategy for the DQN algorithm, in order to achieve high accuracy and learning rate, it is necessary to define the three important elements of reinforcement learning: state, action, and reward.

[0154] When modeling the reservoir area using the DQN algorithm:

[0155] First, we need to define S in Markov decision-making, that is, to define and describe the state of the reservoir area;

[0156] It can accurately describe the current state of the storage area. Based on the description of the storage area state attributes above, we still divide the state into two parts: crane state and task state, in order to study the sudden situations of cranes and tasks in detail.

[0157] The crane status is divided into five states: real-time position of the crane, crane running time, crane utilization rate, crane waiting queue (i.e., crane remaining working time) and crane health status. The task status is based on the factors considered during actual operation in the warehouse area, and is divided into the target location of task entry, task arrival time, task waiting time, and task priority status.

[0158] In the DQN algorithm, the reward function evaluates the quality of an action (rule) in a certain state (reservoir environment), thereby prompting the reservoir to learn and achieve optimal rule allocation. Based on the performance metrics mentioned above, the reward function is set such that when the crane utilization rate E is higher than the crane utilization rate of the previous time step, a positive reward function is given to the action; otherwise, a negative reward function is returned. When the task waiting time C is lower than the estimated waiting time C0, the reward function is applied. i When the reward is positive, a positive reward function is given; otherwise, a negative reward function is given.

[0159] Since this DQN algorithm is designed to determine the selection of rules under certain conditions and thus select the optimal overhead crane, different combination rules are designed based on field experience, the real-time status of the storage area, and the analysis above. The two factors of shortest remaining time and closest distance to the task point are considered respectively to maximize the utilization rate of the overhead crane. Because equipment failure in dynamic scheduling may affect the selection of overhead crane for the current task, this algorithm further considers the health status and utilization rate of the overhead crane. The scheduling rules for overhead crane allocation are shown in Table 7.

[0160] Table 7. Daycar Allocation Rules

[0161]

[0162] The reward function settings are shown in Table 8.

[0163] Table 8. Reward function settings:

[0164]

[0165] The DQN algorithm process includes:

[0166] Initialize the sample set D and its capacity N, initialize the Q network, and randomly generate weight values ​​θ;

[0167] Initialize target In a network, the weights θ_ = θ;

[0168] Forepisode = 1, Mdo;

[0169] Initialize the state set S;

[0170] Fort = 1, Tdo;

[0171] Choose a random action a with probability ε∈(0,1), or choose a = argmax. a (Q(s, a: θ));

[0172] Perform action a, and receive the reward function R and the new state s′;

[0173] Store the sample (s, a, R, s′) into the sample set D;

[0174] Randomly draw samples (s, a, R, s′) from the minibatch of the sample set D;

[0175] When the next step is the endpoint is y i =R, otherwise y i =R+γmax a ′Q(s,a′;θ_);

[0176] For (y) i -Q(s, a: θ)) 2 θ is updated using gradient descent;

[0177] Update the target every C steps Network, θ_=θ.

[0178] Based on actual production data from a steel plant's high-speed wire rod storage area to reflect the real situation of the storage area, the data should include crane information (actual crane location, remaining service time, crane utilization rate, and crane health status) and task information (task entry location, task waiting time, and task priority). This experiment was conducted on a personal laptop equipped with an Intel i5-10200H, 2.40GHz CPU, 16.00GB RAM, and a Win10 64-bit operating system, using PyCharm (Python IDE) under different rules. The parameter settings of this algorithm model are shown in Table 9.

[0179] Table 9 Training Model Parameter Settings

[0180]

[0181] To verify the effectiveness of rule-based scheduling and the DQN algorithm for crane scheduling in the storage area, this application analyzes crane scheduling information at different times of the day and compares it with other scheduling rules, as shown in Table 10:

[0182] Table 10 Average crane utilization and number of tasks processed under different rules at different times.

[0183]

[0184] Statistical analysis of overhead crane utilization and task processing volume under various different rules shows that the decision tree algorithm significantly improves overhead crane utilization and task processing volume compared to a single scheduling rule algorithm. Furthermore, the DQN algorithm, proposed to address dynamic disturbances, is better able to determine the optimal overhead crane under a certain state, and its overhead crane utilization and task processing volume are superior to both decision trees and single scheduling rules.

[0185] Obviously, the above embodiments of the present invention are merely examples for clearly illustrating the present invention, and are not intended to limit the implementation of the present invention. Those skilled in the art can make other variations or modifications based on the above description. It is neither necessary nor possible to exhaustively describe all embodiments here. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the claims of the present invention.

Claims

1. A method for intelligent scheduling of unmanned overhead cranes in metallurgical storage areas, characterized in that: include: The overhead crane operation is divided into two lifting states: transportation operation and stacking operation. A model is established, constraints are set, and the priority of overhead crane operation task type is set. Establish global optimization models and local optimization models to classify the overhead crane scheduling problem into two categories: global scheduling and local optimization scheduling. The global optimization model is based on the known entry and exit order of all inbound slab sets, and calculates the lifting time and the overhead crane that performs the lifting operation for each slab. The local optimization model is designed to achieve real-time allocation of overhead cranes under dynamic disturbances. Based on global optimization scheduling, it selects the optimal overhead crane allocation rule according to the current state of the storage area to complete the rapid real-time allocation of hoisting tasks. The constructed overhead crane scheduling model was compared and tested using Gurobi, CPLEX, SCIP, and CBC solvers. Perform local optimization modeling and solution for the overhead crane; Define the parameters and rules of the local optimization model for overhead cranes, select the assignment rules for overhead cranes based on the actual situation of the storage area, set three basic scheduling rules based on past scheduling rule experience, and establish the avoidance rules between overhead cranes. The overhead crane local optimization algorithm is implemented based on the DQN algorithm. The Q value is approximated by fitting the state and action attributes through a convolutional neural network. An empirical replay strategy is used to train the neural network parameters, and an ε-greedy strategy is used to select actions. Two independent and identical neural networks are used for training. One neural network is used to obtain the latest parameters and is called the training network. The parameters of the other neural network are updated by passing the parameters of the previous neural network to the target network after a period of time. The model constraints include: When the overhead crane performs its work, it moves only one slab at a time. The movement of each slab to its final target position is performed only once by a single overhead crane. Before the target slab is moved, the number of obstructing slabs is zero; The obstruction prevents the slab from being moved to a specific location by the overhead crane, thus preventing it from reaching other slabs. Only when all stack positions have slabs can obstructing slabs be placed at other slab locations; When reloading another obstructing slab, it must be placed on top of the obstructing slab that has already been hoisted. Sufficient time should be allowed for idle operation between two consecutive transport or repacking operations; To avoid collisions between adjacent overhead cranes, a safe distance must be set between them, and they must not operate in a cross-operation manner. Ensure that the processing sequence of each process in the warehouse is consistent to avoid overlap between operations. The priority of the overhead crane operation task types is as follows: The highest priority is for warehousing; slabs produced by the continuous casting machine process must be stored in the warehouse in a timely manner. The priority for outbound shipments is secondary; the supply of slabs required for subsequent hot rolling processes must be followed up in a timely manner. Reloading, transportation, and idle operation have the lowest priority and are sorted according to the specific hoisting plan. The steps for establishing the overhead crane global optimization scheduling model specifically include: Suppose there are K overhead cranes in the work span, and the cranes can only move along the tracks in the same span to perform the work tasks. The area is divided into J rows along the direction of the work span, with a spacing of L between adjacent rows; Each row will be divided into multiple slab stacks according to the width of the span; During operation, the overhead crane can move horizontally along the working span and vertically along the working span. The overhead crane scheduling model for the metallurgical storage area is simplified to intra-span operations, specifically: Given a production order plan containing N job tasks, transported by K overhead cranes, and N will continuously increase as production progresses, forming a static schedule that is linked to the production plan; Define model parameters and decision variables; Model Construction: During the slab storage process, to minimize the overhead crane's heavy-load travel distance during crane scheduling, the objective function is defined as follows: The specific model parameter definitions include: I represents the set of slabs to be stored, i = 1, 2, ..., I, where the sets of slabs corresponding to continuous casting machines No. 1 and No. 2 are I1 and I2 respectively, and I = I1 + I2. J represents the set of slab storage regions, j = 1, 2, ..., J; K represents the set of overhead cranes, numbered sequentially as k = 1, 2, ..., K, where K is the number of overhead cranes in the span, and v is the speed at which the overhead cranes transport the slab. T represents the set of scheduling time periods, t = 1, 2, ..., T; S i Represents the set of target regions for slab i; H j This indicates the maximum slab capacity in region j; and These represent the distances traveled by the overhead crane k from continuous casting machines 1 and 2 to region j, respectively. r1 and r2 represent the areas where the roller conveyors of continuous casting machines No. 1 and No. 2 are located, respectively; δ represents the safe distance between adjacent overhead cranes. The minimum safe distance means ensuring that adjacent overhead cranes do not have a collision risk during operation. M represents the coefficient of the constraint expression, set to 10000. During hoisting, the trolley on the overhead crane will automatically run synchronously with the overhead crane along the cross-zone track, that is, the running time of the trolley is included in the running time of the overhead crane.

2. The intelligent scheduling method for unmanned overhead cranes in metallurgical storage areas according to claim 1, characterized in that, The decision variables are defined as follows: x j，k，t If at time t, the crane k is located in region j, x j,k,t =1; otherwise, x j，k，t =0; y i,j,k,t If slab i is transported to area j by overhead crane k within time period t, then y i，j，k，t =1; otherwise y i，j，k，t =0.

3. The intelligent scheduling method for unmanned overhead cranes in metallurgical storage areas according to claim 1, characterized in that, The model constraints include: This means that each overhead crane must be assigned to a specific area during each time period; The requirement is that no more than one overhead crane can be allocated to each area in each time period; This means that if crane k is assigned to column j, then the cranes to its left and right must maintain a distance of column δ. This indicates that the location of the overhead crane must be consistent with its serial number; This indicates that the overhead crane carrying a heavy load must be an overhead crane that is currently in motion. This means that each slab must be hoisted to a specific location; This indicates that the number of slabs in the j-th column cannot exceed its maximum limit; This indicates that the timing of slab hoisting should be consistent with the output sequence of the continuous casting machine.

4. The intelligent scheduling method for unmanned overhead cranes in metallurgical storage areas according to claim 1, characterized in that, The number of decision variables (NoV) in the model is: NoV=J×K×T+I×J×K×T =I×(J+1)×K×T; The number of constraints (NoC) is: NoC=K×T+J×T+2(J-1)×K×(K-1)×T+2J×(K-1)×T +I×J×K×T+K×T+I+J+(I-2)J 2 ×K 2 ×T 2 =(I-2)J 2 K 2 T 2 +2(JK-K+1)(K-1)T +IJKT+2KT+I+J.

5. The intelligent scheduling method for unmanned overhead cranes in metallurgical storage areas according to claim 1, characterized in that, The basic scheduling rules include: Arbitrarily assign overhead cranes, assign the overhead crane with the shortest remaining time, or assign the overhead crane closest to the mission point; The rules for avoiding collisions between overhead lines include: The system can assign the earliest available overhead crane to give way, assign an empty overhead crane to give way, assign the overhead crane closest to the other party's target point to give way, assign the overhead crane closest to its own target point to give way, or assign an overhead crane with low priority to give way.

6. The intelligent scheduling method for unmanned overhead cranes in metallurgical storage areas according to claim 1, characterized in that, The DQN algorithm process includes: Initialize the sample set D and its capacity N, initialize the Q network, and randomly generate weight values ​​θ; Initialize target Network, weights θ _ =θ; Forepisode = 1, Mdo; Initialize the state set S; Fort = 1, Tdo; Choose a random action a with probability ε∈(0,1), or choose a = argmax. a (Q(s, a: θ)); Perform action a, and receive the reward function R and the new state s′; Store the sample (s, a, R, s′) into the sample set D; Randomly select samples (s, a, R, s′) from the minibatch of the sample set D; When the next step is the endpoint is y i =R, otherwise y i =R+γmax a′ Q(s, a′; θ) - ); For (y) i -Q(s, a: θ)) 2 θ is updated using gradient descent; Update the target every C steps Network, θ _ =θ.

Images