An intelligent scheduling data processing method for quay crane operation queues

Abstract

The present invention discloses an intelligent scheduling data processing method for quay crane operation queues, belonging to the technical field of intelligent scheduling. Step 1: Prepare data, process data including ship information, equipment information, queue information and rule configuration, and then set constraint conditions, including preset limit conditions and target constraint conditions; Step 2: Build a CWP algorithm framework; Step 3: Initialize parameters, perform data preprocessing, initialize the parameters and variables required by the algorithm, and execute scheduling; Step 4: Construct a neural network, establish a neural network model for reinforcement learning, and realize the progressive extraction of data from the input layer, processing layer, feature fusion layer, decision layer to the output layer; Step 5: Train the network and update parameters. It solves the defects of traditional scheduling algorithms, realizes diversified scheduling methods, realizes the global optimization of intelligent scheduling, and provides a more flexible, efficient and intelligent job scheduling solution.

Description

Technical Field

[0001] The present invention relates to the technical field of intelligent scheduling, and in particular to a data processing method for intelligent scheduling of a quay crane operation queue. Background Art

[0002] Intelligent dispatching of port terminals will use the achievements of Internet of Things technology, artificial intelligence, etc., combined with real-time data on the status of machinery and equipment on the production site, berths, vehicle transportation, etc., and interact with the ship's driving status in real time. Through prediction, the best decision-making plan can be formulated to achieve seamless connection between information system instructions and terminal machinery and equipment control functions, improve operational efficiency and accuracy, ensure the continuity, coordination, balance and economic operation of the production process, and maximize production benefits. This is the future development trend.

[0003] In view of the above-mentioned related technologies, the applicant found that in the current intelligent scheduling algorithm for terminals, the scheduling strategy based on the principle of shortest distance is generally adopted. However, with the continuous emergence of complex factors in practical applications and the growing demand for efficient operation, traditional scheduling algorithms have gradually exposed their inherent defects, that is, the scheduling method is single, mainly relying on distance optimization, ignoring factors such as the actual operation progress of the terminal and the urgency of different operation lines. These algorithms may balance factors such as distance and time to a certain extent, but there may be problems of local optimality rather than global optimality. Summary of the invention

[0004] In view of the deficiencies in the prior art, the present invention provides a data processing method for intelligent scheduling of quay crane operation queues, which solves the defects of traditional scheduling algorithms, realizes diversified scheduling methods, achieves global optimization of intelligent scheduling, and provides a more flexible, efficient and intelligent operation scheduling solution.

[0005] The technical solution of the present invention to solve the above technical problems is as follows:

[0006] A method for processing data of intelligent scheduling of a quay crane operation queue comprises the following steps:

[0007] Step 1: Prepare data, process data including ship information, equipment information, queue information and rule configuration, and then set constraints, including preset constraints and target constraints;

[0008] Step 2: Build the CWP algorithm framework and input the data into it. The general process of the CWP algorithm framework is data preprocessing. Divide different priority groups for each job queue according to the bay position, loading and unloading type, and in-cabin position information. Traverse the job queues in each group for allocation. The allocation strategy of the queue is determined by the neural network. During the allocation, whenever a quay crane completes the current queue and is in an idle state, a new task needs to be assigned to it. After the assignment, the quay crane information and the status of the surrounding workable queues at the current moment change. Input the updated new quay crane and queue status into the neural network to solve for the best allocable queue until all the queues in all groups are allocated;

[0009] Step 3: Initialize the parameters, perform data preprocessing, initialize the parameters and variables required by the algorithm, and execute the scheduling;

[0010] Step 4: Build a neural network, establish a neural network model for reinforcement learning, and realize the progressive extraction of data from the input layer, processing layer, feature fusion layer, decision layer to the output layer;

[0011] Step 5: Train the network and update the parameters. Adopt the method of allocating first and then training. In each training, first call the unupdated neural network to execute the CWP algorithm to provide an allocation and scheduling plan for the quay crane and the queue. After obtaining the output results of the current neural network under different inputs, calculate the reward values and evaluation values corresponding to different quay crane queue states at each moment as the basis for policy update.

[0012] Further, in Step 1,

[0013] The ship information is the ship information that needs to be loaded and unloaded given by TOS, including the ship name, ship length, side docking method, total number of bays, positions of the bow and stern of the ship, distances of the bow and stern, and estimated start and end times;

[0014] The equipment information is the information of each working line and its affiliated quay crane, including the serial number, current position, working range, moving speed, single and double hook efficiencies of loading and unloading the ship, and the limit safety distance;

[0015] The queue information includes the queue primary key, queue name, ship voyage number, number on the ship, upper and lower hold marks, operation type, and number of MOVE hooks;

[0016] The rule configuration is other rule conditions that need to be uniformly and specially restricted, including the operation time for loading and unloading one hatch cover on average and the time for the quay crane to pass by the bridge.

[0017] Further, in Step 1, the preset restriction conditions are:

[0018] Condition a, maintain a safe distance between any quay cranes, and it is not allowed for a quay crane to cross other quay cranes to reach a certain bay to prevent crossing:

[0019]

[0020] Condition b: The sum of the distances moved by each quay crane during loading and unloading is minimized as much as possible;

[0021] Condition c: The average efficiency of all quay cranes per hour of operation is as equal as possible;

[0022] The objective constraint conditions are:

[0023] Condition d: Minimize the completion time of the ship:

[0024]

[0025] Condition e: Minimize the difference between the completion times of each quay crane;

[0026]

[0027] where n is the number of quay cranes.

[0028] Furthermore, in step two,

[0029] If a solution is found:

[0030] Execute the allocation result, assign queues to the corresponding quay cranes, update the information of the quay cranes, and update the queue information;

[0031] If no solution is found:

[0032] Place the current idle quay crane in a waiting state until there is an idle quay crane that has completed its queue to execute the scheduling allocation together with it, and update the completion queue times of all quay cranes;

[0033] If the queue allocation in the current grouping is completed, wait until all quay cranes are idle and process the scheduling of the queues in the next grouping.

[0034] Furthermore, in step three,

[0035] Initialize the parameters and variables required by the algorithm as:

[0036] Ship: Calculate the information of the bays to which the operation queues belong based on the berthing method and the relevant information of the bow and stern of the ship;

[0037] Operation line: Determine the range of bays where the quay crane can operate on the ship according to the starting position of the operation range of the operation line. At the same time, randomly assign initial bays, uniformly convert the single and double hook efficiencies of loading and unloading the ship into the time required for one move, and convert the moving speed of the quay crane into the time required to pass through one bay;

[0038] Job queue: Based on the bay where the containers awaiting operation under the hatch are located, add job queues for opening and closing the hatch covers to the queue list. Assign priorities to each queue according to the general principle of first handling the upper hatch and then the lower hatch, and first unloading and then loading. Group different queues with priority as the primary keyword and bay data as the secondary keyword.

[0039] Furthermore, in step three, estimate the execution time of each queue in advance. The scheduling space can be fitted into a two-dimensional space with the bay position and the time axis as dimensions.

[0040] Furthermore, in step three,

[0041] Construct a CWP algorithm framework, including the following steps:

[0042] Step a, first traverse the job queues in each group until the current group is empty;

[0043] Step b, then find all the quay cranes with the earliest completion queue time, i.e., in the idle state, and calculate the bay range that can be operated based on the current bay;

[0044] Step c, find the job queues available for allocation according to the bay range of the idle quay crane;

[0045] Step d, represent the job line information and the current environment queue information as tensors, call the neural network model, and generate a scheduling plan;

[0046] Step e, if the neural network finds a solution, traverse each queue in the solution and allocate it to the corresponding quay crane;

[0047] Step f, if it involves moving the quay crane to a new bay, generate a moving queue at the same time, and update the status of the quay crane, including the current bay and the completion time. Then assign the task to the quay crane, generate a job queue, and update the status of the quay crane. Finally, delete the job queues that have been allocated in the group;

[0048] Step g, if the neural network does not find a solution, wait until a new quay crane is idle, and repeat steps d - g;

[0049] Step h, update the completion time of all quay cranes;

[0050] Step i, repeat the above steps until all the job queues in all groups are allocated.

[0051] Furthermore, in step four, the input layer includes quay crane information and environment and queue information;

[0052] The processing layer includes a fully connected feature encoder and a recurrent neural network encoder;

[0053] The information fusion layer includes a splicing and attention mechanism plus a feature extraction layer.

[0054] Furthermore, in step five, in terms of the evaluation strategy, for the short-term return of the allocation result, during each process of the model calling the allocation queue, the input of the algorithm is the state of the quay crane and the current environmental queue state information, and the output is the selected queue information. The evaluation is carried out by comprehensively considering the current bay position of the model, the bay position of the target queue, and the MOVE number. For the future return of the model allocation, after calculating the allocation plan for all queues each time, the future return of the allocation plan can be comprehensively evaluated based on the gap between the estimated completion time of the ship and the end time of all quay cranes, the total moving distance of all quay cranes, and the total waiting time.

[0055] In summary, compared with the prior art, the beneficial effects of the above technical solutions are as follows:

[0056] (1) By combining various factors such as real-time position information, workable range, requirements of the operation line, and busyness, the intelligent scheduling of the operation queue instructions issued by the Terminal Operating System (hereinafter referred to as TOS) and the quay crane equipment is realized reasonably and efficiently. This algorithm can not only provide a reasonable and efficient queue scheduling plan, effectively reduce equipment congestion and idle waiting situations, but also dynamically adjust the operation queue on the premise of observing a series of constraints. Even when new trailers are added or position changes occur during the execution process, it can also respond quickly to achieve more reasonable resource allocation;

[0057] (2) The present invention realizes the mutual independence and non-interference among the working quay cranes, ensuring that each quay crane can execute the queue safely and efficiently even in a complex operation environment, greatly improving the stability and reliability of the system;

[0058] (3) The present invention provides a general CWP algorithm framework, quantifying the time dimension of the queue into the height in a two-dimensional space, and other algorithms can also be applied in this framework to calculate efficient and feasible scheduling plans;

[0059] (4) The present invention can not only provide the operation scheduling plan of the quay crane in advance, but also respond quickly even when new quay cranes are added or resource changes occur in the time and space dimensions during the operation process to achieve more reasonable resource allocation;

[0060] (5) An intelligent scheduling algorithm for job queues is designed by introducing a deep reinforcement learning algorithm. The scheduling plan provided by the algorithm optimizes the problem of low utilization rate of quay crane resources, reduces traffic congestion in a certain operation area, makes the scheduling process more reasonable and efficient. At the same time, the algorithm quickly responds to idle quay crane resources, balances the progress of each job queue, not only improves the operation efficiency, but also reduces the workload of operators, saves the terminal cost. In addition, the process of allocating queues also reduces the ineffective waiting time of quay cranes and minimizes the latest end time between quay cranes, thus significantly improving the overall operation efficiency. Description of the Drawings

[0061] Figure 1 It is a schematic diagram of the intelligent scheduling logic in the embodiment of the present invention;

[0062] Figure 2 It is a schematic diagram of the neural network structure in the embodiment of the present invention;

[0063] Figure 3 It is a visualization allocation result diagram of 6 quay cranes on the ship in the embodiment of the present invention. Specific Embodiments

[0064] The principles and features of the present invention will be described below in conjunction with all the drawings. The examples given are only used to explain the present invention and are not intended to limit the scope of the present invention.

[0065] The embodiment of the present invention discloses an intelligent scheduling data processing method for quay crane job queues.

[0066] The present invention mainly relates to a quay crane workqueue planning (hereinafter referred to as CWP) algorithm applied to ports and terminals in the field of logistics scheduling. In the current scheduling algorithms, the scheduling strategy based on the principle of the shortest distance is generally adopted. However, with the continuous emergence of complex factors in practical applications and the increasing demand for efficient operation, the traditional scheduling algorithms gradually expose their inherent defects as follows:

[0067] First, the scheduling method is single: mainly relying on distance optimization, ignoring factors such as the actual operation progress of the terminal and the urgency of different operation lines; these methods may balance factors such as distance and time to a certain extent, but there may be problems of local optimum rather than global optimum.

[0068] Second, the utilization rate of quay cranes is poor: the scheduling considering only a single factor often leads to a large area of congestion of some quay cranes in certain areas, while there is a shortage of quay cranes in the operations in other areas, which not only affects the utilization rate of equipment, but also greatly reduces the overall scheduling effect.

[0069] III. Lack of dynamic adjustment: Once the scheduling plan is formulated, it is difficult to adjust. It can only execute tasks according to the pre-allocated quay crane resources and operation queues. When new resources are added midway, the scheduling algorithm cannot dynamically adjust the allocation, easily resulting in problems such as unreasonable resource allocation and slow response speed.

[0070] IV. Dynamic bay allocation: Traditional algorithms often can only operate within the pre-planned bay range and cannot dynamically adjust the bay according to conditions such as the urgency of actual operation requirements during the execution process. In a complex and changeable operation environment, this limitation is more obvious.

[0071] The present invention aims to provide a CWP algorithm based on deep reinforcement learning, which realizes the reasonable and efficient intelligent scheduling of the operation queue instructions issued by the TOS and quay crane equipment by combining various factors such as real-time position information, operable range, requirements of the operation line, and busyness degree. It can not only provide a reasonable and efficient queue scheduling plan, effectively reduce equipment congestion and idle waiting situations, but also dynamically adjust the operation queue on the premise of observing a series of constraint conditions. Even when new trucks are added or position changes occur during the execution process, it can quickly respond and achieve more reasonable resource allocation.

[0072] In short, the present invention provides a more flexible, efficient, and intelligent operation scheduling solution, which helps to improve the overall operation efficiency of ports and terminals, reduce operating costs, and improve service quality.

[0073] An intelligent scheduling data processing method for quay crane operation queues. The idea of the present invention is mainly to assign the to-be-operated instructions issued by the TOS to different quay cranes to guide the quay cranes to automatically load and unload containers on the ship. During the scheduling process, with the goal of minimizing the estimated completion time of the ship and minimizing the gap between the end times of all quay cranes as the target constraints, combined with information such as the current position of the quay crane and the matching degree between the quay crane and the operation position, on the premise of strictly observing a series of preset constraint conditions, the intelligent algorithm automatically calculates the optimal allocation relationship between the quay crane and the queue through comprehensive optimization and allocation.

[0074] Reference Figure 1 , an intelligent scheduling data processing method for quay crane operation queues. The method of the present invention is as follows, including the following steps:

[0075] Step 1: Prepare data

[0076] Process data including ship information, equipment information, queue information, and rule configuration, and then set constraint conditions, including preset limit conditions and target constraint conditions;

[0077] (1) Data processing

[0078] The ship information is the ship information for loading and unloading operations given by TOS, including the ship name, ship length, side docking method, total number of bays, positions of the bow and stern of the ship, distance between the bow and stern, estimated start and end times, etc. The content of Table 1 below can be referred to:

[0079] Table 1 Comparison Table of Variable Information for Ship Voyages

[0080] Data Object Remarks vslVisitGkey Primary Key of Vessel Voyage vslId Vessel Code / ID vslName Vessel Name vslLoa Overall Length, Length Over All bayNum Total Number of Bays berthingSide Berthing Side, PS: Port Side Berthing; SS: Starboard Side Berthing bowPos Bow Position sternPos Stern Position bowToBay Distance from Bow to the First Bay sternToBay Distance from Stern to the Last Bay bowToBridge Number of Bays in Front of Bridge 1 sternToBridge Number of Bays in Front of Bridge 2 visitPhase Vessel Berthing Status etbTime Estimated Berthing Time etwTime Estimated Starting Time etcTime Estimated Completion Time etdTime Estimated Departure Time rtbTime Actual Berthing Time

[0081] The equipment information is the information of each operating line and the affiliated quay crane, including the serial number, current position, operating range, moving speed, single and double hook efficiencies for loading and unloading ships, limit safety distance, etc. The content of Table 2 below can be referred to:

[0082] Table 2 Comparison Table of Variable Information for Quay Crane Operating Lines

[0083] Data Object Remarks lineGkey Primary Key of Operation Line vslVisitGkey Primary Key of Vessel Voyage powId Operation Point, i.e., Quayside Crane Number linearSeq Sequence Number of Quayside Crane on the Quayside Line, EQP_SET startPos Starting Position of Quayside Crane Operation Range, Unit: Meter endPos Ending Position of Quayside Crane Operation Range, Unit: Meter avgGantryVelocity Average Gantry Travel Speed (m / min) vslJobSpace Operation Interval - Liner Operation Interval (BAY) eqpWidth Width of Quayside Equipment (m) deadSafeDist Ultimate Safety Distance DISTANCE etwTime Estimated Start Time etcTime Estimated End Time prodt11 Desired Efficiency, Unloading - Single Hook prodt12 Desired Efficiency, Unloading - Double Hook prodt21 Desired Efficiency, Loading - Single Hook prodt21 Desired Efficiency, Loading - Double Hook

[0084] The queue information includes the queue primary key, queue name, corresponding ship voyages, numbers on the ship, upper and lower hold marks, operation types, number of MOVE hooks, etc. The content of Table 3 below can be referred to:

[0085] Table 3 Comparison Table of Variable Information for Operating Queues

[0086] Data Representation Remarks queueGkey Primary Key of Operation Queue queueName Operation Queue Name lineGkey Primary Key of Operation Line vslVisitGkey Primary Key of Vessel Voyage moveKind Task Type, Values: DSCH, LOAD vslBayNbr Vessel Bay Number deckHold Deck / Hold Indicator DECK / HOLD moveQty Number of MOVEs powId Operation Point, i.e., Quayside Crane Number queueSeq Queue Sequence

[0087] The rule configuration is other rule conditions that require unified special restrictions, including the operation time for loading and unloading one hatch cover on average, the time for the quay crane to pass by the bridge, etc.

[0088] (2)Constraint Conditions

[0089] The preset limitation conditions are as follows:

[0090] Condition a, maintain a safe distance between any quay cranes, and it is not allowed for a quay crane to cross other quay cranes to reach a certain bay to prevent crossing:

[0091]

[0092] Condition b, the sum of the distances moved by each quay crane during the loading and unloading process is minimized as much as possible;

[0093] Condition c, the average efficiency of all quay cranes per hour is as the same as possible;

[0094] The target constraint condition is:

[0095] Condition d, minimize the completion time of the ship:

[0096]

[0097] Condition e, minimizing the difference between the completion times of each quay crane:

[0098]

[0099] where n is the number of quay cranes.

[0100] Step 2: Build the CWP algorithm framework

[0101] Build the CWP algorithm framework and input the data into it. The general process of the CWP algorithm framework is data preprocessing. Divide different priority groups for each job queue according to information such as bay position, loading and unloading type, and in-cargo position. Traverse the job queues in each group for allocation. The allocation strategy of the queue is decided by the neural network. Whenever a quay crane finishes the current queue and is in an idle state during the allocation, a new task needs to be assigned to it. After the assignment, the information of the quay crane at the current moment and the status of the surrounding workable queues change. Input the updated new quay crane and queue status into the neural network to solve for the best allocable queue until all the queues in all groups are allocated.

[0102] Input data: ship information, equipment information, queue information, and rule configuration

[0103] Algorithm process:

[0104] Data preprocessing

[0105] Divide different priority groups for the queues

[0106] Traverse the job queues in each group:

[0107] Find all the idle quay cranes with the earliest completion queue time

[0108] Update the status information of the idle quay cranes and the surrounding workable queues

[0109] Call the neural network, input the quay crane and queue data, and output the solution:

[0110] If a solution is found:

[0111] Execute the allocation result and assign the queue to the corresponding quay crane

[0112] Update the information of the quay crane

[0113] Update the queue information

[0114] If no solution is found:

[0115] Put the current idle quay crane in a waiting state until there is another idle quay crane that has completed a queue to perform the scheduling allocation with it

[0116] Update the completion queue times of all quay cranes

[0117] If the queue allocation in the current group is completed, wait until all quay cranes are idle, process the queue scheduling in the next group, and at the same time reset the values of certain algorithm parameters.

[0118] Output data: The queue allocation list corresponding to each quay crane.

[0119] Step 3: Initialize parameters

[0120] Perform data preprocessing, initialize the parameters and variables required by the algorithm, and execute the scheduling

[0121] (1) Perform data preprocessing

[0122] Initialize the parameters and variables required by the algorithm

[0123] Vessel: Calculate the information of the bay to which the operation queue belongs according to the berthing method and the relevant information of the bow and stern of the ship;

[0124] Operation line: Determine the bay range of the vessels that the quay crane can operate according to the starting position of the operation range of the operation line. At the same time, randomly allocate the initial bay, uniformly convert the single and double hook efficiencies of loading and unloading the ship into the time required for one MOVE (unit: second), and convert the moving speed of the quay crane into the time required to pass through one bay (unit: second).

[0125] Operation queue: Based on the bay where the containers waiting for operation under the hatch are located, add the operation queues that need to open and close the hatch covers to the queue list, and assign priorities to each queue according to the general principles of first above the hatch and then under the hatch, and first unloading and then loading. Group different queues with the priority as the main keyword and other data such as the bay as the secondary keyword. Since there is a certain timing in the process of the quay crane executing the queue, this brings many unpredictable complex situations to the scheduling algorithm. Therefore, it is also necessary to estimate the execution time of each queue in advance. The scheduling space can be fitted into a two-dimensional space with the bay position and the time axis as the dimensions respectively (refer to Figure 3 as shown, the length of the column is the time cost of the corresponding queue).

[0126] (2) Construct the CWP algorithm framework

[0127] Including the following steps:

[0128] Step a, first traverse the operation queues in each group until the current group is empty;

[0129] Step b, then find all the quay cranes with the earliest completion queue time, that is, in the idle state, and calculate the bay range that can be operated based on the current bay;

[0130] Step c, find the operation queues that can be used for allocation according to the bay range of the idle quay crane;

[0131] Step d: Represent the operation line information and information such as the current environment queue as tensors, call the neural network model, and generate a scheduling plan;

[0132] Step e: If the neural network finds a solution, traverse each queue in the solution and assign it to the corresponding quay crane;

[0133] Step f: If it involves moving the quay crane to a new bay, generate a moving queue at the same time, and update the status of the quay crane, including the current bay and the completion time. Then assign the task to the quay crane, generate an operation queue, and update the status of the quay crane. Finally, delete the operation queue that has been assigned in the grouping;

[0134] Step g: If the neural network does not find a solution, wait until a new quay crane is idle, and repeat steps d - g;

[0135] Step h: Update the completion time of all quay cranes;

[0136] Step i: Repeat the above steps until the operation queues of all groupings are assigned.

[0137] Step 4: Construct a neural network

[0138] Construct a neural network, establish a neural network model for reinforcement learning (for reference, see Figure 2 ). The network layer includes an input layer, a processing layer, a feature fusion layer, a decision layer, and an output layer. The composition and feature representation of each network layer are as follows:

[0139] (1) The input layer includes quay crane information and environment and queue information

[0140] Quay crane information: Assume there are N quay cranes (i.e., N operation lines), and the status of each quay crane is represented by a dimensional tensor, then the size of the quay crane tensor is .

[0141] Environment and queue information: Assume that the environment information and the executable queue information can be encoded into a dimensional vector, and the length of this vector can adapt to different amounts of environment information input, but the width is fixed. Therefore, the size of this input tensor is , where M is variable.

[0142] (2) The processing layer includes a fully connected feature encoder and a recurrent neural network encoder

[0143] Fully-connected Feature Encoder: The encoder receives both the quay crane state tensor and an integer i simultaneously. Here, i represents that this is the input of the i-th quay crane. A mask matrix is used for the quay crane tensor to highlight the features of the i-th quay crane while weakening the features of other quay cranes. Then, one or more stacked fully-connected layers are used to process the quay crane state tensor to extract the features of each quay crane. The size of the output tensor remains , where is the encoded feature dimension.

[0144] Recurrent Neural Network Encoder: For the second input tensor, a recurrent neural network (such as LSTM or GRU) is used for processing because its length is variable. This can capture the patterns in the sequence. The output tensor is the hidden state of the last time step with a size of , or if more context information needs to be retained, the states of all time steps can be output, with a size of .

[0145] (3) The information fusion layer includes concatenation, attention mechanism, and a feature extraction layer

[0146] Combine the outputs of the above two encoders in some way. One method is to concatenate the feature vectors of each quay crane with the environmental queue vector to form a new tensor , or use the multi-head attention mechanism to allow each quay crane to dynamically adjust its state representation according to the environmental information.

[0147] (4) Decision-making layer

[0148] After information fusion, further feature extraction is performed through a series of fully-connected layers, and finally, an action probability distribution is output for the quay crane. Since the action space is discrete, a softmax layer is used to generate the probability distribution.

[0149] (5) Output layer

[0150] The size of the final output tensor is , where represents the possibility of selecting each action space. For the current quay crane, the output tensor contains a probability distribution over all possible queues.

[0151] Step Five: Train the network and update the parameters

[0152] Due to the great uncertainty in predicting future rewards based on the current state in each step of allocation, a method of allocating first and then training is adopted. In each training, the unupdated neural network is first called to execute the CWP algorithm to provide an allocation and scheduling plan for quay cranes and queues. After obtaining the output results of the current neural network under different inputs, the reward values and evaluation values corresponding to different quay crane queue states at each moment are calculated as the basis for policy update.

[0153] In terms of evaluating the policy, for the short-term return of the allocation result, in the process of the algorithm calling and allocating the queue each time, the input of the algorithm is the state of the quay crane and the current environmental queue state information, and the output is the selected queue information. The model's current bay position, the bay position of the target queue, and data such as the MOVE number are comprehensively considered through a certain form of metric for evaluation. For the future return of the model allocation, after calculating the allocation plans of all queues each time, the future return of the allocation plan can be comprehensively evaluated based on the gap between the estimated completion time of the ship and the end time of all quay cranes, the total moving distance of all quay cranes, the total waiting time, etc.

[0154] Therefore, the policy update adopts the method of allocating first and then training. First, the unupdated neural network is called to execute the algorithm to generate a complete allocation and scheduling plan for quay cranes and queues. After obtaining the output results of the current neural network under different inputs, various evaluation values are calculated for the output results as the basis for policy update.

[0155] Reference Figure 3 shows the scheduling results calculated by the algorithm for 6 quay cranes on a certain ship. Different colors represent different quay cranes, and the blank in the middle of the rectangular box represents the waiting process.

[0156] The implementation effect of an intelligent scheduling data processing method for quay crane operation queues in an embodiment of the present invention is as follows:

[0157] By introducing a deep reinforcement learning algorithm, an intelligent scheduling method for operation queues is designed. The provided scheduling plan optimizes the problem of low utilization rate of quay crane resources, reduces traffic congestion in a certain operation area, and makes the scheduling process more reasonable and efficient. At the same time, the algorithm quickly responds to idle quay crane resources, balances the progress of each operation queue, not only improves the operation efficiency, but also reduces the workload of operators and saves terminal costs. In addition, the process of allocating queues also reduces the ineffective waiting time of quay cranes and minimizes the latest end time between quay cranes, thus significantly improving the overall operation efficiency.

[0158] The above are only the preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for intelligent scheduling data processing of quay crane operation queues, characterized in that: The steps include: Step 1: Prepare data, process data including ship information, equipment information, queue information and rule configuration, and then set constraints, including preset constraints and target constraints; Step 2: Build the CWP algorithm framework and input the data into the CWP algorithm framework. The general process of the CWP algorithm framework is data preprocessing. Each job queue is divided into groups of different priorities according to the bay position, loading and unloading type, and position information in the cabin. The job queues of each group are traversed for allocation. The queue allocation strategy is determined by the neural network. During the allocation, whenever a quay crane completes the current queue, that is, it is in an idle state, it needs to be assigned a new task. After the allocation, the current quay crane information and the surrounding operational queue status change accordingly. The updated new quay crane and queue status are input into the neural network to solve the best allocable queue until all grouped queues are allocated. Step 3: Initialize parameters, perform data preprocessing, initialize parameters and variables required by the algorithm, and execute scheduling; The execution time of each queue is estimated in advance, and the scheduling space is fitted into a two-dimensional space with bay position and time axis as dimensions; Step 4: Construct a neural network and establish a neural network model for reinforcement learning to achieve progressive data extraction from the input layer, processing layer, feature fusion layer, decision layer to the output layer; The input layer includes quay crane information and environment and queue information; The processing layers include a fully connected feature encoder and a recurrent neural network encoder; Fully connected feature encoder: The encoder receives the shore bridge state tensor and an integer i at the same time, i means that this is the input of the i-th shore bridge. The shore bridge tensor uses a mask matrix to highlight the features of the i-th shore bridge and weaken the features of other shore bridges. Then one or more stacked fully connected layers are used to process the shore bridge state tensor to extract the features of each shore bridge. The size of the output tensor is still ,in is the feature dimension after encoding; Recurrent Neural Network Encoder: For the second input tensor, a recurrent neural network is used to process it because its length is variable, so as to capture the pattern in the sequence. The output tensor is of size The hidden state of the last time step of , or if more context information needs to be retained, the state of all time steps is output, with a size of ; The information fusion layer includes concatenation and attention mechanism plus feature extraction layer; Step 5: Train the network and update the parameters. The method of allocating first and then training is adopted. In each training, the unupdated neural network is first called to execute the CWP algorithm to provide an allocation and scheduling plan for the quay crane and the queue. After obtaining the output results of the current neural network under different inputs, the reward values ​​and evaluation values ​​corresponding to the different quay crane queue states at each time are calculated as the basis for strategy updating.

2. The method for intelligent scheduling data processing of a quay crane operation queue according to claim 1, characterized in that: In step one, The ship information is the ship information required for loading and unloading operations given by TOS, including ship name, hull length, side-to-side approach, total number of shellfish, bow and stern positions, bow-to-stern distance, and estimated start and end time; Equipment information is the information of each operation line and its affiliated quay crane, including sequence number, current position, operation range, moving speed, single and double hook efficiency of loading and unloading ships, and maximum safety distance; The queue information includes the queue primary key, queue name, ship number, ship number, upper and lower cabin marks, operation type, and MOVE hook number; The rules are configured for other rule conditions that require unified special restrictions, including the average operation time for loading and unloading a hatch cover and the time it takes for the quay crane to pass the bridge.

3. The method for intelligent scheduling data processing of quay crane operation queue according to claim 2 is characterized in that: In step 1, the preset constraints are: Condition a: Keep a safe distance between any quay cranes, and do not allow quay cranes to cross other quay cranes to reach a certain bay to prevent crossing: Condition b: the sum of the distances moved by each quay crane during loading and unloading is as small as possible; Condition c: the average efficiency of all quay cranes per hour is as close as possible to the same; The target constraints are: Condition d, minimize the ship's completion time: Condition e, minimize the difference between the completion time of each quay crane: Where n is the number of quay cranes.

4. The method for intelligent scheduling data processing of a quay crane operation queue according to claim 1, characterized in that: In step 2, If you find a solution: Execute the allocation results, allocate queues to the quay cranes, update the quay crane information, and update the queue information; If no solution is found: Put the current idle quay crane into a waiting state until there is an idle quay crane in the next completion queue to perform scheduling allocation together with it, and update the completion queue time of all quay cranes; If the queues in the current group are allocated, wait until all the quay cranes are idle and process the scheduling of the queues in the next group.

5. The method for intelligent scheduling data processing of quay crane operation queue according to claim 1 is characterized by: In step three, The parameters and variables required to initialize the algorithm are: Ship: Calculate the information of the bay position to which the operation queue belongs based on the side-to-side manner and the relevant information of the bow and stern; Operation line: According to the starting position of the operation range of the operation line, the range of bays where the quay crane can operate the ship is determined, and the initial bay is randomly assigned. The single and double hook efficiency of the loading and unloading ship is uniformly converted into the time required for a move, and the moving speed of the quay crane is converted into the time required to pass one bay; Operation queue: According to the bay position of the container to be operated under the cabin, add the operation queue that needs to open and close the hatch cover to the queue list, assign priority to each queue according to the general principle of first loading and then unloading, first unloading and then loading, and group different queues with priority as the main keyword and bay position data as the secondary keyword.

6. The method for intelligent scheduling data processing of quay crane operation queue according to claim 1 is characterized by: In step 2, Building the CWP algorithm framework includes the following steps: Step a, first traverse the job queue in each group until the current group is empty; Step b, then searching for all quay cranes with the earliest completion queue time, i.e., in an idle state, and calculating the range of bay positions that can be operated based on the current bay positions; Step c, searching for a job queue available for allocation according to the bay position range of the idle quay crane; Step d, representing the job line information and the current environment queue information as tensors, calling the neural network model, and generating a scheduling plan; Step e: If the neural network finds a solution, it will traverse each queue in the solution and assign it to the corresponding quay crane; Step f, if it involves moving the quay crane to a new bay, a moving queue is generated at the same time, and the status of the quay crane is updated, including the current bay position and completion time, and then the task is assigned to the quay crane, a job queue is generated, and the status of the quay crane is updated, and finally the assigned job queue in the group is deleted; Step g, if the neural network does not find a solution, wait until a new quay crane is free and repeat step dg; Step h, updating the completion time of all quay cranes; Step i: Repeat the above steps until all the grouped job queues are assigned.

7. The method for intelligent scheduling data processing of quay crane operation queue according to claim 1 is characterized by: In step five, in terms of the evaluation strategy, for the short-term return of the allocation result, in the process of each model calling the allocation queue, the input of the algorithm is the status of the quay crane and the current environment queue status information, and the output is the selected queue information. The current bay position of the model, the bay position of the target queue and the number of MOVEs are comprehensively considered for evaluation. For the future return of the model allocation, after each calculation of the allocation plan of all queues, the future return of the allocation plan is comprehensively evaluated based on the gap between the estimated completion time of the ship and the end time of all quay cranes, the sum of the moving distances of all quay cranes, and the sum of waiting time.

Images