A port horizontal transportation equipment real-time scheduling method and system based on a MIMO mechanism

Abstract

The application discloses a kind of port horizontal transport equipment real-time scheduling method and system based on MIMO mechanism, comprising: obtaining the operating parameters in terminal operation environment and equipment, encapsulate port loading and unloading operation and horizontal transport operation as MIMO interface;Based on MIMO interface, by defining input and output interface and objective function, the operation mechanism of port operation and its mathematical model are established;Based on MIMO interface and mathematical model, by constructing road network and physical facilities, the port simulation environment is built;In simulation environment, reinforcement learning algorithm configuration is configured, and the optimal scheduling strategy of port horizontal transport equipment is obtained.The application combines the modeling framework under MIMO mechanism and the reinforcement learning algorithm configuration based on simulation, interacts simulation environment and scheduling system, so that container task and horizontal transport equipment are matched in real time.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of intelligent logistics and smart port, and particularly relates to a port horizontal transportation equipment real-time scheduling method and system based on a MIMO mechanism. BACKGROUND

[0002] With the intensification of globalization, the importance of sea transportation is increasingly prominent, becoming the backbone of international trade and regional exchanges. As a transfer point between sea and land transportation, the port plays a key role in determining the efficiency of the entire logistics supply chain. Generally, the main equipment of an automated port includes unloading equipment such as shore-based cranes (hereinafter referred to as shore cranes) and yard cranes (hereinafter referred to as yard cranes), and horizontal transportation equipment (such as AGV, ART, IGV, etc.). Among them, horizontal transportation equipment is the key transportation equipment connecting shore cranes and yards, and its scheduling efficiency directly affects the port throughput and operating cost, so there are many studies on the scheduling of horizontal transportation equipment.

[0003] For the scheduling problem of horizontal transportation equipment, the industry mainly adopts heuristic scheduling rules, and the academia usually adopts the method of modeling and solving by precise algorithm or intelligent optimization algorithm. The heuristic scheduling rules adopted by the port industry mainly include first-come-first-served, shortest path priority, etc. These rules are simple to implement, easy to manage, but have poor optimization performance; while the modeling and precise or intelligent optimization algorithm solving method commonly used by the academic circle also faces the dilemma of long calculation time, which is difficult to meet the real-time requirements (usually several minutes to several hours). These methods cannot adapt to dynamic task requests and sudden disturbances (such as horizontal transportation equipment failure, path congestion) due to high computational complexity when facing dynamic environments, and cannot dynamically adjust the strategy according to the real-time state (such as horizontal transportation equipment position, task queue, energy consumption), leading to horizontal transportation congestion, long waiting time, etc., which cannot real-time optimize the scheduling strategy. Therefore, it is difficult to meet the real-time and robustness requirements of large-scale port scenarios.

[0004] In order to solve the above problems, dynamic scheduling methods are increasingly attracting attention. Among the many dynamic scheduling methods, reactive scheduling is the most suitable for highly complex and dynamic scenarios. Reactive scheduling refers to the system's ability to respond to unexpected uncertainties in real time to generate or modify scheduling plans, including predictive reactive scheduling and complete reactive scheduling. Although it can respond to real-time state, it lacks forward-looking optimization and is prone to local optimum.

[0005] Reinforcement learning technology is a research hotspot in recent years, and the idea of complete reactive scheduling and real-time scheduling technology based on reinforcement learning can be organically combined. For example, the reinforcement learning method can better adapt to different environmental states, and the scheduling system interacts with the automated port environment, continuously learns, takes the best action according to the strategy, considers forward-looking optimization, and finally forms a strategy based on state action, which is more suitable for this scene with high real-time requirements.

[0006] Although some reinforcement learning algorithms show potential in dynamic scheduling, their direct application in complex ports faces three major challenges. First, the state space dimension explosion caused by multi-node coupling, second, the difficulty of reward function design under multi-objective conflict. Third, it is difficult to combine the simulation for the port, and the existing simulation does not give its model a real physical meaning (or simplifies many details), mainly showing the function of algorithm principle, and it is difficult to simulate the linkage of various devices. SUMMARY

[0007] In view of the above existing problems, the present application is proposed.

[0008] Therefore, the present application provides a port horizontal transportation equipment real-time scheduling method and system based on MIMO mechanism to solve the problem of poor dynamic environment adaptability and inability to respond to task changes in real time in the traditional scheduling process of port horizontal transportation equipment.

[0009] To solve the above technical problems, the present application provides the following technical solutions:

[0010] In a first aspect, the present application provides a port horizontal transportation equipment real-time scheduling based on MIMO mechanism, comprising:

[0011] Obtain the operation parameters in the terminal operation environment and the equipment, and encapsulate the port loading and unloading operation and the horizontal transportation operation as a MIMO interface;

[0012] Based on the MIMO interface, define the input and output interfaces and the objective function, and establish the operation mechanism and the mathematical model of the port operation;

[0013] Based on the MIMO interface and the mathematical model, construct the road network and the physical facilities, and build the port simulation environment;

[0014] Configure the reinforcement learning algorithm in the simulation environment, and obtain the optimal scheduling strategy of the port horizontal transportation equipment.

[0015] As a preferred scheme of the port horizontal transportation equipment real-time scheduling method based on MIMO mechanism, wherein:

[0016] The operation mechanism and the mathematical model of the port operation are established by defining the input and output interfaces, comprising the following steps:

[0017] According to different container sources, define multiple input interfaces, and define multiple output interfaces according to different destinations;

[0018] Design a job mechanism process, including task allocation, path planning and loading and unloading operation when the ship arrives at the port;

[0019] Convert the job mechanism into a mathematical model, and set the objective function;

[0020] The objective function is set to minimize the sum of the waiting time of the shore crane and the idle waiting time of the horizontal transportation equipment.

[0021] As a preferred scheme of the port horizontal transportation equipment real-time scheduling method based on the MIMO mechanism provided by the application, wherein:

[0022] The reinforcement learning algorithm configuration in the simulation environment comprises the following steps:

[0023] The state information in the horizontal transportation equipment set is sorted and indexed by a sorting index function to generate a state array containing time and distance;

[0024] The state array is input into the Q-learning algorithm, the Q(s,a) value in the Q table is updated through the Training function, and the maximum Q value of all possible actions under the given state is obtained through the get_Max_QValue function;

[0025] Based on the Q value, combined with a variety of rule bases, the optimal action is selected through the epsilon_temperature-action function and the act-action function;

[0026] Based on the optimal action, a hierarchical composite reward function is designed, and the weights are dynamically adjusted according to the MIMO coupling relationship.

[0027] As a preferred scheme of the port horizontal transportation equipment real-time scheduling method based on the MIMO mechanism provided by the application, wherein:

[0028] The act-action function includes a priority setting strategy;

[0029] The priority setting strategy comprises:

[0030] The equipment with the longest waiting time is preferentially selected;

[0031] If there is no equipment with the longest waiting time, the equipment closest to the target is selected;

[0032] If there is no equipment with the longest waiting time and the equipment closest to the target, the equipment farthest from the target is selected;

[0033] The equipment that selects the longest waiting time Represented as:

[0034]

[0035] The equipment closest to the target is selected. Represented as:

[0036]

[0037] The equipment furthest from the target Represented as:

[0038]

[0039] Where C represents the horizontal transport equipment set, v j Represents the set of candidate devices C candidate The device in, v i Denotes a device from the set C of all available devices, d j Indicates equipment v j Distance to the target, d i Indicates equipment v i Distance to the target.

[0040] As a preferred embodiment of the real-time scheduling method for port horizontal transport equipment based on the MIMO mechanism described in this invention, wherein:

[0041] The sorting and indexing process using a sorting index function generates a state array containing time and distance, represented as follows:

[0042]

[0043] Among them, qc id The identifier represents the current quay crane, C represents the set of horizontal transport equipment, |C| represents the number of available transport equipment in set C, and rank represents the number of available transport equipment in set C. T (C1) represents the index array obtained by sorting set C according to the total waiting time, rank d (C2) represents the index array obtained by sorting set C according to the distance to the current quay bridge.

[0044] As a preferred embodiment of the real-time scheduling method for port horizontal transport equipment based on the MIMO mechanism described in this invention, wherein:

[0045] The update of the Q(s,a) value in the Q table using the Training function is expressed as follows:

[0046]

[0047] wherein r represents a reward for taking the current action, r represents a discount factor, a represents a learning rate, and r(s, a) represents an immediate reward obtained after taking action a in state s.

[0048] As a preferred solution of the port horizontal transportation equipment real-time scheduling method based on the MIMO mechanism, wherein:

[0049] The hierarchical composite reward function is represented as:

[0050] r t = (-1) * [a * (T qc ) + b * T + g * V]

[0051] wherein r t is an immediate reward value, T qc is an incremental waiting time of the horizontal transportation equipment, T is an idle waiting time of the quay crane, V is a system efficiency, and a, b, and g are weight coefficients.

[0052] In a second aspect, the present application provides a port horizontal transportation equipment real-time scheduling system based on the MIMO mechanism, comprising:

[0053] The packaging module is configured to obtain operation parameters in a terminal operation environment and equipment, and package port loading and unloading operations and horizontal transportation operations as a MIMO interface.

[0054] The construction module is configured to abstract the operation parameters in the terminal operation environment as network communication parameters, and construct the MIMO interface.

[0055] The modeling module is configured to define input and output interfaces and an objective function based on the MIMO interface, and establish an operation mechanism and a mathematical model of the port operation.

[0056] The simulation module is configured to construct a road network and physical facilities based on the MIMO interface and the mathematical model, and build a port simulation environment.

[0057] The configuration module is configured to configure a reinforcement learning algorithm in the simulation environment, and obtain an optimal scheduling strategy of the port horizontal transportation equipment.

[0058] In a third aspect, the present application provides a computing device, comprising:

[0059] A memory is configured to store a program.

[0060] A processor is configured to execute the computer executable instructions, and the computer executable instructions, when executed by the processor, implement the steps of the port horizontal transportation equipment real-time scheduling method based on the MIMO mechanism.

[0061] In a fourth aspect, the present application provides a computer readable storage medium comprising: the program is executed by a processor to realize the steps of the port horizontal transportation equipment real-time scheduling method based on the MIMO mechanism.

[0062] The present application has the following beneficial effects: the present application proposes a port horizontal transportation equipment real-time scheduling framework based on a multiple-input multiple-output (MIMO) network mechanism, a simulation environment based on a real scene is built by using Anylogic simulation software, and a reinforcement learning module is configured, while realizing full-process visualization and data output, the state space, action space and reward function in the algorithm are improved, the modeling framework based on the MIMO operation mechanism is introduced, the light-weight state space, action space and other modules in the framework are completely constructed in the scheduling system of the simulation platform, and the horizontal transportation equipment real-time scheduling method is formed. The purpose is to solve the problems that the traditional method depends on offline calculation, cannot respond to the influence of dynamic factors such as task priority adjustment and equipment failure on the scheduling system in real time, has higher adaptability, and meets the needs of the current port industry. BRIEF DESCRIPTION OF DRAWINGS

[0063] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced as follows. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor. Among them:

[0064] Figure 1 A basic flowchart of a port horizontal transportation equipment real-time scheduling method based on a MIMO mechanism provided by an embodiment of the present application is shown in the figure;

[0065] Figure 2 A network interface docking schematic diagram of a port horizontal transportation equipment real-time scheduling method based on a MIMO mechanism provided by an embodiment of the present application is shown in the figure;

[0066] Figure 3 A MIMO transportation network construction schematic diagram of a port horizontal transportation equipment real-time scheduling method based on a MIMO mechanism provided by an embodiment of the present application is shown in the figure;

[0067] Figure 4 A simulated horizontal transportation equipment real-time scheduling schematic diagram of a port horizontal transportation equipment real-time scheduling method based on a MIMO mechanism provided by an embodiment of the present application is shown in the figure;

[0068] Figure 5 An algorithm flowchart of a port horizontal transportation equipment real-time scheduling method based on a MIMO mechanism provided by an embodiment of the present application is shown in the figure;

[0069] Figure 6 An embodiment iterative diagram of a port horizontal transportation equipment real-time scheduling method based on a MIMO mechanism is provided for an embodiment of the present application. DETAILED DESCRIPTION

[0070] In order to make the above objectives, characteristics and advantages of the present application more obvious and easy to understand, the specific embodiments of the present application will be described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor should fall within the scope of protection of the present application.

[0071] Embodiment 1

[0072] Reference Figures 1-5 For an embodiment of the present application, a port horizontal transportation equipment real-time scheduling method based on a MIMO mechanism is provided, comprising:

[0073] S1: Obtain the operation parameters in the terminal operation environment and the equipment, and encapsulate the port loading and unloading operation and the horizontal transportation operation as a MIMO interface;

[0074] In the embodiment of the present application, the port loading and unloading operation and the horizontal transportation operation are encapsulated as a MIMO interface, as shown in Figure 2 , comprising the following steps:

[0075] The number of unloading areas is encapsulated as the number of network antenna interfaces, i.e. input and output nodes;

[0076] The number of containers handled by a crane in one operation is encapsulated as the network interface channel capacity;

[0077] The number of parked vehicles in the operation area is encapsulated as the network interface data transmission frame buffer;

[0078] The operation efficiency of the crane is encapsulated as the data transmission rate of the network interface;

[0079] The transportation capacity of the container terminal horizontal transportation equipment and the number of devices are encapsulated as the bandwidth;

[0080] The number of containers transported by the container terminal horizontal transportation equipment in unit time is encapsulated as the communication system capacity;

[0081] Uncertain interference such as AGV failure and weather influence is encapsulated as noise;

[0082] The container task is encapsulated as a data packet, referred to as "box flow";

[0083] The internal roads of the port are encapsulated as channels;

[0084] The container dispatching strategy is encapsulated as modulation / coding;

[0085] In the embodiments of the present application, the MIMO interface of the port operation is extracted and encapsulated, so that the data transmission of the "container flow / truck flow" data is changed into the data transmission of the data packet, each network interface processes the data in parallel, and the effective logical separation of the horizontal transportation system and the stacking / loading / unloading operation system is realized.

[0086] It should be noted that the present application introduces the MIMO communication theory framework, encapsulates the ships, trains, external trucks, and each unloading point (yard bridge unloading point, shore bridge unloading point) of the port, and maps them into the input / output points of the "container flow" data. The internal path network of the port is modeled as a channel matrix. This modeling method quantifies the container flow efficiency between nodes through matrix operation, and provides a low-dimensional state representation and a hierarchical reward function for reinforcement learning. The low-dimensional state representation and the hierarchical reward function can encapsulate the multiple devices of the port, facilitate the systematic modeling and effect realization of the simulation software, and the like.

[0087] S2: Based on the MIMO interface, the input / output interface and the target function are defined, and the operation mechanism and the mathematical model of the port operation are established;

[0088] In the embodiments of the present application, the operation mechanism integrating the MIMO interface is constructed, and the problems and mathematical models to be constructed are described, including the following steps:

[0089] 2.1 Construct the number of interfaces. The MIMO can realize the input of "container flow" from multiple sources, and the input interfaces corresponding to different sources of containers, such as ships, yards, trains, external trucks on highways, and shore cranes, are mathematically represented as X=[x ship ,x train ,x truck ,x q / yc ] T ; The output interfaces corresponding to the output destinations of containers, such as ships, yards, trucks, trains, and yard bridges, are mathematically represented as Y=[y ship ,y train ,y truck ,y q / yc ] T In the present application, a buffer point (waiting area) is added, and the interfaces constitute a multiple-input multiple-output transportation network, as shown in Figure 3 .

[0090] 2.2 Determine the task and target. The goal of the container terminal horizontal transportation system is to efficiently and accurately complete the connection task of containers between loading / unloading and stacking units, that is, the operation control can be abstracted as a real-time scheduling problem of horizontal transportation equipment driven by MIMO mechanism. Therefore, the task of the operation mechanism is to make reasonable real-time decisions through algorithms under the premise of known ("box flow" data), select appropriate horizontal transportation equipment (line in MIMO) to perform tasks, so that the sum of idle waiting time of all horizontal transportation equipment and quay crane waiting time (delay) is minimized.

[0091] 2.3 Construct the operation mechanism. The operation mechanism of the present application is that the ship arrives at the port (data enters), the intelligent agent undertakes the task allocation function, calls the algorithm to realize the real-time assignment of tasks according to the environmental state, and selects appropriate horizontal transportation equipment according to the scheduling strategy (modulation, encoding). When the horizontal transportation equipment arrives at the quay crane (input node), the quay crane and the spreader perform loading and unloading operations. After completing the loading of containers from the quay crane to the horizontal transportation equipment, the horizontal transportation equipment obtains the container task information, moves to the target berth of the target box area on the road (channel) through path planning, and at the same time, the yard crane (output node) moves to the unloading point. Then the spreader on the yard crane performs unloading work, moves to the target box position, and completes the unloading of the horizontal transportation equipment to the yard (realizes multipath data transmission). The horizontal transportation equipment goes to the waiting area and waits for the next task instruction. Complete data transmission, that is, container flow transmission.

[0092] 2.4 Mathematical modeling process based on its operation mechanism

[0093] 2.4.1 Describe the problem solved by the present method: under the premise that the operation task ("box flow" data) is known, make reasonable real-time decisions through algorithms, select appropriate horizontal transportation equipment (line in MIMO) to perform tasks, that is, the matching process of containers ("box flow" data) and horizontal transportation equipment (line in MIMO), so that the sum of idle waiting time of all horizontal transportation equipment and quay crane waiting time (delay) is minimized.

[0094] 2.4.2 Set the objective function:

[0095] min{qctotalwaitTime+agv_totalwaitTime}

[0096] Wherein, qctotalwaitTime is the waiting time of the quay crane; agv_totalwaitTime is the idle waiting time of the horizontal transportation equipment.

[0097] 2.4.3 Markov decision process modeling

[0098] The process is abstracted as a Markov Decision Process (MDP). Through this abstraction, the state of the horizontal transport equipment on the automated container terminal can be recorded and the decision-making behavior can be guided.

[0099] The Markov decision problem can be represented by a six-tuple (S, A, P, γ, R, π). S is the state space, which contains all possible states s of the horizontal transport equipment during the entire scheduling process, used to describe the current environment and provide environmental information for the agent to make decisions; A is the action space composed of all executable actions a according to the scheduling rules; P represents the probability of transitioning from the current state s to the next state s' when taking an action a in the action space; γ is the discount factor, which balances short-term and long-term benefits; R is the reward function, which returns a reward value after executing an action, used to evaluate the goodness of the action; π is a strategy that maps from the state set S to the action set A, and the strategy π(s, a) represents the probability of taking a specific action A in a given state S.

[0100] 2.4.4 Problem assumptions

[0101] (1) All quays (input interfaces), horizontal transport equipment, and yard cranes (output interfaces) are working from time 0;

[0102] (2) All container (box flow) operation plan information is known;

[0103] (3) Each horizontal transport equipment (line) can only transport one container at a time (network interface channel capacity is 1);

[0104] (4) The number of yard cranes (output interfaces), quays (input interfaces), and horizontal transport equipment (lines) is determined at the beginning, and only the scheduling of horizontal transport equipment (lines) needs to be performed;

[0105] (5) When the horizontal transport equipment (line) has no work, it waits in place;

[0106] (6) Energy consumption is not considered;

[0107] S3: Based on the MIMO interface and mathematical model, a port simulation environment is built by constructing a road network and physical facilities;

[0108] In the embodiments of the present application, after the mathematical model integrating the MIMO interface and mechanism is completed, the port simulation environment building includes the following steps:

[0109] 3.1 Road network and physical facility construction

[0110] According to the simulation software Anylogic, combined with the port MIMO operation mechanism and mathematical model in S1 and S2, first, the road network is constructed, according to the size of the port, layout, number of lanes, traffic lights, traffic guide lines, container_height-container length, container_width-container width, yard_length-yard width, yard_num-yard number and other parameters to construct the road network of the model. Secondly, the construction of physical facilities is carried out, according to the actual situation, the shore crane adopts single trolley or double trolley shore crane; horizontal transportation equipment is ART / AGV / IGV, etc.; yard crane is track crane or tire crane, etc. Using discrete event and multi-agent modeling, using built-in modules such as road traffic library, process modeling library, etc., with three-dimensional model or CAD import in the software, get the port model according to the real scene modeling, such as Figure 4

[0111] 3.2 Construction of quay crane, yard crane, horizontal transportation equipment, ship, Q learning agent

[0112] After the construction of road network and physical facilities, the function needs to be realized in each kind of equipment, and each kind of agent is constructed. In the main interface, the following agent collections are constructed: quaycrane_collection quay crane collection-network interface number; yardcrance_collection yard crane collection-network interface number; vehicle-collection all horizontal transportation equipment collection-network bandwidth; collection_vessel_containers import container collection to be transferred-data total amount; agv_stopline horizontal transportation equipment stop line collection; agv_parkinglot_collection horizontal transportation equipment parking lot collection;

[0113] 3.2.1 Ship, quay crane agent construction

[0114] ​The ship agent contains several parameters of the ship itself such as ship_id, numofbay, numofrow, numoftier, which represent the ship ID, bay number, row number, tier number, and the container task agent carried. When docking, the arrival information is sent to the quay crane, and the quay crane assigns the task. The quay crane (input / output node) agent defines the behavior in the form of a flowchart (taking the unloading task as an example): when the ship arrives at the port, each quay crane quaycrane_collection obtains its own task information, loads the container task into its own task collection collection, and processes them in turn. When the task allocation function is completed, the selected horizontal transportation equipment arrives at the quay crane target stop line agv_stopline from the parking lot agv_parkinglot_collection, and the unloading function is started, and the spreader agent performs the unloading action. After the container is unloaded, the next task allocation function is continued, and the cycle is repeated until the task is completed. A collection collection is established in the agent to represent the task list; agv_collection represents the selected horizontal transportation equipment collection;

[0115] 3.2.2 Yard crane agent construction

[0116] The yard crane agent also defines the behavior of the agent in the form of a flowchart (taking the unloading task as an example): when the horizontal transportation equipment igv_collection carries the container igv_container_collection to the destination, the yard crane moves to the container, the spreader agent unloads the container, and the horizontal transportation equipment leaves the field, waits for the arrival of the next container task, and repeats the cycle.

[0117] 3.2.3 Horizontal transportation equipment agent construction

[0118] The running logic (taking the unloading task as an example) is composed of modules in the Anylogic software and some parameters to form the agent. When the horizontal transportation equipment is waiting for task assignment, the initial state before the horizontal transportation equipment performs the task is recorded state0, then the action taken is recorded action0, the horizontal transportation equipment is assigned to the target quay crane destination-qc, the container task is taken, the task information is obtained to go to the yard crane destination-yc, the unloading task is performed, and after completion, the state of the horizontal transportation equipment after performing the task is recorded state1, and finally the reward is recorded by the function reward. Complete a task, and repeat the cycle until the task is completed. Record the totalwaitTime between two tasks, and the qcwaitTime of the quay crane.

[0119] 3.2.4 Q-learning agent

[0120] The qTable parameter of HashMap type is responsible for storing the parameters that need to be written into the Q table; the definitions of learningRate, discountFactor, and epsilon. The get_Max_QValue function and the training function are used to calculate and traverse the optimal Q value.

[0121] S4: Reinforcement learning algorithm configuration is performed in the simulation environment to obtain the optimal scheduling strategy of the port horizontal transportation equipment.

[0122] In the embodiments of the present application, the reinforcement learning algorithm configuration includes the following steps:

[0123] The main process is: 1) the system collects environmental information and inputs the state; 2) then enters the iterative learning of the Q-learning algorithm of the training module and the Q table query; 3) the action space module selects the action and assigns the task; 4) after completing the task, the reward function module is entered to calculate the reward and update the Q value, and the cycle is repeated.

[0124] It mainly includes the construction of the state input module, the construction of the training module, the construction of the action space module, and the construction of the reward function module.

[0125] 4.1 Construction of the state input module

[0126] The state space is constituted, including discretizing and combining the state information representing the system in the simulation model into an array. The traditional state array is improved to be the ranking in the set of available horizontal transportation equipment, reducing the dimension of the state space. The mathematical representation is as follows:

[0127]

[0128] Among them, C1 represents the equipment set sorted by total waiting time T i , C2 represents the equipment set sorted by distance d i , rank k (S) = index (sort (S, k), current): ranking index function, qc id represents the current quay crane identifier.

[0129] Among them, the workflow of the ranking index function is as follows:

[0130] 1) If the number of vehicle-collection elements is not zero, sort the vehicle-collection (ascending order of the totalwaitTime of the members); record the index position, store in the variable timeState;

[0131] 2) At the same time, sort the elements in ascending order of the distance to the current quay crane), record the index position, store in the variable disState.

[0132] 3) Convert the array [qc id , timeState, disState] into a string format and return; otherwise, create a zero array [0, 0, 0] and convert the array into a string format and return.

[0133] 4.2 Construction of the training module

[0134] The Training function and the get_Max_QValue function are included, the former calculates the Q value, and the latter updates the Q value. The flow chart of the Q-learning algorithm is shown in Figure 5 When there is a task demand (MIMO has "box flow" data to be processed), the scheduling system first initializes the environment and the Q table, obtains the input of the system state (MIMO network interface), checks the traversal and selects the optimal action in this state using the historical Q table, selects the optimal action in the action space-scheduling rule, and outputs a scheduling rule, selects the horizontal transportation equipment (MIMO line) that meets the scheduling rule to execute the current task ("box flow" data), when the current task is completed, the state is recorded again and the reward value is calculated, then the Q table is updated and maintained, until all tasks are completed, output a Q table, when the next training starts, the last saved Q table can be used for continuous training, and so on.

[0135] The update rule of the Training function is:

[0136]

[0137] Where, r represents the reward of taking the current action. γ: discount factor. α represents the learning rate (usually 0<α≤1)

[0138] The get_Max_QValue function is expressed as:

[0139]

[0140] Where, s∈S: current state, A={0,1,2} represents the state space, Q(s,a) represents the state-value function.

[0141] In more detail:

[0142]

[0143] At the beginning of the run, the Q table is initialized with the formula:

[0144]

[0145] (r i ~Uniform(0, 1)

[0146] The convergence condition of its operation is expressed as:

[0147] 4.3 Construction of the action space module

[0148] Including the selection and execution of actions of ε_temperature-action and act-action. The scheduling mapping rule formed is: A = {N-CAR = 1; F-CAR = 2; WL-CAR = 3}, that is, the scheduling rule library is expressed as: including "nearest distance first", "farthest distance first", "longest waiting time" and other actions.

[0149] The former dynamically balances exploration and utilization through a double strategy (ε-greedy + Boltzmann). Including: 1) ε-greedy strategy directly selects randomly; 2) Boltzmann soft selection, calculate the temperature parameter, calculate the probability distribution according to the Q value and the temperature. 3) Monitor when the Q value is all zero to force exploration.

[0150] Its mathematical definition is as follows:

[0151]

[0152] Where, ε represents the exploration rate.

[0153] The mathematical definition of Boltzmann probability distribution is:

[0154]

[0155] Where, Indicates the dynamic temperature (N is the training round).

[0156] The zero value protection mechanism is:

[0157]

[0158] The action index is outputted by the function of ε_temperature-action, and the horizontal transportation equipment is outputted by matching the corresponding scheduling rule with act-action. The functions corresponding to the three action indexes in act-action are as follows:

[0159] Case0: nearest distance first

[0160] Select the device closest to the target from all available horizontal transportation equipment set C is expressed as:

[0161]

[0162] Update rule:

[0163]

[0164] v * .idle = 0, v * .work_if = 1

[0165] Case1: farthest distance first

[0166] Select the device farthest from the target from all available horizontal transportation equipment set C is expressed as:

[0167]

[0168] The update rule is the same as Case0.

[0169] Case2: longest waiting time first

[0170] Define a candidate set C andidate is expressed as:

[0171]

[0172] Select the device closest to the target is expressed as:

[0173]

[0174] The update rule is the same as Case0.

[0175] Wherein, C = {v1, v2...v n} represents the horizontal transportation equipment set, d i = v i .distanceTo(this) represents the distance of the equipment v i to the target, T i = v i.totalwaitTime indicates the equipment v i The total waiting time, where t represents the current time (time()). Indicates equipment v i The last time the task was completed, d indicates that the equipment was in an idle state after completing a task (marked as 1), and idle indicates the work state. i f: Is it currently in operation?

[0176] The priority selection process for the three action indices is as follows:

[0177] Step 1: Check if any equipment meets the longest wait time priority strategy (Case 2). If there is equipment that meets the criteria, select the equipment that is closest to the target.

[0178] Step 2: If no equipment matching the criteria is found, switch to the nearest distance priority strategy (Case 0) and select the equipment closest to the target.

[0179] Step 3: If there are still special circumstances or further needs, the farthest distance priority strategy can be selected (Case 1).

[0180] It should be noted that this sequential design aims to ensure efficient scheduling while also considering balanced resource utilization, system stability, and user satisfaction. Prioritizing the equipment with the longest waiting time effectively reduces system bottlenecks and improves overall efficiency; the nearest-distance-first strategy ensures tasks are completed quickly; and the farthest-distance-first strategy serves as a supplementary solution, providing additional flexibility in specific scenarios. This hierarchical strategy selection mechanism is better suited to the complex port operating environment.

[0181] 4.4 Construction of the Reward Function Module

[0182] This invention integrates the system-level objective of MIMO and designs a hierarchical composite reward function that incorporates the waiting time costs of multiple devices, system efficiency, and resource requirements to prevent getting trapped in local optima. Simultaneously, the weights of sub-rewards are dynamically adjusted through MIMO coupling relationships (e.g., increasing α when propagation is concentrated at the port). The reward function designed in this invention is as follows:

[0183] r t = (-1)*[α*(T) qc )+β*T+γ*V]

[0184] Where, r t T is the instant reward value. qc Let T be the waiting time increment for horizontal transport equipment, T be the idle waiting time of the quay crane, V be the system efficiency, and α, β, and γ be weighting coefficients.

[0185] 1) Time parameter update rules

[0186] Update when task complete:

[0187] Waiting time calculation for quay cranes: T qc =tt qc_start →igv.qctotalwaitTime

[0188] 2) Reward Calculation and Learning Updates

[0189]

[0190] Where D represents the set of records of rewards.

[0191] This embodiment also provides a real-time scheduling system for port horizontal transport equipment based on the MIMO mechanism, including:

[0192] The encapsulation module is used to acquire the operating parameters of the terminal operation environment and equipment, and encapsulates port loading and unloading operations and horizontal transportation operations into MIMO interfaces.

[0193] The module is used to abstract operational parameters in the dock operation environment into network communication parameters and build MIMO interfaces;

[0194] The modeling module is used to establish the operational mechanism and mathematical model of port operations based on the MIMO interface by defining input and output interfaces and objective functions;

[0195] The simulation module is used to build a port simulation environment based on the MIMO interface and mathematical model by constructing road networks and physical facilities;

[0196] The configuration module is used to configure reinforcement learning algorithms in a simulation environment to obtain the optimal scheduling strategy for port horizontal transportation equipment.

[0197] Furthermore, this also includes:

[0198] Memory, used to store programs;

[0199] A processor is used to load the program to execute the real-time scheduling method for port horizontal transport equipment based on the MIMO mechanism.

[0200] This embodiment also provides a computer-readable storage medium storing a program that, when executed by a processor, implements the real-time scheduling method for port horizontal transport equipment based on the MIMO mechanism.

[0201] The storage medium provided in the embodiment belongs to the same inventive concept as the port horizontal transportation equipment real-time scheduling method based on the MIMO mechanism provided in the above embodiment, and the technical details not described in detail in the embodiment can be referred to the above embodiment, and the embodiment has the same beneficial effects as the above embodiment.

[0202] Through the above description of the embodiments, those skilled in the art can clearly understand that the present application can be realized by means of software and necessary general hardware, and of course can also be realized by hardware, but in many cases the former is a better embodiment. Based on such understanding, the technical solutions of the present application can be embodied in the form of a software product, which can be stored in a computer readable storage medium, such as a floppy disk, a read-only memory (ROM), a random access memory (RAM), a FLASH, a hard disk or an optical disk, etc., including a plurality of instructions to make a computer device (which can be a personal computer, a server, or a network device, etc.) execute the methods of various embodiments of the present application.

[0203] Embodiment 2

[0204] This is an embodiment of the present application, which provides a port horizontal transportation equipment real-time scheduling method based on MIMO mechanism. In order to verify the beneficial effects of the present application, the specific embodiments and implementation effects are scientifically demonstrated.

[0205] The specific embodiments are as follows:

[0206] Experimental parameters: two quay cranes, six yard cranes and six horizontal transportation equipment were used in this experiment. discountFactor = 0.9; learningRate = 0.1; epsilon = 0.9.

[0207] Step one of the experiment: 40 container tasks were designed. The task table was input into the model in the form of excel, and part of the information is shown in Table 1:

[0208] Table 1 Task table

[0209] container_id current_block current_bay current_row current_tier block bay row tier 1 1 0 0 0 0 5 3 0 2 1 0 2 0 0 5 4 0 3 1 0 1 0 0 0 5 0 4 1 0 3 0 0 0 5 1 5 1 1 3 0 1 4 5 0 6 1 1 4 0 1 5 3 0 7 1 1 0 0 1 7 0 0 8 1 1 0 1 1 7 3 0

[0210] In which, each column of data respectively represents the container ID, the box area on the cargo ship, the berth, the row number, the layer number and the target position of the box area, the berth, the row number and the layer number.

[0211] Step two of the experiment: click the start button to enter the initial interface.

[0212] Step 3: Set initial number of yard cranes, initial number of quay cranes, initial number of horizontal transport equipment, initial running speed.

[0213] Step 4: Use Monte Carlo experiment in simulation software to perform multiple iterations, as shown in the following formula: Figure 6 The reward curve obtained after 1000 training iterations is shown in the following figure, and the waiting time converges to about 8500s.

[0214] Step 5: Collect and plot the output data and reward curve after the iteration is completed.

[0215] After testing, compared with the beginning of operation, the average waiting time of horizontal transport equipment and quay crane waiting time is reduced by about 4000s, the empty mileage of horizontal transport equipment is reduced by 20%, and the overall energy consumption is reduced by 18%. In the simulation of sudden tasks, the system response time is normal, and the processing capacity for uncertain events is met.

[0216] It should be noted that the above examples are only used to illustrate the technical solutions of the present application and are not limiting. Although the present application has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present application can be modified or replaced by equivalents without departing from the spirit and scope of the present application, and they should be covered in the scope of the claims of the present application.

Claims

1. A real-time scheduling method for port horizontal transportation equipment based on MIMO mechanism, characterized in that, The application relates to a MIMO mechanism-based port horizontal transportation equipment real-time scheduling method and a system thereof. The application comprises the following steps: Obtaining operation parameters in a terminal operation environment and equipment, and encapsulating port loading and unloading operations and horizontal transportation operations as a MIMO interface; Based on the MIMO interface, an input / output interface and a target function are defined to establish a terminal operation mechanism and a mathematical model thereof; Based on the MIMO interface and the mathematical model, a road network and physical facilities are constructed to build a terminal simulation environment; In the simulation environment, a reinforcement learning algorithm is configured to obtain an optimal scheduling strategy of the terminal horizontal transportation equipment; The step of defining the input / output interface and establishing the terminal operation mechanism and the mathematical model thereof comprises the following steps: According to different container sources and terminal cranes, a plurality of input interfaces are defined, and according to different destinations, a plurality of output interfaces are defined; An operation mechanism process is designed, including task allocation when a ship arrives at a terminal, path planning and loading and unloading operations; The operation mechanism is converted into a mathematical model, and a target function is set; The target function is set as the sum of the minimum waiting time of the terminal crane and the idle waiting time of the horizontal transportation equipment; The step of configuring the reinforcement learning algorithm in the simulation environment comprises the following steps: State information in a horizontal transportation equipment set is sorted and indexed by a sorting index function to generate a state array containing time and distance; The state array is input into a Q-learning algorithm, the Q(s, a) value in a Q table is updated by a Training function, and the maximum Q value of all possible actions under a given state is obtained by a get_Max_QValue function; Based on the Q value, a plurality of rule bases are combined, an optimal action is selected by an epsilon_temperature-action function and an act-action function, and a hierarchical composite reward function is designed based on the optimal action.

2. The real-time scheduling method of port horizontal transportation equipment based on MIMO mechanism according to claim 1, characterized in that: The act-action function comprises a priority setting strategy; The priority setting strategy comprises the following steps: The longest waiting time equipment is preferentially selected; If there is no longest waiting time equipment, the equipment closest to the target is selected; If there is no longest waiting time equipment and the equipment closest to the target, the equipment farthest from the target is selected. The selection of the longest latency equipment is represented as: The selection distance closest to the target is represented as: the equipment farthest from the target is represented as: where C denotes the set of horizontal transport equipment, v j denotes a device in the set of candidate devices C candidate denotes a device in the set of candidate devices C i denotes a device in the set of all available devices C j denotes the distance of equipment v j to the target, d i denotes the distance of equipment v i to the target.

3. The real-time scheduling method of port horizontal transportation equipment based on MIMO mechanism according to claim 2, characterized in that: The step of sorting and indexing by the sorting index function to generate the state array containing time and distance is represented as follows: where qc id represents the identifier of the current quay crane, C represents the set of horizontal transport equipment, |C| represents the number of transport devices available in the set C, rank T (C1) represents the index array obtained after sorting the set C by total waiting time, rank d (C2) represents the index array obtained after sorting the set C by distance to the current quay crane.

4. The real-time scheduling method of port horizontal transportation equipment based on MIMO mechanism according to claim 3, characterized in that: The step of updating the Q(s, a) value in the Q table by the Training function is represented as follows: Wherein, r represents the reward of taking the current action, gamma represents the discount factor, alpha represents the learning rate, and r(s, a) represents the immediate reward obtained after taking the action a in the state s.

5. The real-time scheduling method of port horizontal transportation equipment based on MIMO mechanism according to claim 4, characterized in that: The hierarchical composite reward function is represented as follows: r t = (-1) * [a1 * (T qc )+ b * T + g1 * V] wherein r t is the instant reward value, T qc is the incremental waiting time of the horizontal transport equipment, T is the idle waiting time of the quay crane, V is the system efficiency, and α1, β, γ1 are weight coefficients.

6. A MIMO mechanism-based port horizontal transportation equipment real-time scheduling method system according to claim 1, wherein: An encapsulation module is used to obtain operation parameters in a terminal operation environment and equipment, and encapsulate port loading and unloading operations and horizontal transportation operations as a MIMO interface; A construction module is used to abstract the operation parameters in the terminal operation environment into network communication parameters and construct the MIMO interface. The modeling module is configured to establish a working mechanism and a mathematical model of port operation based on the MIMO interface by defining an input-output interface and an objective function; The simulation module is configured to build a port simulation environment by constructing a road network and physical facilities based on the MIMO interface and the mathematical model; The configuration module is configured to configure a reinforcement learning algorithm in the simulation environment to obtain an optimal scheduling strategy of the port horizontal transportation equipment.

7. A computing device, comprising: The program is loaded by the processor to execute the steps of the port horizontal transportation equipment real-time scheduling method based on the MIMO mechanism. The program is loaded by the processor to execute the steps of the port horizontal transportation equipment real-time scheduling method based on the MIMO mechanism. The program is loaded by the processor to execute the steps of the port horizontal transportation equipment real-time scheduling method based on the MIMO mechanism.

8. A computer-readable storage medium storing a program, characterized in that, ​

Images