A method and system for regional coordination optimization of shipping centers based on big data

Abstract

The present invention discloses a regional coordination optimization method and system for shipping centers based on big data, including: dividing the working hours of the whole day into multiple decision periods, establishing a state transfer model, maximizing the utilization rate of port resources through iterative models under constraints, obtaining resource allocation strategies for each decision period, obtaining chain reaction chains, identifying logistics bottlenecks and adding queue planning, treating ships, berths, yards and loading and unloading equipment as different intelligent agents, dynamically adjusting queue planning using multi-agent collaborative algorithms, obtaining the final collaborative scheduling method, setting reward functions and using reinforcement learning to update the state transfer model, and realizing continuous optimization of the scheduling method. The method realizes efficient utilization of port resources through multi-agent collaborative algorithms and reinforcement learning technology, improves port operation efficiency, reduces logistics costs, and enhances the port's adaptability to uncertainty, while having good interpretability.

Description

Technical Field

[0001] The present invention relates to the field of scheduling resource management, and in particular to a method and system for regional coordination optimization of shipping centers based on big data. Background Art

[0002] In the context of global economic integration, the efficient operation of the shipping industry, as a key link in international trade, is of vital importance. However, with the continuous growth of trade volume, the flow of ships has become increasingly complex and changeable, which has brought great challenges to the resource management and scheduling of shipping centers. How to achieve efficient allocation of resources such as ships, berths, and loading and unloading equipment under limited resource conditions has become a key issue in enhancing the competitiveness of shipping centers.

[0003] At present, traditional methods of resource scheduling in shipping centers are mostly based on experience or simple rules, and lack full use of big data. These methods are often unable to accurately respond to the dynamic changes in ship traffic in real time, resulting in low resource utilization, long waiting time for ships, increased logistics costs and other problems. At the same time, in the face of complex risk factors, traditional methods are difficult to fully predict their impact on resource allocation strategies, which in turn triggers a chain reaction, causing logistics bottlenecks and seriously affecting the overall operational efficiency of shipping centers.

[0004] A method and system for regional coordination and optimization of shipping centers based on big data. By analyzing shipping data and using algorithms and models to achieve coordinated scheduling, the system can realize scientific planning and dynamic adjustment of shipping center resources, improve resource utilization, reduce operational risks, and enhance overall operational efficiency. Summary of the invention

[0005] The purpose of the present invention is to provide a method and system for regional coordination optimization of shipping centers based on big data.

[0006] To achieve the above object, the present invention is implemented according to the following technical solutions:

[0007] The first aspect of the present invention provides a method for regional coordination optimization of shipping centers based on big data, comprising:

[0008] S100 divides the working hours of the whole day into multiple decision-making periods according to the fluctuation characteristics of ship traffic;

[0009] S200 establishes a state transition model based on time series, and iterates the state transition model under constraints to maximize port resource utilization, thereby obtaining a resource allocation strategy for each decision period;

[0010] S300 predicts the probability of occurrence of risks in the resource allocation strategy to obtain a chain reaction chain;

[0011] S400 simulates the chain reaction chain in the order of low-volume transportation to high-volume transportation, and adds queue planning to the resource allocation strategy according to the logistics bottleneck in the simulation result;

[0012] S500 regards ships, berths, yards and loading and unloading equipment as different agents, and dynamically adjusts the queue planning through a multi-agent collaborative algorithm to obtain the final collaborative scheduling method;

[0013] S600 sets a reward function, updates the state transfer model through reinforcement learning, and realizes continuous optimization of the final collaborative scheduling method.

[0014] As a further method, the method of dividing the full-day working hours into multiple decision-making periods according to the fluctuation characteristics of ship traffic includes:

[0015] Obtain the estimated arrival time of the ship and arrange it in chronological order to form an ordered sample sequence;

[0016] For any time period in the ordered sample sequence, the intra-group difference is calculated, and the expression is:

[0017]

[0018] in, Indicates time period The intra-group differences, and are the start and end time of the time period, For the Ship traffic in a certain time period, For time period Average vessel traffic within the area;

[0019] The optimal split point is determined by minimizing the variance within the group and maximizing the difference in ship flow between different time periods;

[0020] According to the optimal segmentation point, the whole day is divided into multiple decision periods, each of which corresponds to different ship flow characteristics.

[0021] As a further method, the method for establishing a state transition model based on time series includes:

[0022] The port resource status, including berth occupancy, remaining operating capacity of loading and unloading equipment and remaining space in the yard, is taken as the state variable; the resource allocation scheme for different ships and cargoes, including berth allocation, equipment scheduling and manpower arrangement, is taken as the decision variable;

[0023] The state transition equation based on time series is constructed according to the state variables and decision variables. The expression is:

[0024]

[0025]

[0026]

[0027] in, represents the decision period, , is the total number of decision periods, and In the and Berth occupancy during each decision period, For the The ship arrival rate in the decision period, For the The ship departure rate in the decision period, For the The set of arriving ships in the decision period, For allocation to ships The berth occupancy ratio For the The number of available berths in a decision period, and Respectively and The remaining operating capacity of the equipment in each decision period, For the The set of working devices in the decision period, For equipment In the The operation duration requirement for each decision period, For the The average operating efficiency of the equipment in the decision period, For the The remaining operating capacity of the equipment during each decision period, and In the and The remaining space in the yard during the decision period, For ships The area of ​​the yard occupied by the cargo, For the The utilization efficiency of the yard space in each decision period, For the The space released in the yard during each decision period.

[0028] As a further method, the method of iterating the state transition model to maximize the port resource utilization under the constraint condition to obtain the resource allocation strategy for each decision period includes:

[0029] According to the actual physical limitations of the port, the constraints of the number of berths, the maximum operating capacity of equipment and the total area of ​​the yard are obtained;

[0030] Taking maximizing port resource utilization as the objective function, the expression is:

[0031]

[0032] in, is the total number of decision periods, , and are weight coefficients, , and Respectively The number of occupied berths, equipment operating hours and remaining space in the yard for each decision period. , and They are the total number of berths, the maximum operating time of loading and unloading equipment and the maximum yard space. For the The number of ships arriving at the port during the decision period, For the The first decision period The cargo volume of the ship, For the The maximum cargo capacity of a vessel;

[0033] Taking the objective function as the iteration direction, under the constraints, the state variables and decision variables are updated according to the state transition model until the termination conditions are met;

[0034] The termination condition is that the rate of change of the objective function is less than 0.001 in 10 consecutive iterations.

[0035] As a further method, the method of predicting the probability of occurrence of risks in the resource allocation strategy and obtaining a chain reaction chain includes:

[0036] Construct a risk prediction model through a convolutional neural network, obtain historical risk data and corresponding historical resource allocation strategies as training sets, and train the risk prediction model;

[0037] The resource allocation strategy at each decision moment is input into the risk prediction model, and the probability of risk occurrence is output, which is expressed as a probability density function. The expression is:

[0038]

[0039] in, and are resource allocation strategies and model parameters, respectively. and Given a resource allocation strategy, Under this condition, the expected value and standard deviation of the probability of risk occurrence are: is the nonlinear adjustment coefficient;

[0040] The association mining algorithm is used to identify the connection between risk factors in historical risk data, and a Bayesian network model is constructed. The risk probability is input into the Bayesian network model to obtain a chain reaction chain.

[0041] As a further method, the chain reaction chain is simulated in the order of low-volume transportation to high-volume transportation, and a method of adding queuing planning to the resource allocation strategy according to the logistics bottleneck in the simulation result includes:

[0042] The logistics bottlenecks include the shortage of berths, loading and unloading equipment and yard space, as well as the congestion of ship arrival, cargo loading and unloading and yard management;

[0043] The chain reaction is simulated by Monte Carlo simulation method in the order of transportation task volume from low to high, and the logistics bottleneck links that appear in each decision period are obtained;

[0044] The transportation tasks in the logistics bottleneck links are queued in descending order of transportation priority, and the queue length of each logistics bottleneck link is averaged.

[0045] As a further method, the method of dynamically adjusting the queue planning through a multi-agent collaborative algorithm to obtain a final collaborative scheduling method includes:

[0046] Give the agent state variables and action space;

[0047] The state variables include the resource occupancy of the agent;

[0048] The action space is the current task assignment of the agent;

[0049] Each agent communicates and negotiates with other agents through a multi-agent collaborative algorithm. An agent sends resource requests to other agents based on the current task allocation and its own resource occupancy. Other agents respond to the request if there are idle resources based on their own resource occupancy.

[0050] In each decision period, the queue planning is dynamically adjusted according to the interaction between agents.

[0051] As a further method, the method of setting a reward function and updating the state transfer model through reinforcement learning to achieve continuous optimization of the final collaborative scheduling method includes:

[0052] Taking port resource utilization, ship waiting time, loading and unloading equipment utilization and berth turnover efficiency as evaluation indicators, a reward function is constructed, and the expression is:

[0053]

[0054] in, is the reward value, , , and are the weight coefficients of port resource utilization, ship waiting time, loading and unloading equipment utilization and berth turnover time, respectively. is the port resource utilization rate, is a natural constant, is the average waiting time of the ship, is the target value of the ship waiting time, is the fluctuation coefficient of loading and unloading equipment utilization, is the utilization rate of loading and unloading equipment, is the average berth turnover time, is the target value for berth turnover time;

[0055] A deep Q network is used to maximize the reward function for reinforcement learning, where the loss function of the training process is expressed as:

[0056]

[0057] in, is the loss function, are the main Q network parameters, is the expected value, which means averaging all samples sampled from the experience replay pool. is a sample, including the state , actions taken , get instant rewards and the next state , For the experience replay pool, Indicates taking action Back from state Transfer to state Instant rewards received, is the discount factor, For the next state All possible actions The maximum value of Q value, are the parameters of the target Q network;

[0058] The action decisions obtained from reinforcement learning training are mapped to the decision variables in the state transfer model, the state transfer model is updated, and then the final collaborative scheduling method is updated to achieve continuous optimization.

[0059] The second aspect of the present invention provides a shipping center regional coordination optimization system based on big data, comprising:

[0060] The time division module is used to divide the working hours of the whole day into multiple decision-making periods according to the fluctuation characteristics of ship traffic;

[0061] The strategy iteration module establishes a state transition model based on time series, iterates the state transition model under constraints to maximize the utilization rate of port resources, and obtains the resource allocation strategy for each decision period;

[0062] A risk prediction module, used to predict the probability of risk occurrence in the resource allocation strategy and obtain a chain reaction chain;

[0063] A bottleneck planning module is used to simulate the chain reaction chain in the order of low-volume transportation to high-volume transportation, and add queue planning to the resource allocation strategy according to the logistics bottleneck in the simulation result;

[0064] An agent adjustment module is used to treat ships, berths, storage yards and loading and unloading equipment as different agents, and dynamically adjust the queue planning through a multi-agent collaborative algorithm to obtain a final collaborative scheduling method;

[0065] The reinforcement optimization module is used to set the reward function, update the state transfer model through reinforcement learning, and realize continuous optimization of the final collaborative scheduling method.

[0066] Compared with the prior art, the embodiments of the present invention have at least the following advantages or beneficial effects:

[0067] (1) The present invention divides the working hours of the whole day into multiple decision-making periods according to the fluctuation characteristics of ship traffic, and establishes a state transition model based on time series, which can dynamically adjust the resource allocation strategy of each period, maximize the utilization rate of port resources, ensure the efficient utilization of resources such as berths, loading and unloading equipment and yards under different task volumes, and reduce resource waste and idle time;

[0068] (2) The present invention can effectively alleviate the problem of logistics congestion, optimize the berthing, loading and unloading and yard utilization processes of ships, and improve the overall operational efficiency of the port by simulating the chain reaction chain and planning the resource allocation strategy according to the simulation results;

[0069] (3) The present invention regards ships, berths and loading and unloading equipment as different intelligent agents, and dynamically adjusts the queue planning through a multi-agent collaborative algorithm, which can give full play to the advantages of each intelligent agent, realize flexible resource scheduling and collaborative operation, and further improve the utilization efficiency and operational flexibility of port resources;

[0070] (4) By setting a reward function and combining it with reinforcement learning, the present invention can continuously update the state transition model and realize continuous optimization of the final collaborative scheduling method. It can continuously adjust and optimize the resource allocation strategy according to the actual data and dynamic changes of port operations to ensure the long-term efficiency and adaptability of port operations. BRIEF DESCRIPTION OF THE DRAWINGS

[0071] Figure 1 The present invention is a flowchart of the steps of a method for regional coordination and optimization of shipping centers based on big data in an embodiment of the present invention.

[0072] Figure 2 It is a logical schematic diagram of a shipping center area coordination optimization system based on big data in an embodiment of the present invention. DETAILED DESCRIPTION

[0073] The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0074] Reference Figure 1 As shown, the present invention provides a method for regional coordination optimization of shipping centers based on big data, including:

[0075] S100 divides the working hours of the whole day into multiple decision-making periods according to the fluctuation characteristics of ship traffic;

[0076] It needs to be explained that by analyzing its fluctuation characteristics and dividing the whole day into multiple decision-making periods, port resources and ship demand can be matched more accurately. Different ship flow characteristics are corresponding to each period, which provides a refined basis in the time dimension for the formulation of subsequent resource allocation strategies, helping to improve port resource utilization and reduce operating costs.

[0077] In the actual evaluation, taking a certain shipping center as an example, the shipping center has 73 berths, a total of 265 loading and unloading equipment of various types, and a total yard area of ​​68,450 square meters. In a certain week, the estimated arrival time data of 287 ships were collected. The estimated arrival times of these ships were arranged in sequence to form an ordered sample sequence. By minimizing the intra-group variance and maximizing the difference in ship flow between different time periods, 6 optimal segmentation points were determined, and the whole day was divided into 7 decision-making periods, namely: 0-3:55, 3:55-6:20, 6:20-9:45, 9:45-13:10, 13:10-16:35, 16:35-19:50, and 19:50-24:00.

[0078] S200 establishes a state transition model based on time series, and iterates the state transition model under constraints to maximize port resource utilization, thereby obtaining a resource allocation strategy for each decision period;

[0079] It needs to be explained that in actual operation, port resources are subject to multiple constraints, so the constraints need to be considered. By iteratively optimizing the state transition model and maximizing the utilization rate of port resources, the resource allocation strategy can be dynamically adjusted according to the specific conditions of different decision-making periods, which helps to achieve reasonable allocation of resources, make full use of the port's berths, equipment and yard resources, avoid idle or excessive resources, and improve the turnover efficiency of port resources;

[0080] In the actual evaluation, a state transition model based on time series was established, and the berth quantity constraint was obtained as 73, the maximum operating capacity of the equipment was 2650 hours, and the total area of ​​the yard was 68450 square meters. In order to maximize the objective function, the termination condition was met after 42 iterations, and the resource allocation strategy for each decision period was obtained. Among them, the resource allocation strategy for the time period of 16:35-19:50 is as follows: there are 73 berths in total, about 40 of which are occupied, large container ships are allocated to berths 15-18, and bulk carriers are allocated to berths 22-24; among the 265 equipment, 32 are operating equipment, 3 quay cranes and 5 gantry cranes are equipped for container ships, and 2 grab cranes and a belt conveyor system are equipped for bulk carriers. The remaining operating capacity of the loading and unloading equipment is about 856.3 hours; the yard resources have a total area of ​​68450 square meters, and about 21870 square meters are left.

[0081] S300 predicts the probability of occurrence of risks in the resource allocation strategy to obtain a chain reaction chain;

[0082] It needs to be explained that the operation of shipping centers is complex and there are many uncertainties in the implementation of resource allocation strategies. By predicting the probability of risk occurrence, potential risks can be warned in advance, and then the Bayesian network model can be constructed using the association mining algorithm to obtain the chain reaction chain, clarify the association and transmission path between risks, and provide a basis for formulating response and scheduling measures to avoid risks, thereby enhancing the stability and reliability of shipping center operations.

[0083] In the actual assessment, historical risk data of the past year and the corresponding resource allocation strategies were collected as training sets, with a total of 478 groups of data. The convolutional neural network risk prediction model was trained, and the resource allocation strategy of the decision-making period was input into the model to output the probability of risk occurrence. The Bayesian network model was constructed using the association mining algorithm. After inputting the risk probability, a chain reaction chain was obtained, which included: when the berth occupancy rate reaches more than 75%, there is a 45% probability that the ship waiting time will exceed 2.5 hours, which may reduce the utilization rate of loading and unloading equipment by 10.5%.

[0084] S400 simulates the chain reaction chain in the order of low-volume transportation to high-volume transportation, and adds queue planning to the resource allocation strategy according to the logistics bottleneck in the simulation result;

[0085] It should be explained that this step found logistics bottlenecks such as insufficient loading and unloading equipment and insufficient yard space based on the simulation results. Adding queue planning based on these bottlenecks can effectively alleviate resource tension, reasonably allocate resources, improve logistics efficiency, and ensure the smooth operation of the shipping center under different task volumes;

[0086] It should be understood that the purpose of simulating in sequence from low-volume transportation to high-volume transportation is to more effectively identify and alleviate logistics bottlenecks. Low-volume transportation usually requires less resources. Priority simulation can help determine the operating status of the system when resources are relatively sufficient, thereby providing a benchmark for the simulation of high-volume transportation. As the volume of tasks increases, resource requirements rise. This sequential simulation helps to gradually reveal bottleneck problems that may occur when resources are tight.

[0087] In the actual evaluation, in the logistics bottleneck simulation, the Monte Carlo simulation method was used to simulate 60 times. During the decision period, the transportation task volume was sorted from low to high. The simulation found that there was a shortage of loading and unloading equipment during the 9:45-13:10 period, and there was a shortage of yard space during the 16:35-17:50 period. According to the transportation priority, the transportation tasks in the logistics bottleneck link were queued. Among them, during the 9:45-13:10 period when there was a shortage of loading and unloading equipment, there were 15 ships waiting for loading and unloading, and there were 5 berths for loading and unloading. The average queue length for each berth was 3.

[0088] S500 regards ships, berths, yards and loading and unloading equipment as different agents, and dynamically adjusts the queue planning through a multi-agent collaborative algorithm to obtain the final collaborative scheduling method;

[0089] It needs to be explained that, using the multi-agent collaborative algorithm, agents can communicate and collaborate with each other. Through information exchange and collaboration between agents, the queue planning can be flexibly adjusted according to their respective status and needs, which helps to make full use of the resources of each agent, making the ship berthing, loading and unloading operations and cargo storage more efficient and coordinated, overcoming the limitations of single resource allocation, and ultimately obtaining a better collaborative scheduling method;

[0090] In the actual evaluation, state variables and action spaces are assigned to the ship, berth, yard and loading and unloading equipment agents. The current resource occupancy of a ship agent is 68% of the cargo capacity and it is waiting for a berth. The action space is to issue a docking request to a specific berth. The agents negotiate through communication. The ship agent sends a docking request to the berth agent. The berth agent responds to the request based on its own resource occupancy (currently there are 10 free berths). For the period from 9:45 to 13:10 when there is a shortage of loading and unloading equipment, through multi-agent cooperation, an additional 5 loading and unloading equipment are allocated to it during the period from 10:20 to 13:10, and the queue planning is optimized to 2.

[0091] S600 sets a reward function, updates the state transfer model through reinforcement learning, and realizes continuous optimization of the final collaborative scheduling method.

[0092] It should be explained that the reward function is set to quantitatively evaluate the pros and cons of the scheduling results. Deep Q network reinforcement learning is used to maximize the reward function, store experience in the experience replay pool, and update network parameters through training, thereby updating the state transition model. Through continuous optimization, decisions can be learned and adjusted to improve adaptability to complex shipping environments, and ultimately achieve continuous optimization of the final collaborative scheduling method of the shipping center;

[0093] In the actual evaluation, deep Q-network reinforcement learning was adopted, 55 samples were sampled from the experience replay pool, and the discount factor was 0.80. Through continuous training, the action decisions were mapped to the decision variables of the state transfer model to achieve continuous optimization of the final collaborative scheduling method. After 80 training sessions, the port resource utilization rate increased from 0.68 to 0.72, and the average waiting time of ships was reduced from 1.15 hours to 0.9 hours.

[0094] In this embodiment, the method of dividing the full-day working hours into multiple decision-making periods according to the fluctuation characteristics of ship traffic includes:

[0095] Obtain the estimated arrival time of the ship and arrange it in chronological order to form an ordered sample sequence;

[0096] For any time period in the ordered sample sequence, the intra-group difference is calculated, and the expression is:

[0097]

[0098] in, Indicates time period The intra-group differences, and are the start and end time of the time period, For the Ship traffic in a certain time period, For time period Average vessel traffic within the area;

[0099] The optimal split point is determined by minimizing the variance within the group and maximizing the difference in ship flow between different time periods;

[0100] According to the optimal segmentation point, the whole day is divided into multiple decision periods, each of which corresponds to different ship flow characteristics.

[0101] In this embodiment, the method for establishing a state transition model based on time series includes:

[0102] The port resource status, including berth occupancy, remaining operating capacity of loading and unloading equipment and remaining space in the yard, is taken as the state variable; the resource allocation scheme for different ships and cargoes, including berth allocation, equipment scheduling and manpower arrangement, is taken as the decision variable;

[0103] The state transition equation based on time series is constructed according to the state variables and decision variables. The expression is:

[0104]

[0105]

[0106]

[0107] in, represents the decision period, , is the total number of decision periods, and In the and Berth occupancy during each decision period, For the The ship arrival rate in the decision period, For the The ship departure rate in the decision period, For the The set of arriving ships in the decision period, For allocation to ships The berth occupancy ratio For the The number of available berths in a decision period, and Respectively and The remaining operating capacity of the equipment in each decision period, For the The set of working devices in the decision period, For equipment In the The operation duration requirement for each decision period, For the The average operating efficiency of the equipment in the decision period, For the The remaining operating capacity of the equipment during each decision period, and In the and The remaining space in the yard during the decision period, For ships The area of ​​the yard occupied by the cargo, For the The utilization efficiency of the yard space in each decision period, For the The space released in the yard during each decision period.

[0108] In this embodiment, the method of iterating the state transition model to maximize the port resource utilization under the constraint condition to obtain the resource allocation strategy for each decision period includes:

[0109] According to the actual physical limitations of the port, the constraints of the number of berths, the maximum operating capacity of equipment and the total area of ​​the yard are obtained;

[0110] Taking maximizing port resource utilization as the objective function, the expression is:

[0111]

[0112] in, is the total number of decision periods, , and are weight coefficients, , and Respectively The number of occupied berths, equipment operating hours and remaining space in the yard for each decision period. , and They are the total number of berths, the maximum operating time of loading and unloading equipment and the maximum yard space. For the The number of ships arriving at the port during the decision period, For the The first decision period The cargo volume of the ship, For the The maximum cargo capacity of a vessel;

[0113] Taking the objective function as the iteration direction, under the constraints, the state variables and decision variables are updated according to the state transition model until the termination conditions are met;

[0114] The termination condition is that the rate of change of the objective function is less than 0.001 in 10 consecutive iterations.

[0115] In this embodiment, the method of predicting the probability of occurrence of risks in the resource allocation strategy and obtaining a chain reaction chain includes:

[0116] Construct a risk prediction model through a convolutional neural network, obtain historical risk data and corresponding historical resource allocation strategies as training sets, and train the risk prediction model;

[0117] The resource allocation strategy at each decision moment is input into the risk prediction model, and the probability of risk occurrence is output, which is expressed as a probability density function. The expression is:

[0118]

[0119] in, and are resource allocation strategies and model parameters, respectively. and Given a resource allocation strategy, Under this condition, the expected value and standard deviation of the probability of risk occurrence are: is the nonlinear adjustment coefficient;

[0120] The association mining algorithm is used to identify the connection between risk factors in historical risk data, and a Bayesian network model is constructed. The risk probability is input into the Bayesian network model to obtain a chain reaction chain.

[0121] In this embodiment, the chain reaction chain is simulated in the order of low-task volume transportation to high-task volume transportation, and the method of adding queuing planning to the resource allocation strategy according to the logistics bottleneck in the simulation result includes:

[0122] The logistics bottlenecks include the shortage of berths, loading and unloading equipment and yard space, as well as the congestion of ship arrival, cargo loading and unloading and yard management;

[0123] The chain reaction is simulated by Monte Carlo simulation method in the order of transportation task volume from low to high, and the logistics bottleneck links that appear in each decision period are obtained;

[0124] The transportation tasks in the logistics bottleneck links are queued in descending order of transportation priority, and the queue length of each logistics bottleneck link is averaged.

[0125] In this embodiment, the method of dynamically adjusting the queue planning through a multi-agent collaborative algorithm to obtain a final collaborative scheduling method includes:

[0126] Give the agent state variables and action space;

[0127] The state variables include the resource occupancy of the agent;

[0128] The action space is the current task assignment of the agent;

[0129] Each agent communicates and negotiates with other agents through a multi-agent collaborative algorithm. An agent sends resource requests to other agents based on the current task allocation and its own resource occupancy. Other agents respond to the request if there are idle resources based on their own resource occupancy.

[0130] In each decision period, the queue planning is dynamically adjusted according to the interaction between agents.

[0131] In this embodiment, the method of setting a reward function and updating the state transfer model through reinforcement learning to achieve continuous optimization of the final collaborative scheduling method includes:

[0132] Taking port resource utilization, ship waiting time, loading and unloading equipment utilization and berth turnover efficiency as evaluation indicators, a reward function is constructed, and the expression is:

[0133]

[0134] in, is the reward value, , , and are the weight coefficients of port resource utilization, ship waiting time, loading and unloading equipment utilization and berth turnover time, respectively. is the port resource utilization rate, is a natural constant, is the average waiting time of the ship, is the target value of the ship waiting time, is the fluctuation coefficient of loading and unloading equipment utilization, is the utilization rate of loading and unloading equipment, is the average berth turnover time, is the target value for berth turnover time;

[0135] A deep Q network is used to maximize the reward function for reinforcement learning, where the loss function of the training process is expressed as:

[0136]

[0137] in, is the loss function, are the main Q network parameters, is the expected value, which means averaging all samples sampled from the experience replay pool. is a sample, including the state , actions taken , get instant rewards and the next state , For the experience replay pool, Indicates taking action Back from state Transfer to state Instant rewards received, is the discount factor, For the next state All possible actions The maximum value of Q value, are the parameters of the target Q network;

[0138] The action decisions obtained from reinforcement learning training are mapped to the decision variables in the state transfer model, the state transfer model is updated, and then the final collaborative scheduling method is updated to achieve continuous optimization.

[0139] The second aspect of the present invention also provides a shipping center regional coordination optimization system based on big data, comprising:

[0140] The time division module is used to divide the working hours of the whole day into multiple decision-making periods according to the fluctuation characteristics of ship traffic;

[0141] The strategy iteration module establishes a state transition model based on time series, iterates the state transition model under constraints to maximize the utilization rate of port resources, and obtains the resource allocation strategy for each decision period;

[0142] A risk prediction module, used to predict the probability of risk occurrence in the resource allocation strategy and obtain a chain reaction chain;

[0143] A bottleneck planning module is used to simulate the chain reaction chain in the order of low-volume transportation to high-volume transportation, and add queue planning to the resource allocation strategy according to the logistics bottleneck in the simulation result;

[0144] An agent adjustment module is used to treat ships, berths, storage yards and loading and unloading equipment as different agents, and dynamically adjust the queue planning through a multi-agent collaborative algorithm to obtain a final collaborative scheduling method;

[0145] The reinforcement optimization module is used to set the reward function, update the state transfer model through reinforcement learning, and realize continuous optimization of the final collaborative scheduling method.

[0146] The above contents are merely examples and explanations of the structure of the present invention. The technicians in this technical field may make various modifications or additions to the specific embodiments described or replace them in a similar manner. As long as they do not deviate from the structure of the invention or exceed the scope defined by the claims, they should all fall within the protection scope of the present invention.

Claims

1. A method for regional coordination optimization of shipping centers based on big data, characterized in that: The following steps are involved: Divide the working hours of the whole day into multiple decision-making periods according to the fluctuation characteristics of ship traffic; Establish a state transition model based on time series, iterate the state transition model to maximize the utilization rate of port resources under constraints, and obtain the resource allocation strategy for each decision period; Predicting the probability of occurrence of risks in the resource allocation strategy and obtaining a chain reaction chain; Simulating the chain reaction chain in the order of low-volume transportation to high-volume transportation, and adding queuing planning to the resource allocation strategy according to the logistics bottleneck in the simulation results; Taking ships, berths, storage yards and loading and unloading equipment as different intelligent agents, the queue planning is dynamically adjusted through a multi-agent collaborative algorithm to obtain a final collaborative scheduling method; A reward function is set, and the state transfer model is updated through reinforcement learning to achieve continuous optimization of the final collaborative scheduling method.

2. According to the big data-based shipping center regional coordination optimization method of claim 1, it is characterized by: The method of dividing the full-day working hours into multiple decision-making periods according to the fluctuation characteristics of ship traffic includes: Obtain the estimated arrival time of the ship and arrange it in chronological order to form an ordered sample sequence; For any time period in the ordered sample sequence, the intra-group difference is calculated, and the expression is: in, Indicates time period The within-group differences, and are the start and end time of the time period, For the Ship traffic in a certain time period, For time period Average vessel traffic within the area; The optimal split point is determined by minimizing the variance within the group and maximizing the difference in ship flow between different time periods; According to the optimal segmentation point, the whole day is divided into multiple decision periods, each of which corresponds to different ship flow characteristics.

3. According to the big data-based shipping center regional coordination optimization method of claim 1, it is characterized by: The method for establishing a state transition model based on time series includes: The port resource status, including berth occupancy, remaining operating capacity of loading and unloading equipment and remaining space in the yard, is taken as the state variable; the resource allocation scheme for different ships and cargoes, including berth allocation, equipment scheduling and manpower arrangement, is taken as the decision variable; The state transition equation based on time series is constructed according to the state variables and decision variables. The expression is: in, represents the decision period, , is the total number of decision periods, and In the and The number of occupied berths in a decision period, For the The ship arrival rate in the decision period, For the The ship departure rate in the decision period, For the The set of arriving ships in the decision period, For allocation to ships The berth occupancy ratio For the The number of available berths in a decision period, and Respectively and Equipment availability during each decision period, For the The set of working devices in the decision period, For equipment In the The operation duration requirement for each decision period, For the The average operating efficiency of the equipment in the decision period, For the The remaining operating capacity of the equipment during each decision period, and In the and The remaining space in the yard during the decision period, For ships The area of ​​the yard occupied by the cargo, For the The utilization efficiency of the yard space in each decision period, For the The space released in the yard during each decision period.

4. According to the big data-based shipping center regional coordination optimization method of claim 1, it is characterized by: The method of iterating the state transition model under the constraint conditions to maximize the utilization rate of port resources and obtaining the resource allocation strategy for each decision period includes: According to the actual physical limitations of the port, the constraints of the number of berths, the maximum operating capacity of equipment and the total area of ​​the yard are obtained; Taking maximizing port resource utilization as the objective function, the expression is: in, is the total number of decision periods, , and are weight coefficients, , and Respectively The number of occupied berths, equipment operating hours and remaining space in the yard for each decision period. , and They are the total number of berths, the maximum operating time of loading and unloading equipment and the maximum yard space. For the The number of ships arriving at the port during the decision period, For the The first decision period The cargo volume of the ship, For the The maximum cargo capacity of a vessel; Taking the objective function as the iteration direction, under the constraints, the state variables and decision variables are updated according to the state transition model until the termination conditions are met; The termination condition is that the rate of change of the objective function is less than 0.001 in 10 consecutive iterations.

5. According to the big data-based shipping center regional coordination optimization method of claim 1, it is characterized by: The method of predicting the probability of occurrence of risks in the resource allocation strategy and obtaining a chain reaction chain includes: Construct a risk prediction model through a convolutional neural network, obtain historical risk data and corresponding historical resource allocation strategies as training sets, and train the risk prediction model; The resource allocation strategy at each decision moment is input into the risk prediction model, and the probability of risk occurrence is output, which is expressed as a probability density function. The expression is: in, and are resource allocation strategies and model parameters, respectively. and Given a resource allocation strategy, Under this condition, the expected value and standard deviation of the probability of risk occurrence are: is the nonlinear adjustment coefficient; The association mining algorithm is used to identify the connection between risk factors in historical risk data, and a Bayesian network model is constructed. The risk probability is input into the Bayesian network model to obtain a chain reaction chain.

6. The method for regional coordination optimization of shipping centers based on big data according to claim 1 is characterized in that: The method of simulating the chain reaction chain in the order of low-task volume transportation to high-task volume transportation, and adding queuing planning to the resource allocation strategy according to the logistics bottleneck in the simulation result, comprises: The logistics bottlenecks include the shortage of berths, loading and unloading equipment and yard space, as well as the congestion of ship arrival, cargo loading and unloading and yard management; The chain reaction is simulated by Monte Carlo simulation method in the order of transportation task volume from low to high, and the logistics bottleneck links that appear in each decision period are obtained; The transportation tasks in the logistics bottleneck links are queued in descending order of transportation priority, and the queue length of each logistics bottleneck link is averaged.

7. The method for regional coordination optimization of shipping centers based on big data according to claim 1 is characterized in that: The method of dynamically adjusting the queue planning through a multi-agent collaborative algorithm to obtain a final collaborative scheduling method includes: Give the agent state variables and action space; The state variables include the resource occupancy of the agent; The action space is the current task assignment of the agent; Each agent communicates and negotiates with other agents through a multi-agent collaborative algorithm. An agent sends resource requests to other agents based on the current task allocation and its own resource occupancy. Other agents respond to the request if there are idle resources based on their own resource occupancy. In each decision period, the queue planning is dynamically adjusted according to the interaction between agents.

8. The method for regional coordination optimization of shipping centers based on big data according to claim 1 is characterized in that: The method of setting a reward function, updating the state transfer model through reinforcement learning, and realizing continuous optimization of the final collaborative scheduling method includes: Taking port resource utilization, ship waiting time, loading and unloading equipment utilization and berth turnover efficiency as evaluation indicators, a reward function is constructed, and the expression is: in, is the reward value, , , and are the weight coefficients of port resource utilization, ship waiting time, loading and unloading equipment utilization and berth turnover time, respectively. is the port resource utilization rate, is a natural constant, is the average waiting time of the ship, is the target value of the ship waiting time, is the fluctuation coefficient of loading and unloading equipment utilization, is the utilization rate of loading and unloading equipment, is the average berth turnover time, is the target value for berth turnover time; A deep Q network is used to maximize the reward function for reinforcement learning, where the loss function of the training process is expressed as: in, is the loss function, are the main Q network parameters, is the expected value, which means averaging all samples sampled from the experience replay pool. is a sample, including the state , actions taken , get instant rewards and the next state , For the experience replay pool, Indicates taking action Back from state Transfer to state Instant rewards received, is the discount factor, For the next state All possible actions The maximum value of Q value, are the parameters of the target Q network; The action decisions obtained from reinforcement learning training are mapped to the decision variables in the state transfer model, the state transfer model is updated, and then the final collaborative scheduling method is updated to achieve continuous optimization.

9. A system for coordinating and optimizing shipping center regions based on big data, used to execute a method for coordinating and optimizing shipping center regions based on big data as claimed in any one of claims 1 to 8, characterized in that: The system comprises: The time division module is used to divide the working hours of the whole day into multiple decision-making periods according to the fluctuation characteristics of ship traffic; The strategy iteration module establishes a state transition model based on time series, iterates the state transition model under constraints to maximize the utilization rate of port resources, and obtains the resource allocation strategy for each decision period; A risk prediction module, used to predict the probability of risk occurrence in the resource allocation strategy and obtain a chain reaction chain; A bottleneck planning module is used to simulate the chain reaction chain in the order of low-volume transportation to high-volume transportation, and add queue planning to the resource allocation strategy according to the logistics bottleneck in the simulation result; An agent adjustment module is used to treat ships, berths, storage yards and loading and unloading equipment as different agents, and dynamically adjust the queue planning through a multi-agent collaborative algorithm to obtain a final collaborative scheduling method; The reinforcement optimization module is used to set the reward function, update the state transfer model through reinforcement learning, and realize continuous optimization of the final collaborative scheduling method.

Images