A port area collection and distribution automatic driving vehicle platoon task scheduling method and system

Abstract

The application provides a kind of pan-port area collection and distribution automatic driving vehicle formation task scheduling method and system, it is related to intelligent transportation system and port logistics technical field.The method includes system initialization and data acquisition to construct pan-port area digital twin model and global state space;Based on K-means clustering, task grouping and preliminary allocation are completed;Fusion multi-agent deep reinforcement learning MADRL model and auction mechanism realize vehicle collaborative bidding and decision optimization;Based on the action sequence output by auction, dynamic formation and path planning are carried out;Based on the gate reservation decision output by MADRL model, gate intelligent allocation and execution are carried out;Through performance evaluation and safety guarantee by executing monitoring and adaptive optimization.The application effectively promotes vehicle collaborative formation, realizes peak shaving in peak period, significantly improves the efficiency and safety of pan-port area collection and distribution, reduces energy consumption and carbon emissions, and is suitable for large container port pan-port area scene.

Description

Technical Field

[0001] This invention relates to the fields of intelligent transportation systems and port logistics technology, specifically to a method and system for scheduling autonomous vehicle platooning for collection and distribution in a port area. Background Technology

[0002] A "pan-port area" refers to a smaller port area comprised of multiple wharves and storage yards. The pan-port area's transport system is responsible for the transportation and scheduling of containers from the wharves and storage yards to the subsequent transport center. Some large domestic ports can handle up to 47.303 million TEUs annually, with road transport accounting for nearly 50%, leading to peak-hour traffic volumes on key routes approaching their capacity limits. Uneven distribution of external truck arrivals, coupled with concentrated arrivals during peak periods, causes road congestion, extended gate waiting times, and low yard operation efficiency.

[0003] Currently, there is a lack of dynamic scheduling, platoon management, and gate optimization mechanisms for collection and distribution vehicles, which fails to effectively achieve peak shaving and valley filling, leading to increased operational safety risks and overall low efficiency. Existing patents mostly focus on single vehicle control or route planning, lacking comprehensive solutions that integrate task scheduling, multi-vehicle collaboration, and gate allocation.

[0004] This invention aims to address the aforementioned pain points by enabling dynamic platooning and scheduling of autonomous vehicles for collection and distribution, spatiotemporal path planning, and intelligent gate allocation, thereby improving the safety and efficiency of the collection and distribution system in the Pan-Port Area. Summary of the Invention

[0005] This invention addresses the technical problems of traffic congestion, low gate utilization, and low operational efficiency in the port area during peak hours in existing technologies. It provides a method and system for scheduling automated vehicle platooning in the port area, which effectively promotes vehicle platooning coordination, reduces empty mileage, and lowers gate waiting time, achieving peak-shaving and valley-filling during peak hours. This significantly improves the efficiency and safety of port area transportation, resulting in substantial economic and social benefits.

[0006] To achieve the above objectives, the technical solution adopted by the present invention is as follows: A method for scheduling autonomous vehicle platooning for collection and distribution in a port area, the method comprising: Perform system initialization and data acquisition to construct a digital twin model of the Pan-Port Area and a global state space; Task grouping and initial assignment were completed based on K-means clustering; A multi-agent deep reinforcement learning (MADRL) model is integrated with an auction mechanism to achieve collaborative bidding and decision optimization for vehicles. Dynamic formation and path planning are performed based on the action sequence output by the auction. Based on the MADRL model, gate reservation decisions are output, and gates are intelligently allocated and executed. Performance evaluation and security assurance are achieved through monitoring and adaptive optimization.

[0007] On the other hand, the present invention also provides a platooning task scheduling system for automated vehicle platooning in the pan-port area, which is used to implement the above-mentioned platooning task scheduling method for automated vehicle platooning in the pan-port area. The system includes: System initialization and data acquisition unit: provides basic scenario models and full-dimensional real-time data support for the entire scheduling process, and constructs a digital twin environment and global state space for the Pan-Port Area; Task grouping and preliminary allocation unit: Based on container attributes and time constraints, it realizes efficient clustering and preliminary allocation of tasks, providing structured task input for the subsequent auction mechanism; The auction mechanism unit integrating MADRL: By integrating multi-agent deep reinforcement learning with the auction mechanism, dynamic bidding allocation of tasks and optimization of vehicle collaborative decision-making are achieved; Dynamic platooning and route planning unit: Based on the auction results, a reasonable vehicle platoon is formed, and the optimal driving route is generated by combining spatiotemporal constraints and congestion prediction; Intelligent gate allocation and execution unit: realizes the optimal allocation of gate resources and the execution of scheduling instructions, ensuring efficient and smooth transportation processes; Execution monitoring and adaptive optimization unit: performs full-process monitoring, performance evaluation and security assurance of the scheduling execution process, and realizes adaptive iterative optimization of the system.

[0008] Compared with the prior art, the beneficial effects of the present invention are: By integrating the auction mechanism of Multi-Agent Deep Reinforcement Learning (MADRL), dynamic bidding and cooperative allocation among multiple vehicles are achieved, overcoming the limitations of single-vehicle scheduling or static allocation in existing technologies. This effectively promotes the collaborative formation of autonomous vehicles, enabling fully loaded one-way driving and rapid reverse reorganization, and significantly reducing the round-trip mileage of empty vehicles (by more than 30%). By introducing a vehicle-to-vehicle (V2V) communication protocol to dynamically adjust platoon spacing, and combining a variant of the ant colony algorithm with Monte Carlo simulation to optimize routes and gate reservations, peak-hour "peak shaving and valley filling" is achieved, significantly alleviating congestion on key passages such as the Donghai Bridge. The average waiting time at the gates is reduced to less than 5 minutes, and the utilization rate is increased by more than 20%. Through real-time learning and feedback mechanisms, the system's adaptability to uncertain traffic environments and task changes is improved, the success rate of vehicle platooning is increased to over 80%, and the overall collection and distribution efficiency and operational safety are significantly improved. It integrates multiple modules such as task grouping, path planning, and gate management for comprehensive optimization, filling the gap in existing technologies that lack multi-vehicle collaboration, spatiotemporal path optimization, and intelligent gate allocation. It is suitable for large container port pan-area scenarios, has higher economic and social benefits, and helps to reduce energy consumption and carbon emissions.

[0009] Other features and advantages of the embodiments of the present invention will be described in detail in the following detailed description section. Attached Figure Description

[0010] Figure 1 This is a flowchart of the autonomous vehicle platooning task scheduling method for the collection and distribution of goods in the Pan-Port Area according to the present invention; Figure 2 This is a schematic diagram of the overall system architecture according to the present invention; Figure 3 This is a schematic diagram of the functional module structure according to the present invention; Figure 4 This is a schematic diagram of the global state space construction according to the present invention; Figure 5 This is a schematic diagram illustrating the integration principle of MADRL and auction mechanism according to the present invention; Figure 6 This is a schematic diagram of vehicle dynamic formation according to the present invention; Figure 7 This is a schematic diagram of gate scheduling and time window allocation according to the present invention; Figure 8 This is a schematic diagram of a simulation scene according to the present invention; Figure 9 This is a schematic diagram showing the performance comparison results according to the present invention; Figure 10 This is a comparison diagram of vehicle platooning effects according to the present invention; Figure 11 This is a diagram illustrating the effect of traffic flow peak shaving and valley filling according to the present invention. Detailed Implementation

[0011] To enable those skilled in the art to better understand the technical solutions of this invention, the technical solutions of the embodiments of this invention will be clearly and completely described below with reference to the accompanying drawings, so as to more clearly understand the purpose, features and advantages of this invention. It should be understood that the embodiments shown in the drawings are not intended to limit the scope of this invention, but are only for illustrating the essential spirit of the technical solutions of this invention. Obviously, the described embodiments are only a part of the embodiments of this invention, and not all of them. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of this invention.

[0012] Unless the context requires otherwise, throughout the specification and claims, the word “comprising” and its variations, such as “including” and “having”, shall be understood to have an open, inclusive meaning, that is, to be interpreted as “including, but not limited to”.

[0013] Throughout this specification, references to "an embodiment" or "an embodiment" indicate that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment. Therefore, the appearance of "in an embodiment" or "an embodiment" in various places throughout the specification does not necessarily refer to the same embodiment. Furthermore, a particular feature, structure, or characteristic may be combined in any manner in one or more embodiments.

[0014] The singular forms “a” and “the” used in this specification and the appended claims include plural references unless otherwise expressly stated herein. It should be noted that the term “or” is generally used to mean “and / or” unless otherwise expressly stated herein.

[0015] In the following description, in order to clearly demonstrate the structure and working method of the present invention, a number of directional terms will be used. However, terms such as "front", "back", "left", "right", "outside", "inside", "outward", "inward", "up", and "down" should be understood as convenient terms and not as limiting terms.

[0016] The implementation details of the embodiments of the present invention will be described in detail below with reference to the accompanying drawings. The following content is only for the convenience of understanding the implementation details and is not necessary for implementing this solution.

[0017] Example 1 To address the aforementioned technical issues, this invention discloses a method for scheduling autonomous vehicle platooning tasks in a port-wide distribution area. This method utilizes an auction mechanism based on Multi-Agent Deep Reinforcement Learning (MADRL) to achieve dynamic scheduling and optimization. It aims to achieve efficient container distribution, vehicle platooning collaboration, and minimize empty-load mileage through real-time bidding and learning mechanisms. Specifically, this method targets autonomous vehicles (including external trucks and internal transfer vehicles) within a port-wide area (such as an area consisting of multiple terminals and rear storage yards). It employs a MADRL-driven auction mechanism to simulate bidding cooperation among vehicle agents, thereby optimizing the distribution process. The main optimization objectives include: (1) In the rear yard, containers of the same or similar shipments or the same terminal destination are picked up by dispersed vehicles through dynamic scheduling and formed into a convoy to drive to the destination terminal; (2) At the destination terminal, containers of the same owner or adjacent pick-up times are picked up by dispersed vehicles from multiple terminals and then concentrated into a convoy to drive to the destination yard; (3) Through global path planning and task allocation, empty vehicle round trips on logistics roads are reduced as much as possible to improve the overall system efficiency and safety.

[0018] Figure 1 This is a flowchart of the autonomous vehicle platooning task scheduling method for the Pan-Port Area Collection and Distribution Transportation of the present invention. Figure 1 As shown, the specific steps of the autonomous vehicle platooning task scheduling method for the collection and distribution of goods in the Pan-Port Area are as follows: S1. Perform system initialization and data acquisition; System initialization is a fundamental supporting step in the entire autonomous vehicle platooning task scheduling process for the entire port area. Its core lies in constructing a digital twin model of the port area, using digital means to completely replicate the physical scene and operational status within the port area. Specifically, this requires the structured abstraction and digital mapping of all key logistics nodes within the port area (including multiple wharves, rear storage yards, and various gates) and the logistics road network connecting these nodes. This not only accurately restores the static attributes of each node, such as spatial layout and facility capacity, but also synchronizes dynamic information such as the operational status of nodes (e.g., container storage locations in storage yards, and the busyness of wharf loading and unloading equipment), road capacity, and real-time traffic conditions, forming a virtual simulation environment that is highly consistent with the physical world and operates in real time.

[0019] like Figure 2 As shown, the overall architecture of the system of this invention consists of a data acquisition layer, a data processing layer, and a scheduling decision layer. The data acquisition layer acquires real-time data through vehicle-mounted terminals, roadside sensors, gate equipment, and yard / terminal equipment; the data processing layer completes data fusion, state space construction, and congestion prediction; the scheduling decision layer realizes task allocation and formation decision based on MADRL and auction mechanism, forming a cloud-edge-terminal collaborative intelligent scheduling system for the pan-port area.

[0020] like Figure 3 As shown, the functional module structure of this invention mainly includes: a system initialization and data acquisition unit, a task grouping and preliminary allocation unit, an auction mechanism unit integrating MADRL, a dynamic formation and path planning unit, a gate intelligent allocation and execution unit, and an execution monitoring and adaptive optimization unit. Each module works together to achieve efficient collection and distribution scheduling.

[0021] Data acquisition is a crucial step in ensuring the scientific and real-time nature of scheduling decisions. It requires building a comprehensive, three-dimensional data acquisition network based on multi-source sensing devices deployed throughout the port area to achieve full capture and real-time updates of core information. The scope of data acquisition must cover key information throughout the entire container lifecycle, including vessel identification, cargo owner tags, preset pickup time windows, and final destination coordinates; dynamic status data of autonomous vehicles, covering core operational indicators such as current real-time location, actual load, operational status (idle / occupied), and remaining range; regional traffic flow data, combining historical traffic data with real-time monitoring data to predict peak-hour congestion trends and provide congestion references for route planning; and gate operation data such as real-time utilization rate and queue length at each gate. In terms of data acquisition methods, sensors distributed in areas such as yards, wharves, and roads capture environmental and facility status; GPS devices on vehicles obtain precise location information; and IoT terminals enable data interconnection and real-time transmission between containers, vehicles, and gate equipment.

[0022] like Figure 4 The diagram shown illustrates the construction of the global state space. After cleaning and integrating all the collected raw data, a standardized global state space S={V, T, N} is continuously updated. Here, V is the vehicle state vector with dimensions corresponding to the total number of vehicles n, T is the task queue matrix with tasks as rows and attributes such as boat number and time as columns, and N is the network load matrix with roadside weights representing the degree of congestion. This provides comprehensive, accurate, and real-time data support for subsequent task grouping, auction mechanism operation, and path optimization.

[0023] S2. Task grouping and initial allocation; Based on container attributes, including vessel similarity, destination consistency, and pickup time window, the K-means clustering algorithm combined with time constraints is used to group tasks. In the rear yard, containers from the same or similar vessels (time difference less than a threshold T) are preferentially assigned to nearby vehicles to form potential platoons. At the terminal, containers from the same shipper or with adjacent pickup times (time window overlap > 50%) are grouped to facilitate subsequent consolidation. This step outputs preliminary task clusters, which serve as input features for the MADRL-driven auction mechanism. The clustering objective function is to minimize the intra-group distance, and its expression is: , Where J is the clustering objective function value, representing the sum of the total intra-group distances of all clusters. The goal is to minimize this value using the K-means algorithm to achieve tight aggregation of similar tasks (i.e., high similarity and low difference between tasks within the group); K is the total number of clusters, i.e., the final number of task groups into which all container tasks are divided, which needs to be dynamically adjusted according to the actual scenario such as the real-time task scale of the port area and vehicle platooning capacity (e.g., 3-5 vehicles / platoon); k is the index of the cluster (from 1 to K), used to traverse each independent task group, for example, k=1 corresponds to the first task cluster; C k Let be the k-th cluster (task group), containing a set of container tasks with similar attributes—specifically, tasks from the same / similar vessel numbers in the backyard and from the same shipper / adjacent pick-up time windows at the terminal. This is the initial task unit formed after clustering; x is the feature vector of a single container task, with dimensions covering the core attributes of the task (spatial + temporal dimensions), such as vessel number identifier, pick-up time window, destination coordinates, shipper label, etc., serving as the basic data unit for clustering calculation; μ k is the cluster center (feature vector) of the kth cluster, and is the mean of all task feature vectors x in this cluster, representing the "core attributes" of this task group (such as average box pickup time, central destination coordinates, etc.), used to measure the difference between tasks and core attributes within the cluster; For a single task x and its cluster center μ k The square of the Euclidean distance is used to quantify the degree of difference between a single task and the core attributes of its task group. The smaller the distance, the higher the similarity between the task and other tasks in the group.

[0024] S3. Construction and training of an auction mechanism integrating MADRL; 1) Agent Definition and Auction Integration: Each autonomous vehicle acts as a MADRL agent, and the total number of agents N equals the overall vehicle cluster size. Local state of each agent. Including position coordinates Current load (Load ratio, 0-1), surrounding vehicle status and congestion indicators (0-1, 1 indicates severe congestion), subset of surrounding agent states .

[0025] The action of the i-th agent This includes generating bid values. Task selection Formation Operations (Join / Leave), Path Selection Make an appointment with the gate .

[0026] like Figure 5 As shown, the auction mechanism is embedded in the MADRL framework: the agent computes the bidding function through an Actor network. : in, Indicates the savings when not in use. For load matching degree; The required load capacity for the mission; The coordinates of the mission destination or pickup point; The penalty for waiting indicates a delay beyond the pickup time window; This refers to the actual time during which the vehicle can perform its tasks. This is the end time of the pickup window.

[0027] The balancing factor is used to weigh the importance of idle time savings against waiting penalties. A Vickrey auction algorithm is used, with the central server selecting a winner and assigning tasks to the winning agent, forming an initial fleet candidate set.

[0028] 2) Network architecture: First, the MADRL variant QMIX of the Actor-Critic framework is adopted, in which the Actor network uses a deep neural network (DNN), including LSTM layers to process temporal data and attention mechanisms to capture inter-vehicle dependencies, and finally generates an action policy π(a|s) that includes optimal bidding strategy, task selection, formation operation, etc.

[0029] The global value function expression for the Critic network evaluation is: Where Q(s,a) is the global value function value, representing the "comprehensive value" brought to the entire Pan-Port Area collection and distribution system (including core objectives such as platooning efficiency, empty mileage, and gate waiting time) after all vehicle agents take action combination a under the current global state s of the system. The higher this value, the more the current action combination conforms to the global optimization objectives (such as coordinated platooning, reduced empty mileage, and improved overall efficiency).

[0030] 'a' represents the combination of actions of all vehicle agents (global action vector), where a i For the i-th agent, the independent actions include bidding value, task selection, grouping operations, etc.

[0031] N is the total number of agents in the MADRL system, which is equivalent to the size of the autonomous vehicle fleet participating in scheduling. That is, the number of agents is equal to the number of schedulable vehicles (i iterates through all agents from 1 to N).

[0032] Let be the local value function value of the i-th agent, representing the agent's local state s. i Take action a i The resulting "individual value". This function only measures the value of a single agent's decision to its own goals (such as completing the task and reducing its own idle time), without considering the synergistic effects with other agents.

[0033] f ( mixing The hybrid function (cooperative correction term) is the core module of the QMIX architecture, used to integrate the local observations and decision dependencies of all agents, correcting the bias of simply accumulating local values. Its core functions are: capturing cooperative relationships between agents (such as conflicts in multiple vehicles bidding for the same task, and cooperative benefits when forming a platoon); avoiding "individual optimality but global suboptimality" (such as a single vehicle successfully bidding for a task but causing an increase in overall idle load); and ultimately ensuring that the global value function Q(s,a) truly reflects the impact of the action combination on the system as a whole, rather than simply the sum of individual values. The input to this function is the local state s of all agents. i and action a i The output is a collaborative correction value, and the specific form needs to be designed in conjunction with the scheduling scenario of the pan-port area (such as formation requirements and gate resource constraints).

[0034] The Critic network evaluates the global value function by integrating the local observations of all agents: first, it accumulates the observations of each agent in its local state s. i Next, execute action a i Local value Q i (s i ,a i Then through the mixing function f (mixing) It corrects the bias of simply accumulating local value, effectively captures the collaborative benefits and conflict costs between agents (such as the resource waste of multiple vehicles bidding for the same task, and the efficiency improvement brought about by formation cooperation), avoids the decision-making problem of "individual optimal but global suboptimal", and thus promotes collaborative cooperation between agents.

[0035] Secondly, the input feature processing method is optimized. Considering the "node-edge" topological characteristics of the port area's logistics and distribution scenarios (with yards and wharves as core logistics nodes and roads as connecting channels), a Graph Neural Network (GNN) is introduced to embed spatiotemporal path information. Yards and wharves are abstracted as network nodes, and logistics roads as connecting edges. The GNN can model the spatial topological dependencies between nodes and edges (such as the direct connectivity between yards and wharves, and the association logic of multiple wharves sharing gates). Simultaneously, it integrates dynamic spatiotemporal information (such as changes in road congestion over time, the temporal characteristics of vehicle trajectories, and task time window constraints). This transforms scattered spatial structure data and dynamic temporal data into a unified, machine-understandable high-dimensional feature vector, providing accurate and comprehensive input support for the Actor network's action generation and the Critic network's value assessment. This ensures that network decisions align with the spatial constraints and dynamic changes of the actual port area scenario.

[0036] 3) Training process: Offline training is conducted in a simulated environment (based on historical data such as the throughput of a port in 2022 and the proportion of highway traffic). An experience replay buffer is used to store trajectories (s, a, r, s') (the basic unit describing the interaction between the agent and the environment, where s is the state, a is the action, r is the reward, and s' is the next state), and updates are performed using Q-learning. ,in The discount factor (0 < γ < 1) is used to discount future rewards. .

[0037] The reward function is designed as follows: in: This represents the actual number of vehicles in the platoon. Target formation size; This represents the unloaded driving distance during the current mission. To the maximum allowable unloaded distance; This refers to the actual waiting time at the gate. Maximum allowed waiting time; Reward weighting coefficient These correspond to platooning efficiency, reduction of empty mileage, and minimization of gate waiting time, respectively, satisfying the following requirements: =1; p The penalty coefficient is... For collision / congestion risk indicators (optional safety penalty, introducing negative rewards when safety risks exist).

[0038] In addition, an exploration strategy was introduced. -greedy, exploring probability The decay rate starts from 0.1 to handle uncertainty. Furthermore, during online deployment, the model is fine-tuned through real-time feedback, making small, targeted adjustments and optimizations to the parameters of the offline-trained MADRL (Multi-Agent Deep Reinforcement Learning) model. This achieves "peak shaving and valley filling"—distributing tasks during peak periods and concentrating them during off-peak periods, while optimizing auction bidding to encourage collaboration.

[0039] 4) Optimized Target Integration: MADRL prioritizes fleet formation through reward design: In the rear yard, inter-agent communication (shared status) consolidates containers from the same / nearly vessels, reducing vehicle round trips; at the terminal, agents coordinate to merge scattered container pickup tasks into fleets, calculating consolidation benefits for tasks from the same shipper or adjacent time windows, saving empty distances. <5km. Global optimization reduces empty vehicle round trips.

[0040] S4. Dynamic Formation and Path Planning; like Figure 6 As shown, based on the action sequence output by the MADRL auction, vehicle platooning is formed (e.g., lead-follower mode, utilizing V2V communication to ensure safe spacing). Spatiotemporal path planning considers time window constraints (box pickup time ±...). Given t) and road capacity, GNN is used to predict congestion and generate optimal route sequences. The platooning priority is: same vessel > similar vessels > same destination, ensuring a moderate platoon size of 3-5 vehicles to avoid excessively long platoons affecting maneuverability. Path cost function: ,in For edge distance, For congestion weight, This represents the congestion sensitivity coefficient.

[0041] S5. Intelligent gate allocation and execution; like Figure 7 As shown, the MADRL model outputs gate reservation decisions, prioritizing the allocation of low-load gates to platooned vehicles to reduce waiting time, with the goal of controlling gate waiting time to within 5 minutes. During execution, vehicles drive autonomously according to dispatch instructions, with real-time monitoring and adjustments (if congestion occurs, the agent re-decides to disband or reorganize the platoon). The system continuously updates model parameters based on feedback, achieving adaptive optimization. Safe distance calculation: Typical value s, =2m, For speed.

[0042] S6. Performance Evaluation and Security Assurance.

[0043] like Figure 8 The diagram shows a simulation scenario of the present invention. To verify the effectiveness of the method of the present invention, a simulation scenario was constructed as shown below. Figure 8The simulated port area scenario shown includes four wharves, four gates, a storage yard, and connecting roads. The simulation is based on the actual throughput of a large port in 2022 and the proportion of road transport, setting up two traffic flow patterns: peak and off-peak periods. The simulation compares and tests the traditional method with the method of this invention.

[0044] like Figure 9 As shown, the performance comparison results between the method of the present invention and the traditional method show that: (a) the empty mileage comparison shows that the empty mileage of the method of the present invention is significantly lower than that of the traditional method throughout the entire simulation cycle, especially during peak periods; (b) the throughput efficiency comparison shows that the method of the present invention can still maintain a high collection and distribution throughput efficiency during peak periods, while the traditional method shows a significant decrease, verifying the superiority of the present invention in high-load scenarios.

[0045] like Figure 10 As shown, the left side represents the dispersed state, where vehicles drive independently and weave in disorder; the right side represents the platooning state of the method of this invention, where vehicles drive in an orderly manner in a lead-follow mode, reducing road occupation and traffic conflicts, and improving road utilization and driving safety.

[0046] like Figure 11 As shown, the method of this invention achieves peak-shaving and valley-filling effects on traffic flow. Compared with traditional methods, the peak traffic flow is effectively reduced during peak hours, while the traffic flow level is improved during off-peak hours through concentrated task formation, resulting in a smoother overall traffic flow curve and effectively alleviating peak-hour congestion on key passages in the port area.

[0047] Real-time monitoring of platoon success rate (target > 80%), reduction in empty mileage (target > 30%), gate utilization improvement (target > 20%), and overall throughput efficiency.

[0048] It integrates a collision avoidance algorithm, MADRL negative reward constraints (avoiding dangerous behaviors), and a manual intervention interface, supporting manual intervention when the congestion index is >0.9 or when a safety risk occurs.

[0049] A security mechanism is adopted for adaptive feedback optimization: data during the execution process (such as task completion status, congestion handling results, and gate waiting time) is fed back to the system to update MADRL model parameters, so as to achieve the goal of "distributing tasks during peak periods and concentrating them during off-peak periods" and continuously optimize the auction bidding strategy and collaborative decision-making logic.

[0050] In some embodiments, the present invention also provides a platooning task scheduling system for automated vehicle convoys in the Pan-Port Area for implementing the above-described method, characterized in that the system comprises: System initialization and data acquisition unit: provides basic scenario models and full-dimensional real-time data support for the entire scheduling process, and constructs a digital twin environment and global state space for the Pan-Port Area; Task grouping and preliminary allocation unit: Based on container attributes and time constraints, it realizes efficient clustering and preliminary allocation of tasks, providing structured task input for the subsequent auction mechanism; The auction mechanism unit integrating MADRL: By integrating multi-agent deep reinforcement learning with the auction mechanism, dynamic bidding allocation of tasks and optimization of vehicle collaborative decision-making are achieved; Dynamic platooning and route planning unit: Based on the auction results, a reasonable vehicle platoon is formed, and the optimal driving route is generated by combining spatiotemporal constraints and congestion prediction; Intelligent gate allocation and execution unit: realizes the optimal allocation of gate resources and the execution of scheduling instructions, ensuring efficient and smooth transportation processes; Execution monitoring and adaptive optimization unit: performs full-process monitoring, performance evaluation and security assurance of the scheduling execution process, and realizes adaptive iterative optimization of the system.

[0051] The above units are integrated into edge computing nodes, supporting low-latency decision-making.

[0052] The autonomous vehicle platooning task scheduling method for the port area's collection and distribution is presented in this embodiment. It constructs a digital twin model and global state space for the port area through system initialization and data acquisition. Task grouping and initial allocation are completed based on K-means clustering. Multi-agent deep reinforcement learning (MADRL) and an auction mechanism are integrated to achieve collaborative bidding and decision optimization among vehicles. Dynamic platooning and path planning, and intelligent gate allocation are performed using V2V communication, ant colony algorithm variants, and Monte Carlo simulation. Simultaneously, execution monitoring and adaptive optimization ensure efficient and safe processes. This method effectively promotes collaborative vehicle platooning, reducing empty vehicle round-trip mileage by more than 30%, shortening the average gate waiting time to less than 5 minutes, increasing utilization by more than 20%, and raising the vehicle platooning success rate to over 80%. It achieves peak-shaving and valley-filling during peak periods, significantly improving the efficiency and safety of the port area's collection and distribution, reducing energy consumption and carbon emissions, and is suitable for large container port port scenarios.

[0053] Example 2 In specific implementation, the implementation process of the autonomous vehicle platooning scheduling method for port area collection and distribution provided in this embodiment is as follows. This embodiment assumes that the port area scenario consists of a rear yard group (containing 3 yards) and a front terminal group (containing 4 terminals), with a critical logistics channel capacity of 100 vehicles per hour and a vehicle fleet size of N=50 autonomous external container trucks. The system runs on an edge computing server and integrates a 5G network to support V2V communication. All parameters are optimized based on a simulated environment, and historical data are derived from the typical throughput of large container ports.

[0054] 1. System Initialization and Data Acquisition First, construct a transportation network graph model for the Pan-Port area, G=(V,E), where V is the set of nodes (including yard nodes H1~H3, wharf nodes D1~D4, and gate nodes G1~G4), and E is the set of edges (road segments, with weights based on distance and real-time congestion). c e Vehicle status is collected via GPS sensors: for each vehicle i Location ( , ),load (Initially 0) and idle state. IoT devices collect container task data: Task queue T, including 100 tasks, each task's attribute being a ship number identifier. Cargo owner label , Box pick-up time window [ , [and destination coordinates. Traffic flow prediction uses historical data to calculate peak-hour congestion.] =0.8 (normally 0.2). Forming the global state space S={ , , },in An N×4 matrix (position) x,y ;load l Congestion c ), It is an M×5 matrix (M=100 tasks). For | E |×1 vector. Data is updated every 5 seconds to ensure real-time performance.

[0055] 2. Task Grouping and Initial Assignment The K-means clustering algorithm (K=10, dynamically adjusted according to task size) is used to group tasks, and the distance metric is combined with spatial Euclidean distance. and time overlap rate objective function J for ,in x For task vectors (ship number similarity) if else , T=30min Destination consistency =1 / Time overlap ), It is the cluster center.

[0056] In the rear storage yard, priority is given to grouping the same vessel batch ( or similar ship numbers Tasks such as allocating 10 containers from vessels S001 and S002 to 5 vehicles near yard H1, forming a potential convoy (3-5 vehicles). On the quay side, grouping containers belonging to the same cargo owner. or adjacent time windows ( O time Tasks with a value >0.5, such as merging the 8 tasks of cargo owner C001. Output 10 task clusters as input for subsequent auctions.

[0057] 3. Construction and training of an auction mechanism integrating MADRL Each vehicle acts as a MADRL agent, in status , This represents a subset of the states of other agents within a 5km radius. (Action) .

[0058] Auction ensemble: Agents calculate bids using an Actor network (64 neurons in the input layer, 128 LSTM hidden layers with 128 attention layers, output action probabilities). ,in , (load matching, =1 indicates full load). =0.5; , Δt =15min. The central server uses a Vickrey auction: the winner is... max The second highest payment To ensure genuine bidding. If there is no winner, the winner will be randomly assigned (probability 0.1).

[0059] Network architecture: QMIX variant, Actor Use DNN; (Hybrid network integration of the global). GNN embedding graph G: node embedding , This includes location and load.

[0060] Training: Train offline for 10,000 episodes in the simulator, using an experience replay buffer (capacity 100,000), Q-update. , γ =0.95. Reward ,in w 1 = 0.4, w 2 = 0.3, w 3 = 0.3, p =1; N formed Number of vehicles in the platoon N target Target size (4 vehicles); Dmax =10km, W max =5min. Exploration , It decays from 0.5 to 0.05.

[0061] Optimization: In the rear storage yard, agents bid for the same vessel's mission, forming a convoy to reduce the number of trips per vehicle. <5km); On the dock side, merge adjacent window tasks. Fine-tune online after training, hourly based on real-time... r i Adjust the weights.

[0062] 4. Dynamic Formation and Path Planning Based on the auction output, a platoon is formed: the lead vehicle selects the vehicle with the highest load, and the follow-up vehicles join via V2V (…). Path planning combined with ant colony optimization and MADRL: ant colony pheromone Initialize to a historical success rate of 0.5, then update. ,in =0.1 evaporation rate, if full else 0; , MADRL Actions Guided exploration will terminate when the empty load ratio is less than 10% or after 50 iterations. The path will consider time constraints to ensure the container is retrieved within [time constraints]. Inside, Formation priority: Same vessel number (weight 1) > Similar vessel numbers (0.8) > Same destination (0.6), with a size of 3-5 vessels.

[0063] 5. V2V Communication and Intelligent Gate Allocation V2V Protocol: Vehicle Leading Broadcast Congestion c lead and speed v lead Calculated from the car ,in Deceleration or reorganization occurs when the deviation exceeds 5%. Gate reservation: Monte Carlo simulation of 1000 load curve predictions (based on current traffic) prioritizes low-utilization gates (<70%), reserving continuous time slots (queue size × 2 minutes). In case of conflict, 5G redirection uses backup gates, which are then fed back as MADRL samples (negative). r i =-0.5).

[0064] 6. Execution and Performance Monitoring Vehicle execution: Autonomous driving according to the path sequence, real-time monitoring of KPIs: Formation completeness = actual formation / target > 80%; empty load ratio < 30%; gate throughput improvement of 20%. Self-diagnosis occurs when deviation > threshold (10%), rolling back to random mode and storing data in a buffer for fine-tuning MADRL. Safety: Negative rewards for collision avoidance; if... c i >0.9 manual intervention.

[0065] Through the above steps, this embodiment reduces peak-hour waiting time from 15 minutes to 4 minutes in the simulation, reduces empty-load mileage by 35%, and improves overall efficiency by 25%.

[0066] This embodiment sets up a port-wide scenario including 3 yards, 4 wharves, and 4 gates, with key logistics channels capable of handling 100 and 50 autonomous external container trucks per hour. The system runs on an edge computing server and integrates a 5G network to support V2V communication. Parameters are optimized based on historical throughput data typical of large container ports. It collects data on vehicles, container tasks, and traffic flow by constructing a port-wide traffic network graph model and updates it every 5 seconds. A K-means clustering algorithm (K=10) is used to group and initially assign 100 tasks. Each vehicle is treated as a MADRL agent, and the process is handled by Actor-Cri. The tic framework (QMIX variant) combines GNN embedding processing for offline training and online fine-tuning. Based on the auction results, it forms a platoon of 3-5 vehicles. It combines ant colony algorithm and MADRL to complete path planning, dynamically adjusts the platoon spacing through V2V communication, and realizes intelligent gate allocation based on Monte Carlo simulation. At the same time, it monitors performance and ensures safety in real time. In the simulation, this embodiment successfully reduced the peak waiting time from 15 minutes to 4 minutes, reduced the empty mileage by 35%, improved the overall efficiency by 25%, achieved platoon integrity of over 80%, and improved the gate throughput by 20%, effectively realizing efficient coordination and optimization of collection and distribution in the pan-port area.

[0067] This invention discloses a method and system for scheduling autonomous vehicle platooning tasks in a port-wide transportation hub. Addressing issues such as traffic congestion, low gate utilization, and low operational efficiency during peak hours in port-wide areas, this method is applied to large container port-wide areas. It involves six steps: system initialization and data acquisition, task grouping and preliminary allocation, construction and training of an auction mechanism integrating MADRL, dynamic platooning and path planning, intelligent gate allocation and execution, and performance evaluation and safety assurance. The method integrates K-means clustering, multi-agent deep reinforcement learning (MADRL), V2V communication, ant colony algorithm variants, Monte Carlo simulation, and graph neural networks. This method and system utilizes technologies such as Generative Neural Networks (GNN) to achieve efficient task clustering, dynamic collaborative vehicle platooning, spatiotemporal path optimization, and intelligent gate allocation. It overcomes the limitations of existing technologies that rely on single vehicle scheduling and static allocation, filling the gaps in multi-vehicle collaboration, spatiotemporal path optimization, and intelligent gate allocation. It can reduce empty vehicle round-trip mileage by more than 30%, shorten the average gate waiting time to less than 5 minutes, increase utilization by more than 20%, and improve vehicle platooning success rate to more than 80%. It achieves peak shaving and valley filling during peak periods, significantly improving the efficiency and safety of the port area's collection and distribution, while reducing energy consumption and carbon emissions, resulting in significant economic and social benefits.

[0068] Although the present invention has been described in detail with reference to the accompanying drawings and preferred embodiments, the invention is not limited thereto. Various equivalent modifications or substitutions can be made to the embodiments of the invention by those skilled in the art without departing from the spirit and essence of the invention. Such modifications or substitutions should all fall within the scope of the invention, or any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the invention should be covered within the protection scope of the invention. Therefore, the protection scope of the invention should be determined by the scope of the claims.

Claims

1. A method for scheduling autonomous vehicle platooning for collection and distribution in a port area, characterized in that, The method includes: Perform system initialization and data acquisition to construct a digital twin model of the Pan-Port Area and a global state space; Task grouping and initial assignment were completed based on K-means clustering; A multi-agent deep reinforcement learning (MADRL) model is integrated with an auction mechanism to achieve collaborative bidding and decision optimization for vehicles. Dynamic formation and path planning are performed based on the action sequence output by the auction. Based on the MADRL model, gate reservation decisions are output, and gates are intelligently allocated and executed. Performance evaluation and security assurance are achieved through monitoring and adaptive optimization.

2. The method according to claim 1, characterized in that, The process of initializing the system and collecting data to construct a digital twin model and global state space for the Pan-Port area specifically includes: The construction of a digital twin model for the Pan-Port Area involves the structured abstraction and digital mapping of key logistics nodes within the Pan-Port Area, including multiple wharves, rear storage yards, various gates, and the logistics road network connecting these nodes. This model is used to recreate the static attributes of each logistics node, including its spatial layout and facility capacity. It also synchronizes dynamic information such as the operational status of logistics nodes, road capacity, and real-time traffic conditions in real time, forming a virtual simulation environment that is highly consistent with the physical world in real time. Simultaneously, the initialization phase also completes the configuration of system parameters, allocation of computing resources, and definition of the state space. In terms of data collection scope, it covers key information throughout the entire lifecycle of containers, including the container's vessel identification, cargo owner's label, preset pick-up time window, and final destination coordinates; dynamic status data of autonomous vehicles, including the vehicle's current real-time location, actual load, operating status, and remaining range; regional traffic flow data, which combines historical traffic data with real-time monitoring data to predict peak-hour congestion trends and provide congestion references for route planning; and gate operation data, including the real-time utilization rate of each gate and the length of the work queue. In terms of data collection methods, the environmental and facility status is captured by sensors distributed in various areas, location information is obtained by GPS devices on vehicles, and data interconnection and real-time transmission of various devices are realized by IoT terminals. All the collected raw data are cleaned and integrated, and continuously updated to form a standardized global state space S={V, T, N}, where V is the vehicle state vector with dimension corresponding to the total number of vehicles n, T is the task queue matrix, and N is the network load matrix that represents the degree of congestion with roadside weights.

3. The method according to claim 2, characterized in that, The task grouping and initial assignment based on K-means clustering specifically includes: Based on container attributes, including vessel similarity, destination consistency, and pickup time window, the K-means clustering algorithm combined with time constraints is used to group tasks. In the rear yard, containers from the same vessel or from similar vessels with a time difference less than a threshold T are assigned to nearby vehicles to form potential platoons. At the terminal, containers from the same shipper or with adjacent pickup times and a time window overlap rate >50% are grouped. The clustering objective function is to minimize the intra-group distance, and its expression is: ； In the formula, J is the clustering objective function value, representing the sum of the intra-group distances of all clusters. The goal is to minimize this value using the K-means algorithm to achieve tight aggregation of similar tasks; K is the total number of clusters, which is the final number of task groups into which all container tasks are divided, and is dynamically adjusted according to the actual scenario; k is the index of the cluster, used to traverse each independent task group; C k Let be the k-th cluster, containing a group of container tasks with similar attributes; it is the initial task unit formed after clustering. x is the feature vector of a single container task, with dimensions covering the core attributes of the task; it is the basic data unit for clustering computation. μ k Let x be the cluster center of the kth cluster, and let x be the mean of all task feature vectors x in the cluster, representing the core attribute of the task group, used to measure the difference between the tasks and the core attribute within the cluster. For a single task x and its cluster center μ k The square of the Euclidean distance is used to quantify the degree of difference between a single task and the core attributes of its task group. The smaller the distance, the higher the similarity between the task and other tasks in the group.

4. The method according to claim 3, characterized in that, The integrated multi-agent deep reinforcement learning (MADRL) model and auction mechanism enable collaborative vehicle bidding and decision optimization, specifically including: Agent definition and auction integration: Each autonomous vehicle acts as a MADRL agent, and the total number of agents N equals the overall size of the vehicle cluster; the local state of each agent... Including position coordinates Current load Surrounding vehicle status and congestion indicators Peripheral agent status subset ; The action of the i-th agent This includes generating bid values. Task selection Formation Operations Path selection Make an appointment with the gate ; Auction mechanism embedded in MADRL framework: Agents compute bid functions through Actor network The expression is as follows: ； in, Indicates the savings when not in use. For load matching degree; The required load capacity for the mission; The coordinates of the mission destination or pickup point; The penalty for waiting indicates a delay beyond the pickup time window; This refers to the actual time during which the vehicle can perform its tasks. This is the end time of the container pickup window; This is a balancing factor used to weigh the importance of idle savings against waiting penalties; The Vickrey auction algorithm is used to select a winner from the central server, and the task is bound to the winning agent to form an initial candidate set of car teams.

5. The method according to claim 4, characterized in that, The integrated multi-agent deep reinforcement learning (MADRL) model and auction mechanism enable collaborative vehicle bidding and decision optimization, specifically including: Network architecture: First, the MADRL variant QMIX of the Actor-Critic framework is adopted, in which the Actor network uses a deep neural network (DNN), including LSTM layers to process temporal data and attention mechanisms to capture inter-vehicle dependencies, and finally generates the action policy π(a|s); The global value function expression for the Critic network evaluation is: ； Where Q(s,a) is the global value function value, representing the comprehensive value brought to the entire port area's collection and distribution system after all vehicle agents take action combination a under the current global system state s. The higher the value, the more the current action combination conforms to the global optimization objective; a is the action combination of all vehicle agents, where a i For the i-th agent's independent action; N is the total number of agents in the MADRL system, which is equal to the size of the autonomous vehicle cluster participating in the scheduling; Let be the local value function value of the i-th agent, representing the agent's local state s. i Take action a i The resulting "individual value" is a function that measures only the value of a single agent's decision to its own goals, without considering the synergistic effects with other agents. f ( mixing ) is a hybrid function, which is the core module of the QMIX architecture. It is used to integrate the local observation and decision dependencies of all agents and correct the bias of simple local value accumulation. The Critic network evaluates the global value function by integrating the local observations of all agents: first, it accumulates the observations of each agent in its local state s. i Next, execute action a i Local value Q i (s i ,a i Then through the mixing function f (mixing) It corrects the bias of simply accumulating local value and effectively captures the synergistic benefits and conflict costs between agents; Secondly, the input feature processing method is optimized. For the "node-edge" topological characteristics of the collection and distribution scenario in the pan-port area, a graph neural network (GNN) is introduced to embed spatiotemporal path information. The storage yard and wharf are abstracted as network nodes, and logistics roads are abstracted as connecting edges. The GNN models the spatial topological dependence between nodes and edges, and integrates dynamic spatiotemporal information to transform the scattered spatial structure data and dynamic time series data into high-dimensional feature vectors.

6. The method according to claim 5, characterized in that, The integrated multi-agent deep reinforcement learning (MADRL) model and auction mechanism enable collaborative vehicle bidding and decision optimization, specifically including: Offline training is performed in a simulated environment, using an experience replay buffer to store trajectories (s, a, r, s'), where s is the state, a is the action, r is the reward, and s' is the next state, updated via Q-learning: ,in γ is a discount factor, 0 < γ < 1, used for discounting future rewards. ; The reward function is designed as follows: ； in: This represents the actual number of vehicles in the platoon. Target formation size; This represents the unloaded driving distance during the current mission. To the maximum allowable unloaded distance; This refers to the actual waiting time at the gate. Maximum allowed waiting time; Reward weighting coefficient These correspond to platooning efficiency, reduction of empty mileage, and minimization of gate waiting time, respectively, satisfying the following requirements: =1; p The penalty coefficient is... For collision and congestion risk indicators; Introducing exploration strategies -greedy, exploring probability The decay rate starts from 0.1 to handle uncertainty; and during online deployment, the model is adjusted through real-time feedback to optimize the parameters of the MADRL model that has been trained offline, and to optimize auction bidding to encourage collaboration. MADRL prioritizes fleet formation through reward-based design: in the rear yard, inter-agent communication consolidates containers from the same or similar vessels, reducing individual vehicle round trips; at the terminal, agents coordinate to merge scattered container pickup tasks into fleets, calculating consolidation benefits for tasks from the same shipper or adjacent time windows, saving empty distances. <5km, global optimization reduces empty vehicle round trips.

7. The method according to claim 6, characterized in that, The dynamic grouping and path planning based on the action sequence output by the auction specifically includes: Vehicle platooning is formed based on the action sequence output by the MADRL auction. Spatiotemporal route planning considers time window constraints and road capacity, uses GNN to predict congestion, and generates the optimal route sequence. The priority of the formation is: same vessel number > similar vessel number > same destination, to ensure that the formation size is moderate and to avoid the formation being too long and affecting maneuverability; Path cost function: ,in For edge distance, For congestion weight, This represents the congestion sensitivity coefficient.

8. The method according to claim 7, characterized in that, The gate reservation decision based on the MADRL model, and the intelligent allocation and execution of gates, specifically include: The MADRL model outputs gate reservation decisions, prioritizing the allocation of low-load gates to convoy vehicles to reduce waiting time, with the goal of controlling gate waiting time to within 5 minutes. During the execution phase, the vehicle drives autonomously according to the dispatch instructions, and is monitored and adjusted in real time; the system feeds back and updates the model parameters cyclically to achieve adaptive optimization.

9. The method according to claim 8, characterized in that, The performance evaluation and security assurance through monitoring and adaptive optimization specifically includes: Real-time monitoring of platoon success rate, target >80%; reduction in empty mileage, target >30%; improvement in gate utilization, target >20%; and overall throughput efficiency; It integrates collision avoidance algorithms, MADRL negative reward constraints, and manual intervention interfaces, and supports manual intervention when the congestion index is >0.9 or when a safety risk occurs; A security mechanism is adopted for adaptive feedback optimization: data during the execution process is fed back to the system to update the MADRL model parameters, so as to achieve the goal of distributing tasks during peak periods and concentrating them during off-peak periods, and continuously optimize the auction bidding strategy and collaborative decision-making logic.

10. A task scheduling system for automated vehicle platooning in a port area, the system being used to implement the task scheduling method for automated vehicle platooning in a port area as described in any one of claims 1 to 9, characterized in that, The system includes: System initialization and data acquisition unit: provides basic scenario models and full-dimensional real-time data support for the entire scheduling process, and constructs a digital twin environment and global state space for the Pan-Port Area; Task grouping and preliminary allocation unit: Based on container attributes and time constraints, it realizes efficient clustering and preliminary allocation of tasks, providing structured task input for the subsequent auction mechanism; The auction mechanism unit integrating MADRL: By integrating multi-agent deep reinforcement learning with the auction mechanism, dynamic bidding allocation of tasks and optimization of vehicle collaborative decision-making are achieved; Dynamic platooning and route planning unit: Based on the auction results, a reasonable vehicle platoon is formed, and the optimal driving route is generated by combining spatiotemporal constraints and congestion prediction; Intelligent gate allocation and execution unit: realizes the optimal allocation of gate resources and the execution of scheduling instructions, ensuring efficient and smooth transportation processes; Execution monitoring and adaptive optimization unit: performs full-process monitoring, performance evaluation and security assurance of the scheduling execution process, and realizes adaptive iterative optimization of the system.

Images