A berth scheduling method and apparatus based on physical constraint embedding and graph attention
By adopting a berth scheduling method based on physical constraint embedding and graph attention, the problems of real-time response and hard constraint processing in inland river port berth allocation are solved, achieving efficient end-to-end berth allocation decision-making and improving the accuracy and speed of port scheduling.
Patent Information
- Application Number
- CN202611300259.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-08-26
- Publication Date
- 2026-11-17
AI Technical Summary
Existing technologies cannot achieve second-level online response in the inland river port berth allocation problem, rely on manual features and rules, fail to handle hard constraints, and lose the ship-berth relationship structure due to flat coding.
A berth scheduling method based on physical constraint embedding and graph attention is adopted. Through real-time data acquisition, ship-berth bipartite graph construction, rule mask calculation, graph attention encoding, Q-value decoding and mask fusion, end-to-end compliant berth allocation decision is achieved.
It achieves zero hard constraint violations, second-level online decision-making, end-to-end learning, and effective utilization of ship-berth relationships, improving the efficiency and accuracy of berth scheduling and reducing hardware requirements.
Smart Images

Figure CN123117639A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to a berth scheduling method and apparatus based on physical constraint embedding and graph attention; belonging to the field of intelligent scheduling technology for inland waterway ports. Background Technology
[0002] The berth allocation problem (BAP) in inland river ports is a core optimization decision problem in port operations. Its core objective is to minimize the total waiting time for all vessels, while meeting hard constraints such as vessel length, draft, arrival time, and berth capacity. Existing BAP optimization schemes can be categorized into three types: 1. Metaheuristic Methods – Represented by Genetic Algorithms (GA), Particle Swarm Optimization (PSO), and Tabu Search, these methods solve the BAP combinatorial optimization model through global search, achieving high-quality solutions in offline batch processing scenarios. A representative patent is CN120996469A (Optimization Method for Collaborative Scheduling of River-Sea Intermodal Berths and Shore-to-Crane Operations under Ship-to-Ship Direct Transfer Mode), which employs a heuristic iterative solution using a "destruction operator + repair operator." Representative academic work includes Bierwirth & Meisel (2015, EJOR).
[0003] 2. Data-driven / rule-based scoring method – This method learns a berth suitability feature library from historical operational data and combines it with rule-based constraints for scoring. Representative patents include CN122414486A (Intelligent Berth Scheduling Method and System for Bulk Cargo Terminals Based on Historical Operational Data of TOS), which extracts historical features from the TOS system to establish a berth suitability scoring model; and CN122453035A (Construction and Fusion Method and System for Smart Water Transport Knowledge Graph in Ports), which uses a knowledge graph to express scheduling constraints and combines it with a large model to score candidate solutions.
[0004] 3. Deep Reinforcement Learning (DRL) Methods – In recent years, DRL has been introduced into BAP. Representative academic works include Kool et al. (2019, ICLR) proposing an Attention Model to solve combinatorial optimization; and Lv et al. (2024, T-ITS) using Pointer Network + Actor-Critic to solve the harbor BAP.
[0005] All three of the above technical approaches have drawbacks that cannot be simultaneously addressed, specifically: 1. Metaheuristic methods cannot respond to dynamic port arrivals online. The arrival time of vessels at inland ports is affected by upstream water conditions and lock scheduling, with ETA fluctuations reaching ±2 hours. GA / PSO / Tabu need to be recalculated from zero after each change in arrival information; actual measurements show that, under the same scale and hardware, the time for a single integer instance solution is approximately 0.73 s for GA, 0.65 s for PSO, and 1.84 s for Tabu, which is 20 to 62 times that of this invention (approximately 0.03 s) (the above four times are actual measured values under the same hardware with a vessel scale of 7 to 13 ships, with the numerator and denominator taken from the same scale). CN120996469A, which uses a "destruction-repair" heuristic, belongs to the same type of algorithm and also cannot update the allocation scheme within seconds when vessels arrive in port in real time.
[0006] 2. Data-driven / rule-based scoring methods rely on static historical data and manually generated features, lacking end-to-end learning capabilities. CN122414486A relies on historical task data from the TOS system to build an adaptive feature library, requiring manual design of scoring formulas; CN122453035A relies on manually constructed knowledge graphs and large-scale model inference, resulting in high inference latency, and requiring manual rule addition for new scenarios not covered by the knowledge graph (such as drastic water level changes). These methods lack end-to-end neural network learning strategies and cannot adapt to runtime changes such as water level and ETA fluctuations.
[0007] 3. Existing DRL methods suffer from action space explosion and reliance on soft penalties for hard constraints, leading to frequent violations. Consider a port with |V| ships, |B| berths, and |H| time slices. The action space is |V|×|B|×|H|. In a typical Shashi Port on the middle reaches of the Yangtze River (15-25 ships × 4 berths × 96 time slices), the action space reaches 5761-9601 dimensions (calculated separately based on a lower bound of 15 and an upper bound of 25 for the number of ships, considering the possibility of "no allocation" with one additional dimension). Existing DRL methods handle hard constraints in the following ways: - Method A: Allowing agents to output violating actions, then penalizing them with large negative rewards. Disadvantages: Violating samples are repeatedly collected, requiring tens of thousands of episodes for the policy to "remember" the constraints, and intermittent violations may still occur after deployment; - Method B: Action filtering is performed during the rollout phase, but the filtering is not perceived during training, leading to inconsistencies between the training and deployment distributions.
[0008] 4. MLP-based general encoders cannot effectively utilize the ship-berth relationship structure. The state of BAP is naturally a matching relationship between two types of entities: "ship" and "berth". If the state is directly flattened into a vector and input into the MLP, the relational inductive bias is lost, making it difficult for the policy to learn the core decision pattern of "matching the right ship with the right berth".
[0009] In summary, for inland waterway port scheduling scenarios, a viable technical solution must simultaneously meet the following requirements: 1. Zero hard constraint violations (not relying on luck with soft penalties); 2. Second-level online decision-making (not relying on metaheuristic global search); 3. End-to-end learning (not relying on manual feature scoring or manual knowledge graph input); 4. Effective utilization of the ship-berth relationship structure (not using a flat MLP). Existing technologies cannot simultaneously meet all four requirements, therefore, improvements are necessary. Summary of the Invention
[0010] The purpose of this invention is to provide a berth scheduling method and apparatus based on physical constraint embedding and graph attention that can effectively solve the problems of offline scheduling computation delay, reliance on manual design of features and rules, failure of hard constraint control, and loss of ship-berth matching relationship in existing metaheuristic algorithms, rule scoring schemes, and traditional deep reinforcement learning techniques.
[0011] The technical solution of this invention is: A berth scheduling device based on physical constraint embedding and graph attention includes: a real-time data acquisition unit, a ship-berth bipartite graph construction unit, a rule mask calculation unit, a graph attention encoding unit, a Q-value decoding and mask fusion unit, and a scheduling decision and model training unit; wherein, Real-time data acquisition unit Externally, it interfaces with AIS ship receivers, water level sensors, and berth status monitoring terminals; internally, it has two outputs: one connects to the ship-berth bipartite diagram construction unit, outputting raw feature data of the ship, berth, and water level; the other connects to the rule mask calculation unit, outputting ship dimensions, berth parameters, time-sharing water level, and time slice information. Ship-berth bipartite structural unit The input end is connected to the real-time data acquisition unit; the output end is unidirectionally connected to the graph attention encoding unit, which outputs a complete bipartite graph of ship-berth with 7-dimensional ship nodes, 7-dimensional berth nodes, and 2-dimensional edge features. Rule mask calculation unit The input end receives all constraint parameters from the real-time data acquisition unit; the output end is unidirectionally connected to the Q-value decoding and mask fusion unit, and outputs a three-dimensional Boolean feasibility mask tensor; this unit is a pure logic operation module and has no network weight parameters. Graph Attention Encoding Unit The input end connects to the ship-berth bipartite graph construction unit; the output end connects to the Q-value decoding and mask fusion unit, outputting 64-dimensional embedding vectors of ships and berths; the internal structure contains fixed two-layer GATv2 network weights.
[0012] Q-value decoding and mask fusion unit Dual input channels: Channel 1 receives the node embeddings output by the graph attention encoding unit, and Channel 2 receives the feasibility mask output by the rule mask calculation unit; single output is sent to the scheduling decision and model training unit, outputting the compliant Q-value tensor after masking illegal actions.
[0013] Scheduling Decision and Model Training Unit The input channel connects to the Q-value decoding and mask fusion unit; internally, it contains two sub-modules: an online inference sub-unit and an offline training sub-unit. The online inference sub-unit parses compliant Q tensors using argmax and outputs the optimal berth allocation scheme to the port scheduling upper-level system. The offline training sub-unit incorporates a Double DQN training framework, a priority experience replay pool, and an Adam optimizer. After training, it reverse-engineers the updated network weights and distributes them to the graph attention encoding unit and the Q-value decoding and mask fusion unit.
[0014] A berth scheduling method based on physical constraint embedding and graph attention includes an offline model training phase and an online real-time scheduling phase; wherein, The offline model training phase includes: 1) Data preprocessing and bipartite graph construction Collect AIS data on vessel and berth specifications and time-of-day water levels at inland river ports, verify data compliance, and divide the dataset; construct a complete bipartite graph of vessel and berth for each scenario, and configure 7-dimensional vessel node features, 7-dimensional berth node features, and 2-dimensional physical difference edge features. 2) Generate a feasibility rule mask The rule mask embedding module performs parallel verification of four hard constraints: ship length, dynamic draft, berth occupancy time without conflict, and ship ETA, generating a three-dimensional Boolean mask tensor; a mask value of 1 represents a compliant allocation scheme, and 0 represents a non-compliant one; 3) Graph attention feature encoding The bipartite graph is fed into a two-layer GATv2 encoder. Each layer has four attention heads, and edge features are used for attention scoring. Intra-layer activation is LeakyReLU, and inter-layer activation is ELU. The multi-head features of each layer are concatenated and projected to compress to 64 dimensions, outputting 64-dimensional embedding vectors for ships and berths. 4) Q-value decoding and mask fusion By concatenating ship embedding, berth embedding, and normalized time scalar, a 129-dimensional feature is obtained; the original Q tensor is output through two layers of MLP (129→64→1); and the Q value corresponding to the violation is set using a mask. This yields a compliant Q tensor containing only feasible solutions; 5) Double DQN Iterative Training A dual-architecture system with a main network and a target network is established, configured with a 20,000-capacity priority experience replay pool and an Adam optimizer learning rate. Discount factor (γ=0.99); hard synchronization of the target network every 200 steps, ε-greedy exploration value linearly annealed from 1.0 to 0.15, for a total of 3000 training rounds; reward function integrates the number of service ships, total waiting time, and full ship completion identification calculation; MSE loss is calculated by sampling empirical samples, and encoder and decoder parameters are updated by backpropagation; 6) Model evaluation and weight saving The violation rate and ship service rate are calculated in a fixed assessment scenario, and the mask and network performance are verified; the network weights are stored permanently after training.
[0015] The online real-time scheduling phase includes, 1) Real-time data acquisition The system uses AIS equipment, water level sensors, and berth monitoring terminals to collect real-time information on vessel dynamics, berth occupancy periods, and real-time water levels.
[0016] 2) Dynamically constructing bipartite graphs and real-time masks Based on real-time data updates of ship, berth, and edge features, a complete bipartite graph for the current period is generated; the masking module is simultaneously invoked to calculate the feasibility mask for the current scenario in parallel.
[0017] 3) Network forward inference The real-time bipartite graph is input into the trained GATv2 encoder to obtain node embeddings, which are then fed into the decoder to complete temporal decoding and masking, and output a compliant three-dimensional Q tensor.
[0018] 4) Output the optimal berth allocation scheme Perform argmax operation on the compliant Q tensor, select the triple (ship, berth, time slice) with the largest Q value as the optimal scheduling scheme, send it to the port scheduling control system and store the log locally.
[0019] 5) Circular update scheduling Continuously collect real-time operational data and repeat steps D1 to D4 to achieve dynamic ship arrivals and continuous scheduling under water level fluctuations at the second level.
[0020] Compared with the prior art, the present invention has the following beneficial effects. 1. Achieve zero hard-restraint violations throughout the entire process. This invention utilizes the synergistic effect of rule mask embedding in Module A and mask fusion in Module C to directly mask all non-compliant actions during model training and actual deployment. Verification across multiple simulation scenarios shows that this invention maintains a stable violation rate of 0 and a service rate of 100%. Ablation studies demonstrate that without the rule mask, the model service rate is only 6.1 ± 1.2%, failing to learn feasible berth allocation strategies. This proves that the rule mask is the core means to ensure the legality and feasibility of the scheduling scheme, rather than merely accelerating training.
[0021] 2. Improve ship-berth matching decision-making performance by leveraging graph attention networks. This invention employs a ship-berth bipartite graph attention network, introducing an inductive bias for inter-entity relationships to fully exploit the matching associations between ships and berths. Under the same parameter count and training conditions, a comparative scheme using a conventional MLP to replace the graph attention network achieves a service rate of only 63.9±15.1%, failing to achieve effective scheduling of all ships. This demonstrates that simply increasing the number of network parameters cannot replace the graph structure's ability to learn ship-berth matching relationships; this invention significantly improves the effectiveness of berth scheduling strategies.
[0022] 3. The model is lightweight and supports deployment and operation on ordinary CPU devices. The complete model of this invention has only 192,257 parameters in actual testing, and its memory usage in FP32 format is approximately 751KB. Compared with the traditional flattened Vanilla DQN baseline model for the same task, the number of parameters is reduced by about an order of magnitude. This invention does not require GPU acceleration and can complete model training and inference using only a general-purpose x86 CPU platform. It can be directly deployed on control servers in ports where there is no dedicated acceleration hardware, thus lowering the hardware threshold for on-site deployment.
[0023] 4. Achieve online decision-making within seconds, adapting to dynamic port arrival scenarios for inland waterway vessels. The inference process of this invention includes mask vectorization operation, graph attention forward calculation, and optimal action selection. Under the normal operating scale of 7-13 ships, the average time for a single CPU decision is 0.030s; even under the high load scenario of 18-35 ships, the maximum time for a single decision is only 0.106s. Compared with metaheuristic algorithms such as genetic algorithms, particle swarm optimization, and tabu search (0.65-1.84s per solution), the response speed is significantly improved, and the berth allocation scheme can be updated in real time under the conditions of dynamic fluctuations in the ETA of inland waterway vessels and continuous arrival of ships. Attached Figure Description
[0024] Figure 1 is a diagram of the overall architecture of the present invention; Figure 2 shows the ablation experiment learning curve of the present invention; Figure 3 is a berth allocation diagram for a case study of the present invention. Detailed Implementation
[0025] See attached document Figure 1-3 To provide further explanation, among which, Figure 1 is a diagram of the overall architecture of the present invention, which is used to illustrate the complete system architecture of the present invention and the data flow relationship during the training and deployment stages. The diagram contains seven functional units enclosed in dashed boxes, each with corresponding reference numerals: Unit 1 is a data acquisition unit, comprising a ship dynamic data acquisition module 11, a water level data acquisition module 12, and a berth static parameter storage module 13; Unit 2 is a ship-berth complete bipartite graph construction module 21, used to generate node and edge features; Unit 3 is a rule mask embedding unit, comprising a hard constraint parallel decision module 31 and a feasibility Boolean tensor output module 32; Unit 4 is a graph attention encoding unit, comprising a first graph attention convolutional layer 41, a second graph attention convolutional layer 42, a node embedding output module 43, and a feature splicing module 44; Unit 5 is a decoding and mask fusion unit, comprising a temporal decoding module 51, an action value tensor output module 52, and a mask fusion and non-action masking module 53; Unit 6 is a training unit, comprising a dual deep Q-learning training module 61, a gradient backpropagation and parameter update module 62, and an experience playback module 63; Unit 7 is an online deployment unit, comprising an optimal action selection module 71 and a scheduling scheme output module 72.
[0026] Data flow: Data acquisition unit 1 outputs data to bipartite graph construction unit 2, which then feeds it in parallel to rule mask embedding unit 3 and graph attention encoding unit 4. The outputs of both units enter decoding and mask fusion unit 5, where the Q-value of inactive actions is set to [value missing]. In training mode, the results of unit 5 are fed back to training unit 6, and the target network performs hard synchronization every 200 steps. In deployment mode, the output of unit 5 is passed to online deployment unit 7, which outputs the berth scheduling scheme. Among them, rule mask embedding unit 3 is a pure rule operation with no training parameters; graph attention encoding unit 4 has 183,872 parameters, decoding and mask fusion unit 5 has 8,385 parameters, and the total number of parameters in the overall model is 192,257. The number of parameters is not marked in the figure.
[0027] Figure 2 shows the ablation experiment learning curves of this invention; used to verify the contribution of each core module of this invention to scheduling performance. Sub-figure (a) shows the evaluation reward variation curve: the horizontal axis represents the training rounds 0-3000, and the vertical axis represents the evaluation reward value. The curve shows the mean of three random seeds for the three groups of experiments: the complete scheme, the removal of priority experience replay, and the removal of graph attention network. The error bars represent ±1 standard deviation; the reward value of the group without the rule mask is too low and is not shown in this figure. Sub-figure (b) shows the evaluation service rate variation curve: the horizontal axis represents the training rounds 0-3000, and the vertical axis represents the service rate 0-100%, including the results of all four groups of comparative experiments. The error bars represent ±1 standard deviation. It can be seen intuitively from the figure that the overall service rate is at an extremely low level after removing the rule mask, proving that the rule mask is an essential module to ensure the feasibility of the scheduling scheme.
[0028] Figure 3 is a schematic diagram of the berth allocation results under the simulation case of 15 ships in Shashi Port during the flat water period of this invention. It consists of sub-figure (a) berth allocation time sequence diagram, sub-figure (b) bar chart of the number of ships serving each berth, and key indicator text boxes on the right. In sub-figure (a), the horizontal axis represents time slices, each time slice corresponding to 15 minutes, and the vertical axis represents the berth number; the rectangular hollow blocks represent the ship operation service period, and the diagonal striped blocks represent the waiting period of the ship from arrival to the start of operation, and are labeled with the corresponding ship numbers V0-V14. Sub-figure (b) is a bar chart showing the actual number of ships serving each berth, with the top of the bars labeled with the number of ships serving the corresponding berth: 9, 4, 2, and 0, respectively. The text box on the right records the key operational indicators of the case: Shashi Station during the normal water period, random seed 123, all 15 ships were assigned, service rate 100%, no illegal assignments, average waiting time for ships was 110.0 minutes, the longest waiting time for a single ship was 630 minutes, and 3 berths were actually used; the footnote below the figure indicates that berth 3 was not assigned a ship in this case, and the length of all ships in the case is no more than 166m.
[0029] Overall process of the system of this invention Referring to Figure 1, the complete execution process of this invention is divided into two parts: the training phase (offline, executed in a general x86 computing platform CPU environment) and the deployment phase (online, executed on a CPU server).
[0030] Training phase steps: S1 Data Preparation - Based on the publicly available water level statistics of the Shashi Hydrological Station on the Yangtze River and the ship type classification of the national standards for inland waterway vessel dimensions, parameterize and generate the ship arrival sequence (length, draft, ETA, priority) and berth specifications (length, static water depth); - Verify the rationality of the data according to the GB / T 3471 standard for inland waterway vessel dimensions; - Randomly sample one set of scene instances for each episode as training scenes, and reserve 10 sets of fixed random seed scenes as evaluation scenes.
[0031] S2 Constructing the bipartite graph state - For each time slice, construct a complete ship-berth bipartite graph G_t = (V_t ∪B, V_t × B, X_v, X_b, X_e); - Node features X_v (7-dimensional), X_b (7-dimensional), and edge features X_e (2-dimensional) are defined according to §3.2 Module B.
[0032] S3 generates rule masks - For each (V_t, B, H) tuple, it computes four types of hard constraints C1~C4 in parallel and outputs a Boolean tensor M_t; - Mask generation is fully vectorized, and multiple states within a batch can be computed simultaneously.
[0033] S4 Training Main Model - Double DQN Algorithm; Both the main network and the target network structure are Module B + Module C; - State input G_t, action output (v, b, t) triplet; - Training superprocessor: see §3.2 Training Algorithm Configuration; - Training scale: 3000 episodes, single seed single machine time is measured to be 6.8~8.1 hours (CPU only).
[0034] S5 Model Evaluation - Calculate EvaluationReward, Serve Rate, and Violation Rate in a hold-out evaluation scenario with 10 fixed random seeds; - If Violation Rate ≠ 0, backtrack to check the mask module (Module A); - If Serve Rate is consistently below 90%, backtrack to check the graph attention module (Module B) and training hyperparameters.
[0035] Deployment phase steps: D1 Real-time data acquisition: The AIS receiver and water level sensor periodically push data to the local database.
[0036] D2 Construct the current bipartite graph: Read the current arriving vessel V_now, berth status B_now, and water level w_now from the database, and construct the bipartite graph G_now.
[0037] D3 generates the rule mask M_now: vectorized computation of four types of hard constraints.
[0038] D4 Model Inference: Input G_now, Output Tensor; Mask fusion: Non-actionable Q-position argmax selects the optimal (v*, b*, t*).
[0039] D5 output allocation scheme: Write (v*, b*, t*) to the local log and scheduling system.
[0040] 5.2 Pseudocode for each module (Algorithm 1 and Algorithm 2) Algorithm 1: Rule Mask Generation (Module A) Input: V (set of ships), B (set of berths), H (number of time slices), w (real-time water level array) Output: M ∈ {0,1}^(|V|×|B|×|H|) for i in range(|V|): for j in range(|B|): for k in range(|H|): # Vectorized parallel execution c1 = (V[i].L <= B[j].L_hat) # Captain c2 = (V[i].D + delta <= B[j].H_hat + w[k]) # Draft (including real-time water level and safety margin delta = 0.3 m) c3 = (k >= B[j].occupied_until) # Occupied (starting slot not earlier than the occupancy deadline slot for this berth) c4 = (k >= V[i].ETA) AND (k >= t_now) # Timing (not earlier than ETA, and not earlier than the current decision time slice) M[i,j,k] = c1 AND c2 AND c3 AND c4 return M # Vectorized implementation does not generate a neural network computation graph in a single generation Algorithm 2: PC-GAT Encoding (Module B) Input: V, B, X_v (ship node features), X_b (berth node features), X_e (edge features) Output: h_v , h_b ; # Layer 1 GATv2 (4 heads × 64 dimensions per head, edge features directly participate in scoring) for each edge (v_i, b_j) in V × B: for head in range(4): α_ij^head = softmax( a^T · LeakyReLU(W_s·X_v[i] + W_t·X_b[j] + W_e·X_e[i,j]) ) h_v[i]^head = Σ_j α_ij^head · W_v · X_v[i] h_b[j]^head = Σ_i α_ij^head · W_b · X_b[i] h_v_layer1 = concat( h_v[i]^head, dim=1 ) # 4 × 64 = 256 dimensions h_b_layer1 = concat( h_b[j]^head, dim=1 ) # 4 × 64 = 256 dimensions # The output of layer 1 is 256 dimensions, which is then directly fed into layer 2 after passing through ELU (no dimensionality reduction projection is performed within the layer). h_v_layer1 = ELU( h_v_layer1 ) # Still 256 dimensions h_b_layer1 = ELU( h_b_layer1 ) # Still 256 dimensions # Second layer GATv2 (Input 256 dimensions; heads = 4, 64 dimensions per head, output still 256 dimensions) h_v_256, h_b_256 = ELU( GATv2Layer2( h_v_layer1, h_b_layer1, X_e ) ) h_v, h_b = Linear_256_to_64(h_v_256), Linear_256_to_64(h_b_256) # The only output projection layer at the end return h_v, h_b 5.3 Pseudocode for each module (Algorithm 3 and Algorithm 4) Algorithm 3: Q-value decoding and mask fusion (Module C) Input: h_v (ship embedding), h_b (berth embedding), M (rule mask), |H| (number of time slices) Output: ; # Step C1: Concatenation (including normalized time scalar) for i in range(|V|): for j in range(|B|): for k in range(|H|): z_ijk = concat( h_v[i], h_b[j], k / |H| ) # 129 dimensions # Step C2: 2-layer MLP time decoder (129 → 64 → 1, intermediate ReLU) Q_hat[i,j,k] = Linear_64_to_1( ReLU( Linear_129_to_64( z_ijk ) ) ) # Step C3: Masking Q = where( M == 1, Q_hat, -1e9 ) return Q Algorithm 4: Double DQN Training Main Loop Input: environment env, main network Q_online, target network Q_target (initialized as a copy of Q_online) Output: Trained Q_online* replay_buffer = ReplayBuffer(capacity=20000) # Sampling strategy options: uniform sampling, or priority sampling (α=0.6, β linearly annealed from 0.4 to 1.0) for episodes 1 to 3000: G_t = env.reset() while not done: M_t = generate_mask(G_t) # Algorithm 1 Q_t = Q_online(G_t, M_t) # Algorithm 2 + 3 (mask fusion) a_t = ε-greedy(Q_t, ε) # ε: 1.0 → 0.15, over a period of 2500 ep G_{t+1}, r_t, done = env.step(a_t) # Reward function R (see §3.2) replay_buffer.push(G_t, a_t, r_t, G_{t+1}, done) if buffer.size >= batch_size: batch = replay_buffer.sample(batch_size=16) # Execute once every 2 environment steps; either sampling strategy is acceptable. # Double DQN Target Calculation: Main network selects actions, target network evaluates value a_next = argmax( Q_online(G_{t+1}, M_{t+1}) ) y = r_t + γ · Q_target(G_{t+1}, M_{t+1})[a_next] · (1-done) loss = MSE( Q_online(G_t, M_t)[a_t], y) loss.backward() clip_grad_norm_(Q_online.parameters(), max_norm=1.0) optimizer.step() # Target network hard synchronization if global_step % 200 == 0: Q_target.load_state_dict( Q_online.state_dict() ) # ε Linear Annealing ε = max(0.15, 1.0 - episode × (1.0-0.15) / 2500) return Q_online Example 1: Normal water level regulation at Shashi Station (Main Experiment) Scene Setup: - Port: A simulated port scene constructed using parameterized parameters based on publicly available hydrological statistical characteristics from the Shashi Station in the middle reaches of the Yangtze River; - Number of Berths: 4; - Number of Ships: 15-25 ships randomly generated per episode; - Number of Time Slices: 96 (15 minutes per slice, totaling 24 hours); - Water Level: Normal water level scenario (based on publicly available water level statistical characteristics, with periodic terms and random disturbances added); - Hardware: General-purpose x86 computing platform, CPU only (no GPU acceleration used).
[0041] The complete parameter configuration of the scene generator is as follows: berth length is uniformly sampled within the range of 200.0 to 320.0 m; berth static design water depth is uniformly sampled within the range of 4.0 to 9.0 m; single-ship service duration is taken as an integer value within the range of 4 to 16 time slices; ship priority is divided into 3 levels, with sampling probabilities of 0.6, 0.3, and 0.1 for levels 1, 2, and 3, respectively. The larger the priority value, the higher the priority. Waiting costs are weighted according to priority values and included in the reward function; draft safety margin is taken as 0.3 m; time slice length is 15 minutes, and the number of time slices per day is 96; the maximum number of ships per scene is 25, and the number of ships per episode is randomly selected between 15 and 25.
[0042] Table 6-0a: Ship type classification parameters of the scene generator (simplified to 5 levels according to the national standard GB / T 3471 series of inland waterway vessel dimensions, covering inland dry bulk carriers, container ships and river-sea direct vessels)
[0043] Self-verification of Table 6-0a: The sum of the weights of the 5 grades is 1.0. When sampling, the ship type grade is extracted after normalization according to this weight. Then, the ship length and draft of the grade are sampled evenly. Therefore, the generated ship dimensions always fall within the grade range of the national standard.
[0044] The real-time water level sequence is synthesized from a baseline term and three superimposed terms: the baseline term is the publicly available statistical baseline value of the hydrological station for the season, which in this embodiment is 1.8m for the Shashi station during the normal water period (i.e., the normal water level scenario referred to in this paper) (the baseline value for the same station during the dry season is 0.3m, and the baseline value for the flood season is 8.5m, which can be used to generate the dry season and flood season scenarios respectively; the corresponding baseline values for Yichang station and Hankou station are also obtained from publicly available hydrological statistics); the first superimposed term is the semi-diurnal tidal cycle term with an amplitude of 0.15m and a period of 50 time slices; the second superimposed term is the intraday slow drift term with an amplitude of 0.3m; and the third superimposed term is the observation noise, which follows a normal distribution with a mean of 0 and a standard deviation of 0.05m. The water level sequence participates in the determination of draft constraints on a time slice basis, thereby reproducing the impact of inland river water level fluctuations on available water depth.
[0045] Training configuration: - Number of Episodes: 3000; - Evaluation every 200 episodes, with the evaluation set consisting of 10 fixed random seed scenes; - Training duration: 6.8~8.1 hours per seed (CPU) in actual tests.
[0046] Table 6-0b: Complete list of hyperparameters for training and evaluation in this embodiment (36 items in total, listed in six categories: network structure, training scheduling, exploration strategy, replay mechanism, reward and evaluation, and scenario setting). ;
[0047] The 36 parameters listed in Table 6-0b have completely consistent values across all ablation experiment groups in this application; only 5 parameters differ between the experimental groups: the rule mask switch, graph attention switch, and priority playback switch (three ablation switches), and the random seed and log output path (two parameters used only to identify the experiment and not involved in model calculation). Therefore, the performance differences between groups can be uniquely attributed to the ablated module itself, rather than differences in training conditions. Those skilled in the art can reproduce the training process of this embodiment using Tables 6-0a and 6-0b and the reward function and constraint decision formula described below.
[0048] The weights for each item in the reward function are as follows: a positive reward of 5.0 for each ship served; a deduction of 8.0 for each unserved ship at the end of the episode; an additional completion reward of 20.0 when all ships are served; a deduction of 2.0 for each violation; and a deduction for waiting time calculated by dividing the total number of waiting time segments for all ships by 10. These four weights appear in both the training startup parameters and the main training program, and their values are consistent after item-by-item comparison. The violation penalty item, due to the masking of module A eliminating violations, has an actual trigger frequency of 0 during the training and deployment phases and is only retained for the completeness of the reward function.
[0049] The determination formulas and implementation methods for the four types of hard constraints are as follows. Constraint C1 (Ship Length): The berth length is not less than the ship length, and the determination result is a Boolean array of shape (|V|,|B|). Constraint C2 (Draft): The sum of the berth's static design water depth and the real-time water level in this time slice is not less than the sum of the ship's draft and the safety margin of 0.3m, and the determination result is a Boolean array of shape (|V|,|B|,|H|). This formula incorporates the real-time water level increment on the available water depth side and the safety margin on the required water depth side. Constraint C3 (Occupancy): The start time slice is not earlier than the current occupancy deadline time slice of the berth, and the determination result is a Boolean array of shape (|B|,|H|). The berth occupancy deadline time slice is updated by the environment after each allocation, thus ensuring that the occupancy periods of any two ships on the same berth do not overlap. Constraint C4 (Timing): The starting time slice is neither earlier than the ship's ETA time slice nor earlier than the current decision time slice. The decision results are Boolean arrays of shapes (|V|,|H|) and (|H|,) respectively. That is, while prohibiting berthing before the arrival time, it is also prohibited to allocate the ship to a time slice that has already passed. The above decision results are combined by bitwise logical AND operation after broadcast alignment to obtain the feasibility Boolean tensor M of shape (|V|,|B|,|H|). The entire process is an array operation. A single generation does not construct a neural network computation graph and does not participate in backpropagation.
[0050] Main experimental results (3 random seeds 42 / 123 / 456, using the evaluation reward at time 3000 of Episode):
[0051] Key metrics: - Violation rate (VR): 0.0% (guaranteed by mask hard constraint); - Service rate: 100.0% (consistent across 3 seeds); - Model parameter count: 192,257 (approximately 751 KB for FP32).
[0052] Table 6-0c: List of tensor parameters of the model in this embodiment (a total of 22 trainable tensors, where the encoder prefix corresponds to the graph attention encoder of module B, and the decoder prefix corresponds to the temporal decoder of module C).
[0053] Table 6-0c's self-verification: Module B has a total of 183,872 parameters, and Module C has a total of 8,385 parameters, totaling 192,257, consistent with the overall parameter count described in §3.2, §3.3, and §7; Module A is a pure array operation and does not contain trainable parameters. The network structure can be calculated layer by layer from Table 6-0c: The encoder input projection layer upscales the 7-dimensional node features to 64 dimensions; the first layer's graph attention linear transformation weights have a shape of 256×64, corresponding to a 4-head × 64-dimensional spliced output; the second layer's graph attention linear transformation weights have a shape of 256×256, meaning no dimensionality reduction is performed within the layer; the final output projection layer has a shape of 64×256, merging the node representations back to 64 dimensions; the decoder's first layer has a shape of 64×129, corresponding to the splicing dimension of ship embedding, berth embedding, and normalized time scalar, and the second layer has a shape of 1×64, outputting the value scalar of a single action triple.
[0054] Example 2: Ablation experiments (Rule Mask, GAT, and experience playback factors) To verify the necessity of the three modules and key mechanisms in the invention, three ablation variants were designed, each with three random seeds × 3000 rounds of training. Together with the three random seeds of the complete scheme, a total of 12 independent experiments were conducted.
[0055] Table 6-1: Ablation experiment results (mean ± standard deviation of 3 seed, time of Episode 3000)
[0056] Observation 1 (Variation without mask): - Service rate plummeted from 100.0% to 5.0%~7.7% (mean of 3 seeds: 6.1 ± 1.2%); - Evaluation reward plummeted from +100.2 to -569.9; - None of the 3 seeds reached 100% service rate → The rule mask is the core of feasibility guarantee, not an optional component to accelerate training; - This phenomenon has a very small variance (±0.2) among the 3 seeds, indicating that "without a mask, the policy cannot learn a feasible solution" is a deterministic conclusion, not an isolated incident.
[0057] Observation 2 (without GAT variant): - Replace the two layers of GATv2 with an MLP (195.6K parameters) with a parameter count comparable to the encoder of this invention, while maintaining complete consistency in the training protocol; - The service rate dropped to 45.5%~82.5%, and the variance among the three seeds was relatively large (±15.1%), indicating that the strategy without graph structure awareness is highly dependent on seed initialization; - The evaluation reward was -62.9~+22.4 points (mean -16.7 ± 35.2), which is much lower than the +90.0~+105.7 points of the Full version → The relational inductive bias of the graph attention network makes a significant contribution to this task and cannot be replaced by simply increasing the capacity of the MLP.
[0058] Observation 3 (Variation without Priority Experience Replay): - Use a uniform sampling replay buffer while keeping all other hyperparameters unchanged; - All 3 seeds converge earlier (the first episode reaching 100% service rate, with a mean of approximately 933 for the 3 seeds, and 1000 / 1000 / 800 per seed; the complete solution has approximately 2933, with 2800 / 3000 / 3000 per seed) and smaller variance; the final evaluation reward is +114.4 ± 1.2, and the service rate is 100.0%; - This shows that the technical effect of this invention does not depend on a specific replay sampling strategy: both uniform sampling and priority sampling can achieve 100% service rate, and they are interchangeable equivalent implementations; - The core innovation of this invention lies in the combination of rule masking and graph attention encoding (removing the mask reduces the service rate to 6.1%, and removing graph attention reduces it to 63.9%), therefore, the replay sampling strategy is not a necessary technical feature.
[0059] In Table 6-1, the definition of "convergence episode" is uniformly defined as: the episode in which the service rate first reaches 100%, calculated as the average of three random seeds; configurations that fail to reach 100% are marked as "not reached". The per-seed count for the complete solution is 2800 / 3000 / 3000 (mean 2933), and the per-seed count after removing priority replay is 1000 / 1000 / 800 (mean 933). All standard deviations in this table are population standard deviations (divided by the sample size), consistent with the definitions used throughout the text.
[0060] Example 3: Single-scenario case analysis Scene Configuration: - Port: Shashi Station, normal water level; - Vessels: 15 vessels (V0 ~ V14), length approximately 67.9~165.6 m, draft 2.5~6.7 m, this instance includes 2 priority levels (scene generator supports 3 levels); - Berths: 4 (numbered 0~3); - Time slices: 96 (15 minutes / slice, total 24 hours); - Model: Full version, seed 123.
[0061] Allocation Results (Excerpt): - All 15 vessels were allocated service, 100% service rate; - Number of violations: 0; - Average waiting time: 110.0 minutes; - Maximum waiting time: 630 minutes (V5, priority level 2, long period without compatible berth windows after arrival); - Berth utilization distribution: Berth 0 served 9 vessels, Berth 1 served 4 vessels, Berth 2 served 2 vessels, Berth 3 served 0 vessels.
[0062] Attention Weight Interpretation: - GATv2 Layer 1: 60 ships-berth edges, attention weight range [0.030, 0.110], mean 0.065 ± 0.022, p90 / p10 ratio 2.60; - GATv2 Layer 2: 60 ships-berth edges, attention weight range [0.044, 0.095], mean 0.063 ± 0.018, p90 / p10 ratio 2.05; - Interpretation: The contribution of the GATv2 encoder mainly comes from the aggregation of graph structured features (i.e., the relational inductive bias of the message passing mechanism itself), rather than the interpretable edge-level attention sharpening of "which edge has a higher weight". This is consistent with Brody et al. (2022)'s observations on GATv2 in dense graphs.
Claims
1. A berth scheduling method based on physical constraint embedding and graph attention, characterized in that, It comprises two main stages: offline model training and online real-time scheduling. The offline model training stage includes the following steps: S1. Preprocessing and constructing a bipartite graph: Collect AIS data of inland river port vessels, berth parameters, and time-sharing water level data, and complete data verification and divide the dataset; construct a complete bipartite graph of vessels and berths, and configure 7-dimensional vessel node features, 7-dimensional berth node features, and 2-dimensional vessel-berth difference edge features; S2. Generate a feasibility mask: Parallel verification of four hard constraints: ship length, dynamic draft, berth occupancy time (no conflict), and ship ETA. Output a three-dimensional Boolean mask tensor. A value of 1 in the mask tensor indicates that the allocation scheme is compliant, and a value of 0 indicates that the allocation scheme is non-compliant. S3, GATv2 Feature Encoding: The complete bipartite graph of the ship-berth is input into a two-layer GATv2 encoder; each layer has 4 attention heads, and edge features participate in the attention scoring calculation. LeakyReLU activation is used within the layer, and ELU activation is used between layers; the multi-head features of the first layer are concatenated to obtain 256-dimensional features and directly input into the second layer without dimensionality reduction projection within the layer; the output of the second layer is still 256-dimensional features, which are compressed to 64-dimensional through the output projection layer, and the 64-dimensional embedding vectors corresponding to the ship and berth are output; S4, Q-value decoding and mask fusion: splicing ship embedding vector, berth embedding vector and normalized time scalar to obtain 129-dimensional features; The 129-dimensional feature is input into a two-layer MLP decoder to output an original Q tensor; the Q value corresponding to the position with a mask value of 0 is set to , to obtain a compliant Q tensor after shielding illegal actions. S5, Double DQN iterative training: build a double network architecture of main network and target network, set up an experience replay pool, use Exploration strategy; set Adam optimizer and discount factor, target network performs hard synchronization according to preset step size; build a reward function containing the number of service ships, total waiting time, and full-ship service completion identifier; sample experience samples to calculate MSE loss, and update network parameters through back propagation; S6. Model Evaluation and Weight Storage: Under the preset evaluation scenario, detect the model's violation rate and ship service rate, verify the feasibility mask and network performance, and store the network weights after verification. The online real-time scheduling phase includes the following steps: D1. Real-time collection of AIS ship data, water level data, and berth occupancy status data; D2. Dynamically update the complete binary diagram of the ship and berth based on the real-time collected data, and simultaneously generate a real-time feasibility mask; D3. Input the updated bipartite graph into the trained GATv2 encoder to obtain node embedding vectors. After decoding and mask fusion, output a compliant Q tensor. D4. Perform argmax operation on the compliant Q tensor, select the optimal (ship, berth, time slice) triple as the berth scheduling scheme and issue it, while retaining the operation log; D5. Collect real-time data on site in a loop, repeat steps D1-D4, and achieve second-level continuous scheduling under the conditions of water level fluctuation and dynamic arrival of ships.
2. The berth scheduling method based on physical constraint embedding and graph attention according to claim 1, characterized in that, The specific rules for determining the four hard constraints are as follows: Length constraint: The ship's length is less than or equal to the rated length of the berth; Dynamic draft constraint: The ship's actual draft is less than or equal to the static water depth of the berth plus the real-time water level of the current time slice; Berth occupancy constraint: The starting time slice for berth allocation does not conflict with the existing operation window of the berth; ETA constraint: The starting time slice for allocation is not earlier than the ship's estimated arrival time.
3. The berth scheduling method based on physical constraint embedding and graph attention according to claim 1, characterized in that, The reward function is calculated as follows: R = 5.0 × number of serviced vessels - 8.0 × number of unserviced vessels - 2.0 × number of violations - total waiting time / 10 + 20.0 × full vessel service completion flag; All violations are masked by a feasibility mask, and the number of violations is always 0 during training and scheduling.
4. The berth scheduling method based on physical constraint embedding and graph attention according to claim 1, characterized in that, The model with a dual-layer GATv2 encoder and MLP decoder has a total of 192,257 parameters and a memory footprint of approximately 751KB in FP32 format. Model training and inference can be completed using only the CPU. Under the condition of 15-25 ships, 4 berths, and 96 15-minute time slices, the CPU inference time for a single operation is less than 1 second.
5. The berth scheduling method based on physical constraint embedding and graph attention according to claim 1, characterized in that, When using the priority sampling experience replay pool, the parameters are configured as follows: the priority index α is set to 0.6, the importance weight β is annealed from 0.4 to 1.0; the training batch is set to 16, the gradient clipping threshold is set to 1.0, and the ε annealing period is 2500 episodes.
6. A berth scheduling device based on physical constraint embedding and graph attention, used to implement the berth scheduling method according to any one of claims 1-5, characterized in that, The system includes a real-time data acquisition unit, a ship-berth bipartite graph construction unit, a rule mask calculation unit, a graph attention encoding unit, a Q-value decoding and mask fusion unit, and a scheduling decision and model training unit. The real-time data acquisition unit interfaces with an AIS ship receiver, water level sensor, and berth monitoring terminal, outputting raw feature data and constraint parameters in two separate channels. The ship-berth bipartite graph construction unit outputs a complete ship-berth bipartite graph containing 7-dimensional ship node features, 7-dimensional berth node features, and 2-dimensional edge features. The rule mask calculation unit is a pure logic judgment module, without trainable network weight parameters, and outputs a 3D feasibility mask tensor. The graph attention encoding unit incorporates a two-layer GATv2 network, receives bipartite graph input, and outputs 64-dimensional ship and berth embedding vectors. The Q-value decoding and mask fusion unit receives node embedding vectors and feasibility mask tensors through two channels, respectively, and outputs a compliant Q-tensor after filtering out illegal actions. The scheduling decision and model training unit includes an online inference subunit and an offline training subunit. The online inference subunit outputs the optimal berth scheduling scheme. The offline training subunit uses the Double DQN algorithm to complete network training and solidifies the updated network weights.
7. A berth scheduling device based on physical constraint embedding and graph attention according to claim 6, characterized in that, In the graph attention coding unit, the number of attention heads in the two layers of GATv2 network is consistent with the output dimension of each head. Each layer is equipped with 4 attention heads, and the edge features participate in the attention scoring calculation.
8. A berth scheduling device based on physical constraint embedding and graph attention according to claim 6, characterized in that, The Q-value decoding and mask fusion unit incorporates a two-layer MLP decoder, performing fully connected operations (129→64, 64→1) sequentially on the 129-dimensional features; and sets the Q-value at the mask position where the value is 0 to... To block illegal allocation schemes.
9. A berth scheduling device based on physical constraint embedding and graph attention according to claim 6, characterized in that, The device runs solely on the CPU, requiring no GPU hardware acceleration, making it suitable for low-cost deployment in small and medium-sized Yangtze River inland ports. It updates the scheduling scheme online in a cyclical manner, achieving second-level dynamic berth scheduling with zero violations.
Citation Information
Patent Citations
River-sea combined transportation berth quay crane cooperative scheduling optimization method in ship-ship direct rotation mode
CN120996469A
Bulk and general cargo wharf intelligent berth arrangement method and system based on TOS historical operation data
CN122414486A
Construction and fusion method and system for port intelligent water transportation knowledge graph
CN122453035A
