Intelligent port scheduling method based on mathematical model dual drive

Abstract

The invention belongs to the technical field of power system optimization scheduling, and discloses a mathematical model dual-drive-based port intelligent scheduling method, which comprises the following steps of inputting real-time operation data of a port logistics system and an energy system; on the basis of a deep reinforcement learning model, operation data is adopted for training, a state-action-reward mapping relation is established, and a data-driven preliminary scheduling strategy is generated; based on the operation data, constructing a model-driven traffic distribution-user balance optimization model, and obtaining an energy pricing strategy capable of minimizing the operation cost; constructing a multi-agent collaborative decision framework to coordinate a preliminary scheduling strategy and a traffic distribution-user equilibrium optimization model, and realizing iterative optimization of dual-drive strategy collaboration; dynamically selecting an optimal sub-heuristic strategy by adopting a dual deep Q network structure based on an iterative optimization result; the problems of insufficient strategy generalization ability and poor dynamic environment adaptability in the prior art are solved.

Description

Technical Field

[0001] This invention belongs to the field of power system optimization and scheduling technology, specifically involving a port intelligent scheduling method based on a dual-drive mathematical model. Background Technology

[0002] With the rapid development of global trade, ports, as logistics hubs, face increasingly complex scheduling demands, particularly in container loading and unloading, vehicle routing planning, and energy coordination management. Traditional port scheduling methods mainly rely on manual experience or static optimization models, making it difficult to cope with uncertainties in dynamic environments, such as traffic congestion, equipment failure, and energy price fluctuations. While existing methods based on single reinforcement learning can achieve partially adaptive scheduling, they suffer from insufficient policy generalization ability and slow convergence speed. Furthermore, the strong coupling between port logistics and energy systems makes it difficult for traditional methods to optimize energy consumption while ensuring transportation efficiency. Therefore, existing technologies suffer from insufficient policy generalization ability and poor adaptability to dynamic environments. Summary of the Invention

[0003] To address the shortcomings of existing technologies, the present invention aims to provide a port intelligent scheduling method based on a dual-drive mathematical model, which solves the problems of insufficient strategy generalization ability and poor adaptability to dynamic environments in existing technologies.

[0004] The objective of this invention can be achieved through the following technical solutions:

[0005] A port intelligent scheduling method based on a dual-drive mathematical model includes the following steps:

[0006] Input real-time operational data of the port logistics system and energy system;

[0007] Based on a deep reinforcement learning model, running data is used for training to establish a mapping relationship between state, action, and reward, and a data-driven preliminary scheduling strategy is generated.

[0008] Based on operational data, a model-driven traffic allocation-user equilibrium optimization model is constructed to obtain an energy pricing strategy that minimizes operating costs.

[0009] A multi-agent collaborative decision-making framework is constructed to coordinate the initial scheduling strategy and the traffic allocation-user equilibrium optimization model, so as to achieve iterative optimization of the dual-drive strategy.

[0010] Based on the iterative optimization results, a dual-depth Q-network structure is adopted to dynamically select the optimal sub-heuristic strategy, generate vehicle route planning and resource scheduling instructions, and realize the coordinated scheduling of port logistics and energy.

[0011] Operational data includes vehicle location and trajectory, yard operation status, charging pile load, electricity node price, equipment operating time, and operation scheduling records.

[0012] Based on a deep reinforcement learning model, training is performed using runtime data to establish a mapping relationship between state, action, and reward. This includes the following steps:

[0013] Based on vehicle location and trajectory, yard operation status, and charging pile load data, a time-series snapshot of the port operation process is constructed.

[0014] In each time slice, extract sub-task information corresponding to the port status, including vehicle routing planning, yard loading and unloading scheduling, and charging queue management;

[0015] By calling expert systems or historical scheduling data, reference scheduling strategies for each task scenario are obtained, and a training sample set containing "state-action-reward" triples is constructed.

[0016] By combining electricity node price and equipment uptime data, a composite reward function is constructed that includes transportation efficiency, energy consumption, and waiting time.

[0017] Using the training sample set as input, a deep reinforcement learning model learns the mapping relationship between state and action, and combines a composite reward function to optimize the strategy, ultimately generating a complete mapping relationship between state, action, and reward.

[0018] Constructing a model-driven traffic assignment-user equilibrium optimization model to obtain an energy pricing strategy that minimizes operating costs includes the following steps:

[0019] The objective function of the traffic assignment-user equilibrium optimization model is defined as follows: it considers traffic delay, charging waiting time, energy cost, and user utility, and seeks an energy incentive strategy that minimizes the operating cost of the logistics-energy system and maximizes user welfare.

[0020]

[0021] In the formula, U(π) represents the overall system operating cost under scheduling strategy π; ω represents the load cost weight; t a (z) and t k (z) represents the unit delay function of path a and energy node k when the load is z; and Let A and B represent the optimal loads of path A and node K, respectively. and f represents the charging cost and time cost for user p in region w, respectively; p,w This represents the amount of charge user p receives in region w; d represents the inverse function of the excitation response function in region w; w Indicates the user response volume guided by region w; A, K, W, and E w Let w represent the path set, energy node set, region set, and user set in region w, respectively.

[0022] The traffic assignment-user equilibrium optimization model takes into account the differences between users, allowing different users to have heterogeneous travel flexibility and initial power status, and characterizes the regional energy supply capacity limit through node-level energy consumption constraints.

[0023] In route decision modeling, a logistics-energy coupling model with incentive signals is adopted to regulate vehicle route and charging behavior;

[0024] Since vehicle travel demand and initial battery power are uncertain, an active path generation mechanism is introduced to transform the path optimization problem into a two-layer iterative solution process. The two-layer iterative solution process is as follows: In each iteration, the optimal path and traffic allocation are first solved based on the current path set; then, a local search is performed on the traffic network to generate potential optimal paths and expand the path set; the process continues until the path set no longer expands.

[0025] Constructing a multi-agent collaborative decision-making framework to coordinate the initial scheduling strategy and the traffic assignment-user equilibrium optimization model includes the following steps:

[0026] The port area is divided into multiple autonomous control zones, each with an independent incentive and control unit.

[0027] Each incentive control unit generates energy incentive signals based on the path load rate, charging pile utilization rate, and marginal cost of electricity in its region.

[0028] By dynamically adjusting the strength of the energy incentive signal, the vehicle is guided to select the optimal path and charging strategy;

[0029] Each incentive control unit makes independent decisions based on partial observation information, and the system as a whole forms a non-cooperative game process;

[0030] To collaboratively optimize multi-region strategies, a centralized training-distributed execution mechanism is adopted.

[0031] The incentive control unit guides the vehicle's route selection and charging behavior by issuing appropriate regional energy incentive signals to maximize benefits. The specific expression is as follows:

[0032]

[0033] In the formula, Ω k (π k ,π k- ) represents the strategy combination (π) k,π k- Under π, the revenue function of the incentive control unit in region k; k This represents the energy incentive signal strategy set for region k; π k- This represents the combination of excitation strategies for control units in regions other than region k; This represents the marginal cost or shadow price of electricity in region k. In terms of strategy π k The optimal amount of electrical energy used in region k under guidance.

[0034] In non-cooperative game theory, the system equilibrium state is measured by the no-regret index. When all control units are unable to unilaterally optimize their own payoffs, the system reaches an approximate Nash equilibrium. The expression for the no-regret index is as follows:

[0035]

[0036] In the formula, NI represents the no-regret index.

[0037] Centralized training-distributed execution mechanism, specifically including the following:

[0038] In centralized training, a multi-agent proximal policy optimization framework is used to share global information, calculate the gradient of agent policies, and solve the policy bias under local observation.

[0039] An implicit interaction structure is introduced, and each excitation control unit will share local observation information. i and action a i Mapped to an embedding vector e through the embedding function i It outputs the external information representation of agent i, x. i The specific expression is as follows:

[0040] e i =g i (o i ,a i )

[0041] Based on the embedding vector e i and external information representation x i The Q-value function is reconstructed as follows:

[0042]

[0043] Among them, f i It is a two-layer perceptron; g i It is an embedded function;

[0044] The external information received by any excitation control unit represents x i The embedding vectors from other agent-aware systems are weighted and fused using an attention mechanism, as shown in the following expression:

[0045]

[0046] Where, η j Attention weights; v j Let j be the value vector of the agent's perception; h(·) is the ReLU activation function;

[0047] Attention weight η j The calculation formula is as follows:

[0048]

[0049] Among them, H k With H q This represents the shared matrix for key / query.

[0050] The centralized training process specifically includes the following steps:

[0051] Each stimulus control unit updates the strategy μφ sequentially according to a preset order.

[0052] When the current cell is updated, the advantage function is calculated based on the latest policy of the updated cells. The joint policy is optimized through the following objective function:

[0053]

[0054] Where κ=μ φ (a∣s) / μ φ[l] (a|s) represents the strategy ratio; κ is an approximate estimator of the advantage function; clip is a cutoff function that modifies the policy ratio κ. i The range is restricted to the interval [1-δ, 1+δ].

[0055] When any excitation control unit is unable to upload data due to a network failure, its state and embedding vector are predicted by other normal units based on historical trajectories using the following Bayesian formula:

[0056]

[0057] in, Let be the embedding vector of the i-th excitation control unit at level j; The observation data of the i-th excitation control unit at level j; The prior distribution obtained from historical samples during training can be updated in each round of training to reflect the current estimate of the disconnected agent; Given observation data At that time, the embedding vector The posterior probability distribution; Let be the likelihood function, representing the likelihood of a given embedding vector. Observed at time The probability of;

[0058] After training is completed, the policy network parameters of all incentive control units are frozen. During the execution phase, each incentive control unit independently generates energy incentive signals based on its local policy.

[0059] The dynamic selection of the optimal sub-heuristic strategy using a dual-depth Q-network structure includes the following steps:

[0060] Receive the explicit partial solution status s of the currently scheduled task. a This includes vehicle location and trajectory, yard operation status, and charging pile load;

[0061] Derivation of future predicted state s based on traffic assignment-user equilibrium optimization model b This includes path delay prediction, energy node load forecasting, and electricity price fluctuation analysis.

[0062] The explicit partial solution state s a With future predicted state s b The complete current job state s, s∈S, is formed by splicing the states together;

[0063] Employing a main network Q(s,a;θ) and a target network Q(s,a;θ) - A dual-depth Q-network structure is used, and the target value is defined as follows:

[0064]

[0065] In the formula, t is the time step; γ is the discount factor; R t+1 θ represents the reward at time t+1; θ represents the main network parameters updated online, used to select actions in real time and optimize directly through gradient descent; θ - The stable target network parameters represent periodic synchronization and are used to calculate the target value to reduce overestimation bias.

[0066] Based on the current job state s, dynamically select the optimal sub-heuristic strategy that maximizes the main network Q(s,a;θ);

[0067] The selected optimal sub-heuristic strategy is output to the port operation system for execution, and the decision-making experience is recorded in the experience replay pool for subsequent strategy optimization.

[0068] The beneficial effects of this invention are:

[0069] This invention achieves coordinated scheduling of port logistics and energy systems through a dual-drive mechanism of deep reinforcement learning and traffic assignment-user equilibrium optimization model. It adopts a dual deep Q-network structure to dynamically select the optimal sub-heuristic strategy, combined with a multi-agent collaborative decision-making framework and a centralized training-distributed execution mechanism, which significantly improves scheduling efficiency and system stability. By guiding vehicle route selection and charging behavior through energy incentive signals issued by the incentive control unit, it reduces system operating costs while improving user welfare, effectively solving the problems of insufficient policy generalization ability and poor adaptability to dynamic environments in existing technologies. Attached Figure Description

[0070] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0071] Figure 1 This is a schematic diagram of the overall structure of an embodiment of the present invention. Detailed Implementation

[0072] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0073] like Figure 1 As shown, a port intelligent scheduling method based on a dual-drive mathematical model includes the following steps:

[0074] Input real-time operational data of the port logistics system and energy system;

[0075] Based on a deep reinforcement learning model, running data is used for training to establish a mapping relationship between state, action, and reward, and a data-driven preliminary scheduling strategy is generated.

[0076] Based on operational data, a model-driven traffic allocation-user equilibrium optimization model is constructed to obtain an energy pricing strategy that minimizes operating costs.

[0077] A multi-agent collaborative decision-making framework is constructed to coordinate the initial scheduling strategy and the traffic allocation-user equilibrium optimization model, so as to achieve iterative optimization of the dual-drive strategy.

[0078] Based on the iterative optimization results, a dual-depth Q-network structure is adopted to dynamically select the optimal sub-heuristic strategy to generate vehicle path planning and resource scheduling instructions.

[0079] Operational data includes vehicle location and trajectory, yard operation status, charging pile load, electricity node price, equipment operating time, and operation scheduling records.

[0080] Based on a deep reinforcement learning model, training is performed using runtime data to establish a mapping relationship between state, action, and reward. This includes the following steps:

[0081] Based on vehicle location and trajectory, yard operation status, and charging pile load data, a time-series snapshot of the port operation process is constructed.

[0082] In each time slice, extract sub-task information corresponding to the port status, including vehicle routing planning, yard loading and unloading scheduling, and charging queue management;

[0083] By calling expert systems or historical scheduling data, reference scheduling strategies for each task scenario are obtained, and a training sample set containing "state-action-reward" triples is constructed.

[0084] By combining electricity node price and equipment uptime data, a composite reward function is constructed that includes transportation efficiency, energy consumption, and waiting time.

[0085] Using the training sample set as input, a deep reinforcement learning model learns the mapping relationship between state and action, and combines a composite reward function to optimize the strategy, ultimately generating a complete mapping relationship between state, action, and reward;

[0086] It also employs batch training and experience replay mechanisms to improve the training efficiency and policy stability of deep reinforcement learning models.

[0087] Constructing a model-driven traffic assignment-user equilibrium optimization model to obtain an energy pricing strategy that minimizes operating costs includes the following steps:

[0088] The objective function of the traffic assignment-user equilibrium optimization model is defined as follows: it considers traffic delay, charging waiting time, energy cost, and user utility, and seeks an energy incentive strategy that minimizes the operating cost of the logistics-energy system and maximizes user welfare.

[0089]

[0090] In the formula, U(π) represents the overall system operating cost under scheduling strategy π; ω represents the load cost weight; t a (z) and t k (z) represents the unit delay function of path a and energy node k when the load is z; and Let A and B represent the optimal loads of path A and node K, respectively. and f represents the charging cost and time cost for user p in region w, respectively; p,w This represents the amount of charge user p receives in region w; d represents the inverse function of the excitation response function in region w; w Indicates the user response volume guided by region w; A, K, W, and E w Let w represent the path set, energy node set, region set, and user set in region w, respectively.

[0091] The traffic assignment-user equilibrium optimization model takes into account the differences between users, allowing different users to have heterogeneous travel flexibility and initial power status, and characterizes the regional energy supply capacity limit through node-level energy consumption constraints.

[0092] In route decision modeling, a logistics-energy coupling model with incentive signals is adopted to regulate vehicle route and charging behavior;

[0093] Since vehicle travel demand and initial battery power are uncertain, an active path generation mechanism is introduced to transform the path optimization problem into a two-layer iterative solution process. The two-layer iterative solution process is as follows: In each iteration, the optimal path and traffic allocation are first solved based on the current path set; then, a local search is performed on the traffic network to generate potential optimal paths and expand the path set; the process continues until the path set no longer expands.

[0094] Constructing a multi-agent collaborative decision-making framework to coordinate the initial scheduling strategy and the traffic assignment-user equilibrium optimization model includes the following steps:

[0095] The port area is divided into multiple autonomous control zones, each with an independent incentive and control unit.

[0096] Each incentive control unit generates energy incentive signals based on the path load rate, charging pile utilization rate, and marginal cost of electricity in its region.

[0097] By dynamically adjusting the strength of the energy incentive signal, the vehicle is guided to select the optimal path and charging strategy;

[0098] Each incentive control unit makes independent decisions based on partial observation information, and the system as a whole forms a non-cooperative game process;

[0099] To collaboratively optimize multi-region strategies, a centralized training-distributed execution mechanism is adopted.

[0100] The incentive control unit guides the vehicle's route selection and charging behavior by issuing appropriate regional energy incentive signals to maximize benefits. The specific expression is as follows:

[0101]

[0102] In the formula, Ω k (π k ,π k- ) represents the strategy combination (π) k ,π k- Under π, the revenue function of the incentive control unit in region k; k This represents the energy incentive signal strategy set for region k; π k- This represents the combination of excitation strategies for control units in regions other than region k; This represents the marginal cost or shadow price of electricity in region k. In terms of strategy π k The optimal amount of electrical energy used in region k under guidance.

[0103] In non-cooperative game theory, the system equilibrium state is measured by the no-regret index. When all control units are unable to unilaterally optimize their own payoffs, the system reaches an approximate Nash equilibrium. The expression for the no-regret index is as follows:

[0104]

[0105] In the formula, NI represents the no-regret index.

[0106] Centralized training-distributed execution mechanism, specifically including the following:

[0107] In centralized training, a multi-agent proximal policy optimization framework is used to share global information, calculate the gradient of agent policies, and solve the policy bias under local observation.

[0108] An implicit interaction structure is introduced, and each excitation control unit will share local observation information. i and action a i Mapped to an embedding vector e through the embedding function i It outputs the external information representation of agent i, x. i The specific expression is as follows:

[0109] e i =g i (o i ,a i )

[0110] Based on the embedding vector e i and external information representation x i The Q-value function is reconstructed as follows:

[0111]

[0112] Among them, f i It is a two-layer perceptron; g i It is an embedded function;

[0113] The external information received by any excitation control unit represents x i The embedding vectors from other agent-aware systems are weighted and fused using an attention mechanism, as shown in the following expression:

[0114]

[0115] Where, η j Attention weights; v j Let j be the value vector of the agent's perception; h(·) is the ReLU activation function;

[0116] Attention weight η j The calculation formula is as follows:

[0117]

[0118] Among them, H k With H q This represents the shared matrix for key / query.

[0119] The centralized training process specifically includes the following steps:

[0120] Each stimulus control unit updates the strategy μφ sequentially according to a preset order.

[0121] When the current cell is updated, the advantage function is calculated based on the latest policy of the updated cells. The joint policy is optimized through the following objective function:

[0122]

[0123] Where κ=μ φ (a∣s) / μ φ[l] (a|s) represents the strategy ratio; κ is an approximate estimator of the advantage function; clip is a cutoff function that modifies the policy ratio κ. i The range is restricted to the interval [1-δ, 1+δ].

[0124] When any excitation control unit is unable to upload data due to a network failure, its state and embedding vector are predicted by other normal units based on historical trajectories using the following Bayesian formula:

[0125]

[0126] in, Let be the embedding vector of the i-th excitation control unit at level j; The observation data of the i-th excitation control unit at level j; The prior distribution obtained from historical samples during training can be updated in each round of training to reflect the current estimate of the disconnected agent; Given observation data At that time, the embedding vector The posterior probability distribution; Let be the likelihood function, representing the likelihood of a given embedding vector. Observed at time The probability of;

[0127] After training is completed, the policy network parameters of all incentive control units are frozen. During the execution phase, each incentive control unit independently generates energy incentive signals based on its local policy. All energy incentive signal policies can be deployed independently without communication, ensuring the practicality and scalability of the system.

[0128] The dynamic selection of the optimal sub-heuristic strategy using a dual-depth Q-network structure includes the following steps:

[0129] Receive the explicit partial solution status s of the currently scheduled task. a This includes vehicle location and trajectory, yard operation status, and charging pile load;

[0130] Derivation of future predicted state s based on traffic assignment-user equilibrium optimization model b This includes path delay prediction, energy node load forecasting, and electricity price fluctuation analysis.

[0131] The explicit partial solution state s a With future predicted state s b The complete current job state s, s∈S, is formed by splicing the states together;

[0132] The relationship between the current job state s and the action is constructed as a state-action value function, with the following specific expression:

[0133] Q π (s,a)=E[R t +γR t+1 +γ 2 R t+2 +…|S t =s,A t =a,π]

[0134] t is the time step; γ is the discount factor; R represents the reward, R t R t+1 R t+2 These represent the rewards at the corresponding time points; π represents the strategy.

[0135] A deep Q-network is a deep neural network that outputs an action preference vector (i.e., the preference of the low-level heuristic) given a state s, denoted by Q(s,·;θ), where θ is the set of parameters of the network, which can be trained to help select the most suitable heuristic.

[0136] The target value used in standard deep Q-networks is defined as follows:

[0137] Y t DQN =R t+1 +γmax a Q(S t+1 ,a;θ t )

[0138] Employing a main network Q(s,a;θ) and a target network Q(s,a;θ) - The dual-depth Q-network structure is introduced to separate the "action selection" and "action evaluation" processes, and the target value is redefined as:

[0139]

[0140] In the formula, t is the time step; γ is the discount factor; R t+1 θ represents the reward at time t+1; θ represents the main network parameters updated online, used to select actions in real time and optimize directly through gradient descent; θ - The stable target network parameters represent periodic synchronization and are used to calculate the target value to reduce overestimation bias.

[0141] Based on the current job state s, dynamically select the optimal sub-heuristic strategy that maximizes the main network Q(s,a;θ);

[0142] The selected optimal sub-heuristic strategy is output to the port operation system for execution, and the decision-making experience is recorded in the experience replay pool for subsequent strategy optimization.

[0143] The deep reinforcement learning model described in this application employs a hyperheuristic approach, constructing a complete scheduling scheme by dynamically selecting and combining low-level heuristic strategies (such as vehicle path planning, yard loading and unloading scheduling, and charging queue management). In each decision step, the hyperheuristic framework receives two key state inputs—the explicit partial solution state of the current operation (such as vehicle position and trajectory, yard operation state, and charging pile load) and the future prediction state derived by the model (such as path delay prediction and energy node load estimation). This method achieves efficient collaborative scheduling of port logistics and energy systems by repeatedly calling and optimizing the combination of sub-strategies.

[0144] In this embodiment of the application, a simulation example is used to simulate a typical operating scenario involving a variety of port equipment. The simulated port equipment includes standard cranes, gantry cranes, reach stackers, stackers, and vehicles; the simulated vehicles include port-in-port trucks, port-out-of-port trucks, and unmanned trucks.

[0145] The table below shows the logistics scheduling results for a typical operating scenario simulated in this application. The forklift achieves a 79% efficiency improvement under the energy incentive strategy, while the internal cash register experiences a 3.9% efficiency decrease due to fluctuations in electricity prices, reflecting the model's dynamic trade-off capability in logistics-energy coupling optimization.

[0146]

[0147] In the description of this specification, references to terms such as "an embodiment," "example," "specific example," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0148] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely illustrative of the principles of the invention. Various changes and modifications can be made to the present invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the claimed invention.

Claims

1. A port intelligent scheduling method based on a dual-drive mathematical model, characterized in that, Includes the following steps: Input real-time operational data of the port logistics system and energy system; Based on a deep reinforcement learning model, running data is used for training to establish a mapping relationship between state, action, and reward, and a data-driven preliminary scheduling strategy is generated. Based on operational data, a model-driven traffic allocation-user equilibrium optimization model is constructed to obtain an energy pricing strategy that minimizes operating costs. A multi-agent collaborative decision-making framework is constructed to coordinate the initial scheduling strategy and the traffic allocation-user equilibrium optimization model, so as to achieve iterative optimization of the dual-drive strategy. Based on the iterative optimization results, a dual-depth Q-network structure is adopted to dynamically select the optimal sub-heuristic strategy, generate vehicle route planning and resource scheduling instructions, and realize the coordinated scheduling of port logistics and energy.

2. The port intelligent scheduling method based on dual-drive mathematical model according to claim 1, characterized in that, Operational data includes vehicle location and trajectory, yard operation status, charging pile load, electricity node price, equipment operating time, and operation scheduling records.

3. The port intelligent scheduling method based on dual-drive mathematical model according to claim 2, characterized in that, Based on a deep reinforcement learning model, training is performed using runtime data to establish a mapping relationship between state, action, and reward. This includes the following steps: Based on vehicle location and trajectory, yard operation status, and charging pile load data, a time-series snapshot of the port operation process is constructed. In each time slice, extract sub-task information corresponding to the port status, including vehicle routing planning, yard loading and unloading scheduling, and charging queue management; By calling expert systems or historical scheduling data, reference scheduling strategies for each task scenario are obtained, and a training sample set containing "state-action-reward" triples is constructed. By combining electricity node price and equipment uptime data, a composite reward function is constructed that includes transportation efficiency, energy consumption, and waiting time. Using the training sample set as input, a deep reinforcement learning model learns the mapping relationship between state and action, and combines a composite reward function to optimize the strategy, ultimately generating a complete mapping relationship between state, action, and reward.

4. The port intelligent scheduling method based on dual-drive mathematical model according to claim 3, characterized in that, Constructing a model-driven traffic assignment-user equilibrium optimization model to obtain an energy pricing strategy that minimizes operating costs includes the following steps: The objective function of the traffic assignment-user equilibrium optimization model is defined as follows: it considers traffic delay, charging waiting time, energy cost, and user utility, and seeks an energy incentive strategy that minimizes the operating cost of the logistics-energy system and maximizes user welfare. In the formula, U(π) represents the overall system operating cost under scheduling strategy π; ω represents the load cost weight; t a (z) and t k (z) represents the unit delay function of path a and energy node k when the load is z; and Let A and B represent the optimal loads of path A and node K, respectively. and f represents the charging cost and time cost for user p in region w, respectively; p,w This represents the amount of charge user p receives in region w; d represents the inverse function of the excitation response function in region w; w Indicates the user response volume guided by region w; A, K, W, and E w Let w represent the path set, energy node set, region set, and user set in region w, respectively. The traffic assignment-user equilibrium optimization model takes into account the differences between users, allowing different users to have heterogeneous travel flexibility and initial power status, and characterizes the regional energy supply capacity limit through node-level energy consumption constraints. In route decision modeling, a logistics-energy coupling model with incentive signals is adopted to regulate vehicle route and charging behavior; Since vehicle travel demand and initial battery power are uncertain, an active path generation mechanism is introduced to transform the path optimization problem into a two-layer iterative solution process. The two-layer iterative solution process is as follows: In each iteration, the optimal path and traffic allocation are first solved based on the current path set; then, a local search is performed on the traffic network to generate potential optimal paths and expand the path set; the process continues until the path set no longer expands.

5. The port intelligent scheduling method based on a dual-drive mathematical model according to claim 4, characterized in that, Constructing a multi-agent collaborative decision-making framework to coordinate the initial scheduling strategy and the traffic assignment-user equilibrium optimization model includes the following steps: The port area is divided into multiple autonomous control zones, each with an independent incentive and control unit. Each incentive control unit generates energy incentive signals based on the path load rate, charging pile utilization rate, and marginal cost of electricity in its region. By dynamically adjusting the strength of the energy incentive signal, the vehicle is guided to select the optimal path and charging strategy; Each incentive control unit makes independent decisions based on partial observation information, and the system as a whole forms a non-cooperative game process; To collaboratively optimize multi-region strategies, a centralized training-distributed execution mechanism is adopted.

6. The port intelligent scheduling method based on dual-drive mathematical model according to claim 5, characterized in that, The incentive control unit guides the vehicle's route selection and charging behavior by issuing appropriate regional energy incentive signals to maximize benefits. The specific expression is as follows: In the formula, Ω k (π k ,π k- ) represents the strategy combination (π) k ,π k- Under π, the revenue function of the incentive control unit in region k; k This represents the energy incentive signal strategy set for region k; π k- This represents the combination of excitation strategies for control units in regions other than region k; This represents the marginal cost or shadow price of electricity in region k. In terms of strategy π k The optimal amount of electrical energy used in region k under guidance.

7. The port intelligent scheduling method based on dual-drive mathematical model according to claim 6, characterized in that, In non-cooperative game theory, the system equilibrium state is measured by the no-regret index. When all control units are unable to unilaterally optimize their own payoffs, the system reaches an approximate Nash equilibrium. The expression for the no-regret index is as follows: In the formula, NI represents the no-regret index.

8. The port intelligent scheduling method based on dual-drive mathematical model according to claim 7, characterized in that, Centralized training-distributed execution mechanism, specifically including the following: In centralized training, a multi-agent proximal policy optimization framework is used to share global information, calculate the gradient of agent policies, and solve the policy bias under local observation. An implicit interaction structure is introduced, and each excitation control unit will share local observation information. i and action a i Mapped to an embedding vector e through the embedding function i It outputs the external information representation of agent i, x. i The specific expression is as follows: and i =g i (the i ,the i ) Based on the embedding vector e i and external information representation x i The Q-value function is reconstructed as follows: Among them, f i It is a two-layer perceptron; g i It is an embedded function; The external information received by any excitation control unit represents x i The embedding vectors from other agent-aware systems are weighted and fused using an attention mechanism, as shown in the following expression: Where, η j Attention weights; v j Let j be the value vector of the agent's perception; h(·) is the ReLU activation function; Attention weight η j The calculation formula is as follows: Among them, H k With H q This represents the shared matrix for key / query.

9. The port intelligent scheduling method based on a dual-drive mathematical model according to claim 8, characterized in that, The centralized training process specifically includes the following steps: Each stimulus control unit updates the strategy μφ sequentially according to a preset order. When the current cell is updated, the advantage function is calculated based on the latest policy of the updated cells. The joint policy is optimized through the following objective function: Where κ=μ φ (a∣s) / μ φ[l] (a|s) represents the strategy ratio; κ is an approximate estimator of the advantage function; clip is a cutoff function that modifies the policy ratio κ. i The range is restricted to the interval [1-δ, 1+δ]. When any excitation control unit is unable to upload data due to a network failure, its state and embedding vector are predicted by other normal units based on historical trajectories using the following Bayesian formula: in, Let be the embedding vector of the i-th excitation control unit at level j; The observation data of the i-th excitation control unit at level j; The prior distribution obtained from historical samples during training can be updated in each round of training to reflect the current estimate of the disconnected agent; Given observation data At that time, the embedding vector The posterior probability distribution; Let be the likelihood function, representing the likelihood of a given embedding vector. Observed at time The probability of; After training is completed, the policy network parameters of all incentive control units are frozen. During the execution phase, each incentive control unit independently generates energy incentive signals based on its local policy.

10. The port intelligent scheduling method based on dual-drive mathematical model according to claim 9, characterized in that, The dynamic selection of the optimal sub-heuristic strategy using a dual-depth Q-network structure includes the following steps: Receive the explicit partial solution status s of the currently scheduled task. a This includes vehicle location and trajectory, yard operation status, and charging pile load; Derivation of future predicted state s based on traffic assignment-user equilibrium optimization model b This includes path delay prediction, energy node load forecasting, and electricity price fluctuation analysis. The explicit partial solution state s a With future predicted state s b The complete current job state s, s∈S, is formed by splicing the states together; Employing a main network Q(s,a;θ) and a target network Q(s,a;θ) - A dual-depth Q-network structure is used, and the target value is defined as follows: In the formula, t is the time step; γ is the discount factor; R t+1 θ represents the reward at time t+1; θ represents the main network parameters updated online, used to select actions in real time and optimize directly through gradient descent; θ - The stable target network parameters represent periodic synchronization and are used to calculate the target value to reduce overestimation bias. Based on the current job state s, dynamically select the optimal sub-heuristic strategy that maximizes the main network Q(s,a;θ); The selected optimal sub-heuristic strategy is output to the port operation system for execution, and the decision-making experience is recorded in the experience replay pool for subsequent strategy optimization.

Images