Charging scheduling method for electric transfer vehicle of inland port

Abstract

The application discloses an inland port electric transfer vehicle charging scheduling method, comprising the following steps: constructing a transfer vehicle task matching model with the purpose of maximizing the utilization rate of the transfer vehicle and completing all transfer tasks; constructing a transfer vehicle charging scheduling model with the purpose of minimizing the delay, maximizing the transfer vehicle efficiency and minimizing the charging cost; constructing an energy-traffic collaborative scheduling model through the transfer vehicle task matching model and the transfer vehicle charging scheduling model; taking the transfer task and the initial state of all transfer vehicles as the input of the energy-traffic collaborative scheduling model, solving the energy-traffic collaborative scheduling model with the input of the transfer task and the initial state of all transfer vehicles through a deep Q network algorithm, and generating an optimal energy-traffic collaborative scheduling scheme; and scheduling the transfer vehicle through the optimal energy-traffic collaborative scheduling scheme to complete the transfer task of the port. The application can improve the overall transportation efficiency of the port and reduce the transportation cost.

Description

Technical Field

[0001] This invention relates to the fields of Internet big data and port scheduling, specifically to a method for scheduling the charging of electric transfer vehicles in inland waterway ports. Background Technology

[0002] With energy consumption and greenhouse gas emissions constantly escalating, the task of carbon reduction in ports is becoming increasingly urgent. Diesel-powered transfer vehicles account for nearly half of port emissions. To address this challenge, electric transfer vehicles (EVs) have become key to the green transformation.

[0003] However, the 24-hour operation requirements of electric transfer vehicles stand in stark contrast to the vulnerability of the port's power infrastructure. Developing an optimized transfer vehicle scheduling strategy is crucial to balancing the port's green transformation with economic efficiency. This not only relates to the port's environmental image but also directly impacts its operational efficiency and costs. Therefore, we need to comprehensively consider various factors, such as transfer vehicle charging needs, port operating hours, and cargo types, to formulate a scientific and reasonable scheduling plan to achieve green, efficient, and sustainable port operations.

[0004] Since the smooth and timely operation of transfer vehicles has a significant impact on the overall operational efficiency of inland river ports, delays in their arrival can negatively affect port operations. Battery charging is an effective method for refueling transfer vehicles at inland river ports, especially for vehicles equipped with this feature, as it saves considerable charging time and offers advantages over plug-in hybrid transfer vehicles. However, transfer vehicles face energy and transportation-related constraints; to effectively complete cargo transfer tasks, they need to periodically visit charging stations to maintain sufficient state of charge (SoC). Therefore, designing a scheduling method that considers both the constraints of transfer vehicles and energy is a pressing technical problem that needs to be solved. Summary of the Invention

[0005] To address the shortcomings of existing technologies, the technical problem this invention aims to solve is: how to provide a charging scheduling method for electric transfer vehicles in inland waterway ports. This method ensures the utilization rate of transfer vehicles and the completion rate of transfer tasks by constructing an accurate transfer vehicle task matching model, and simultaneously constructs a dynamic transfer vehicle charging scheduling model to maximize transfer vehicle efficiency, minimize delays and charging costs, and optimize energy use efficiency. Furthermore, this method should be able to manage battery status, ensure the proper use and maintenance of batteries, and support integration with existing management systems in inland waterway ports.

[0006] To solve the above-mentioned technical problems, the present invention adopts the following technical solution:

[0007] The charging scheduling method for electric transfer vehicles in inland waterway ports includes:

[0008] S1: Obtain the port's transshipment tasks and the initial status of all transshipment vehicles;

[0009] S2: Construct a transfer vehicle task matching model with the goal of maximizing the utilization rate of transfer vehicles and completing all transfer tasks;

[0010] S3: Construct a charging scheduling model for transfer vehicles with the objectives of minimizing delays, maximizing transfer vehicle efficiency, and minimizing charging costs;

[0011] S4: Use the transfer vehicle task matching model as the first-stage model and the transfer vehicle charging scheduling model as the second-stage model. Construct an energy-transportation collaborative scheduling model through the first-stage model and the second-stage model.

[0012] S5: Take the initial state of the transfer task and all transfer vehicles as input to the energy-traffic cooperative scheduling model, and solve the energy-traffic cooperative scheduling model with the input of the transfer task and the initial state of all transfer vehicles through the deep Q-network algorithm to generate the optimal energy-traffic cooperative scheduling scheme.

[0013] S6: The port's transfer tasks are completed by scheduling transfer vehicles through the optimal energy-transportation coordinated scheduling scheme.

[0014] Preferably, in step S2, the objective function of the transfer vehicle task matching model is expressed as:

[0015] ;

[0016] In the formula: Indicates a transfer task With transfer vehicles Matching indicator, when At that time, it indicates a transfer task. Included in the transfer vehicle In the task list, At that time, it indicates a transfer task. Not included in the transfer vehicle In the task list; each transfer task A tuple Description, in which For pickup point, For delivery nodes, The expected completion time of the task; This represents the reward function for the transfer vehicles in the first-stage model.

[0017] Preferably, in step S2, the constraints of the transfer vehicle task matching model include the number of tasks.

[0018] The task quantity constraint is expressed as:

[0019] ;

[0020] ;

[0021] ;

[0022] In the formula: This indicates the maximum number of tasks.

[0023] Preferably, in step S3, the objective function of the transfer vehicle charging scheduling model is expressed as:

[0024] ;

[0025] ;

[0026] ;

[0027] In the formula: Indicates a charging station; Indicates the path between nodes; Represents the cost function of a charging station; This indicates the empty-load operating cost of the transfer vehicle; This indicates the cost of purchasing electricity from the grid; Indicates the unit battery rating; This represents the cost of no-load operation per unit of time.

[0028] Preferably, in step S3, the constraints of the transfer vehicle charging scheduling model include the transfer vehicle power constraints.

[0029] The power constraint of the transfer vehicle is expressed as follows:

[0030] ;

[0031] ;

[0032] ;

[0033] In the formula: Indicates transfer vehicle Battery status Is it in time? Actively complete the transfer task ; Indicates a transfer task With transfer vehicles Matching indicator, when At that time, it indicates a transfer task. Included in the transfer vehicle In the task list, At that time, it indicates a transfer task. Not included in the transfer vehicle In the task list; each transfer task A tuple Description, in which For pickup point, For delivery nodes, This refers to the expected completion time of the task.

[0034] Preferably, in step S3, the constraints of the charging scheduling model for the transfer vehicles include the state constraints of the transfer vehicles.

[0035] The state constraints of the transfer vehicle are represented as follows:

[0036] ;

[0037] ;

[0038] ;

[0039] In the formula: , and This indicates the number of transport vehicles that are in a loading, charging, or idle state. Indicates time Battery status is The transfer vehicles at the charging station The number of people waiting; Indicates charging station The total number of batteries; Indicates time charging station Medium battery status The number of batteries; This indicates the number of transfer vehicles that are currently being loaded. This indicates the number of transport vehicles that are currently charging. This indicates the number of transfer vehicles that are currently idle. Indicates the node to be rescheduled; Indicates the battery status.

[0040] Preferably, in step S3, the constraints of the transfer vehicle charging scheduling model include charging power constraints.

[0041] The charging power constraint is expressed as:

[0042] ;

[0043] In the formula: This represents the energy equivalent to a unit SoC.

[0044] Preferably, in step S5, the processing steps of the deep Q-network algorithm are as follows:

[0045] S501: Transform the energy-traffic cooperative scheduling model with inputs of the initial states of the transfer tasks and all transfer vehicles into a Markov model;

[0046] S502: Generates the optimal energy-traffic coordinated scheduling scheme by combining a two-stage deep Q-network algorithm with a Markov model.

[0047] Preferably, in step S501, the Markov model obtained by the conversion includes:

[0048] 1) State space, action space, and reward function in the first stage

[0049] ;

[0050] ;

[0051] ;

[0052] In the formula: , , This represents the state space, the first objective action, and the reward function for the first stage. Indicates a transfer task; Indicates the date; , This indicates the initial location and initial battery status of the transport vehicle; Indicates the first One transfer vehicle; This represents the reward function for the second phase. Indicates a transfer task With transfer vehicles Matching indicator, when At that time, it indicates a transfer task. Included in the transfer vehicle In the task list, At that time, it indicates a transfer task. Not included in the transfer vehicle In the task list;

[0053] 2) The state space, action space, and reward function of the second stage

[0054] ;

[0055] ;

[0056] ;

[0057] ;

[0058] ;

[0059] In the formula: , Represents the first and second state spaces of the second stage; , This indicates the second and third objective actions in the second phase; Indicates time; express Location of the time transfer vehicle; express Battery status of the time-transfer vehicle; express The task of transferring time; Indicates the charging price; express Time Charging Station Battery status of the transfer vehicles; Indicates the first One charging station; Indicates the first The battery status of each transport vehicle; express Time Charging Station The charging power; This represents the cost of no-load operation per unit time. , , This indicates the number of transport vehicles that are in an idle, charging, or waiting state. Indicates transfer vehicle Battery status Is it in time? Actively complete the transfer task .

[0060] Preferably, in step S502, the processing steps of the deep Q-network method are as follows:

[0061] S5021: Design a DQN to approximate the Q-value function and randomly initialize its weights. The DQN is used to predict the Q-value of each action based on the current state.

[0062] S5022: Create an experience replay cache to store the states, actions, rewards, and new states generated by the interaction between the matching agent, the transfer vehicle agent, and the charging station agent and the Markov model;

[0063] S5023: Each time step includes the following two phases:

[0064] Phase 1: The matching agent selects the action space from the current status of the transfer vehicle. Select the first target action and define it in the state space of the first stage. and reward function Perform the first objective action and observe the rewards and new status of the first stage;

[0065] Phase Two: The transfer vehicle agent operates within the second phase's action space based on the current status of the charging station. Select the second target action and place it in the state space of the second phase. and reward function The second objective action is executed, observing the first reward and first new state of the first stage; then the charging station agent moves from the action space of the second stage based on the current state of the transfer vehicle's battery level. Select the third target action and place it in the state space of the second phase. and reward function Execute the third objective action, and observe the second reward and second new state in the first phase;

[0066] S5024: Store the current status, first target action, reward and new status of the transfer vehicle in the first stage, the current status, second target action, first reward and first new status of the charging station in the second stage, and the current status, third target action, second reward and second new status of the transfer vehicle's battery status as a set of experiences in the experience replay cache.

[0067] S5025: Randomly select a batch of experiences from the experience replay cache to train DQN, and update the weights of DQN by minimizing the error between the predicted Q value and the target Q value;

[0068] The loss function for training DQN is as follows:

[0069] ;

[0070] In the formula: Indicates the current reward; Indicates the attenuation coefficient; Indicates the state given and actions The maximum target below value; Indicates the state given and actions The model currently predicts value;

[0071] S5026: Repeat steps S5023 to S5025. Through continuous iteration and optimization, DQN will gradually learn how to make the best decisions in the energy-transportation coordinated scheduling model to maximize long-term cumulative rewards.

[0072] S5027: Input the current state of the transfer vehicles into the trained DQN and output the optimal energy-traffic cooperative scheduling scheme.

[0073] This invention also discloses a charging scheduling system for electric transfer vehicles in inland waterway ports, implemented based on the charging scheduling method for electric transfer vehicles in inland waterway ports of this invention; the system includes:

[0074] The acquisition module is used to acquire the port's transshipment tasks and the initial status of all transshipment vehicles;

[0075] The first model building module is used to build a transfer vehicle task matching model with the goal of maximizing the utilization of transfer vehicles and completing all transfer tasks.

[0076] The second model building module is used to build a charging scheduling model for transfer vehicles with the goals of minimizing delays, maximizing transfer vehicle efficiency, and minimizing charging costs.

[0077] The third model construction module is used to use the transfer vehicle task matching model as the first-stage model and the transfer vehicle charging scheduling model as the second-stage model, and to construct an energy-transportation collaborative scheduling model through the first-stage model and the second-stage model.

[0078] The calculation module is used to take the transfer task and the initial state of all transfer vehicles as input to the energy-traffic cooperative scheduling model, and solve the energy-traffic cooperative scheduling model with the input of the transfer task and the initial state of all transfer vehicles through the deep Q-network algorithm to generate the optimal energy-traffic cooperative scheduling scheme.

[0079] The execution module is used to schedule transfer vehicles to complete the port's transfer tasks through the optimal energy-transportation coordinated scheduling scheme.

[0080] Preferably, in the first model construction module, the objective function of the transfer vehicle task matching model is expressed as:

[0081] ;

[0082] In the formula: Indicates a transfer task With transfer vehicles Matching indicator, when At that time, it indicates a transfer task. Included in the transfer vehicle In the task list, At that time, it indicates a transfer task. Not included in the transfer vehicle In the task list; each transfer task A tuple Description, in which For pickup point, For delivery nodes, The expected completion time of the task; This represents the reward function for the transfer vehicles in the first-stage model.

[0083] Preferably, in the first model construction module, the constraints of the transfer vehicle task matching model include the task quantity constraint;

[0084] The task quantity constraint is expressed as:

[0085] ;

[0086] ;

[0087] ;

[0088] In the formula: This indicates the maximum number of tasks.

[0089] Preferably, in the second model construction module, the objective function of the transfer vehicle charging scheduling model is expressed as:

[0090] ;

[0091] ;

[0092] ;

[0093] In the formula: Indicates a charging station; Indicates the path between nodes; Represents the cost function of a charging station; This indicates the empty-load operating cost of the transfer vehicle; This indicates the cost of purchasing electricity from the grid; Indicates the unit battery rating; This represents the cost of no-load operation per unit of time.

[0094] Preferably, in the second model construction module, the constraints of the transfer vehicle charging scheduling model include the transfer vehicle power constraints;

[0095] The power constraint of the transfer vehicle is expressed as follows:

[0096] ;

[0097] ;

[0098] ;

[0099] In the formula: Indicates transfer vehicle Battery status Is it in time? Actively complete the transfer task ; Indicates a transfer task With transfer vehicles Matching indicator, when At that time, it indicates a transfer task. Included in the transfer vehicle In the task list, At that time, it indicates a transfer task. Not included in the transfer vehicle In the task list; each transfer task A tuple Description, in which For pickup point, For delivery nodes, This refers to the expected completion time of the task.

[0100] Preferably, in the second model construction module, the constraints of the transfer vehicle charging scheduling model include transfer vehicle state constraints;

[0101] The state constraints of the transfer vehicle are represented as follows:

[0102] ;

[0103] ;

[0104] ;

[0105] In the formula: , and This indicates the number of transport vehicles that are in a loading, charging, or idle state. Indicates time Battery status is The transfer vehicles at the charging station The number of people waiting; Indicates charging station The total number of batteries; Indicates time charging station Medium battery status The number of batteries; This indicates the number of transfer vehicles that are currently being loaded. This indicates the number of transport vehicles that are currently charging. This indicates the number of transfer vehicles that are currently idle. Indicates the node to be rescheduled; Indicates the battery status.

[0106] Preferably, in the second model construction module, the constraints of the transfer vehicle charging scheduling model include charging power constraints;

[0107] The charging power constraint is expressed as:

[0108] ;

[0109] In the formula: This represents the energy equivalent to a unit SoC.

[0110] Preferably, in the computation module, the processing steps of the deep Q-network algorithm are as follows:

[0111] Step 1: Convert the energy-traffic cooperative scheduling model with input transfer tasks and the initial states of all transfer vehicles into a Markov model;

[0112] Step 2: Generate the optimal energy-traffic coordinated scheduling scheme by combining a two-stage deep Q-network algorithm with a Markov model.

[0113] Preferably, in step one of the calculation module, the Markov model obtained by the transformation includes:

[0114] 1) State space, action space, and reward function in the first stage

[0115] ;

[0116] ;

[0117] ;

[0118] In the formula: , , This represents the state space, the first objective action, and the reward function for the first stage. Indicates a transfer task; Indicates the date; , This indicates the initial location and initial battery status of the transport vehicle; Indicates the first One transfer vehicle; This represents the reward function for the transfer vehicles in the first-stage model; This represents the reward function for the second phase. Indicates a transfer task With transfer vehicles Matching indicator, when At that time, it indicates a transfer task. Included in the transfer vehicle In the task list, At that time, it indicates a transfer task. Not included in the transfer vehicle In the task list;

[0119] 2) The state space, action space, and reward function of the second stage

[0120] ;

[0121] ;

[0122] ;

[0123] ;

[0124] ;

[0125] In the formula: , Represents the first and second state spaces of the second stage; , This indicates the second and third objective actions in the second phase; Indicates time; express Location of the time transfer vehicle; express Battery status of the time-transfer vehicle; Indicates a transfer task; This indicates the cost of purchasing electricity from the grid; express Time Charging Station Battery status of the transfer vehicles; Indicates the first One charging station; Indicates the first The battery status of each transport vehicle; Indicates the unit battery rating; This represents the cost of no-load operation per unit time. , , This indicates the number of transport vehicles that are in an idle, charging, or waiting state. Indicates transfer vehicle Battery status Is it in time? Actively complete the transfer task .

[0126] Preferably, in step two of the computation module, the processing steps of the deep Q-network method are as follows:

[0127] 1) Design a DQN to approximate the Q-value function and randomly initialize its weights. The DQN is used to predict the Q-value of each action based on the current state.

[0128] 2) Create an experience replay cache to store the states, actions, rewards, and new states generated by the interaction between the matching agent, the transfer vehicle agent, and the charging station agent and the Markov model;

[0129] 3) Each time step includes the following two phases:

[0130] Phase 1: The matching agent selects the action space from the current status of the transfer vehicle. Select the first target action and define it in the state space of the first stage. and reward function Perform the first objective action and observe the rewards and new status of the first stage;

[0131] Phase Two: The transfer vehicle agent operates within the second phase's action space based on the current status of the charging station. Select the second target action and place it in the state space of the second phase. and reward function The second objective action is executed, and the first reward and first new state of the second stage are observed; the charging station agent, based on the current state of the transfer vehicle's battery level, moves from the action space of the second stage. Select the third target action and place it in the state space of the second phase. and reward function Execute the third objective action, and observe the second reward and second new state in the second phase;

[0132] 4) Store the current status, first target action, reward and new status of the transfer vehicle in the first stage, as well as the current status, second target action, first reward and first new status of the charging station in the second stage, and the current status, third target action, second reward and second new status of the transfer vehicle's battery status as a set of experiences in the experience replay cache.

[0133] 5) Randomly select a batch of experiences from the experience replay cache to train DQN, and update the weights of DQN by minimizing the error between the predicted Q value and the target Q value;

[0134] The loss function for training DQN is as follows:

[0135] ;

[0136] In the formula: Indicates the current reward; Indicates the attenuation coefficient; Indicates the state given and actions The maximum target below value; Indicates the state given and actions The model currently predicts value;

[0137] 6) Repeat steps 3) to 5), and through continuous iteration and optimization, DQN will gradually learn how to make the best decisions in the energy-transportation cooperative scheduling model to maximize long-term cumulative rewards;

[0138] 7) Input the current state of the transfer vehicles into the trained DQN and output the optimal energy-traffic coordinated scheduling scheme.

[0139] The present invention also discloses a computer device, comprising: one or more processors;

[0140] The processor is used to store one or more programs;

[0141] When the one or more programs are executed by the one or more processors, the charging scheduling method for electric transfer vehicles in inland waterway ports of the present invention is implemented.

[0142] This invention also discloses a computer-readable storage medium, characterized in that it stores a computer program thereon, which, when executed, implements the charging scheduling method for electric transfer vehicles in inland waterway ports according to this invention. Compared with the prior art, the charging scheduling method for electric transfer vehicles in inland waterway ports of this invention has the following advantages:

[0143] This invention constructs a transfer vehicle task matching model with the goal of maximizing the utilization rate of transfer vehicles and completing all transfer tasks. First, through reasonable task allocation, it ensures that each electric transfer vehicle is fully utilized, reducing empty runs or waiting time and thus improving the overall transportation efficiency of the port. Second, optimizing the task matching model ensures that all transfer tasks can be completed within the specified time, avoiding delays caused by improper vehicle allocation and guaranteeing the completion rate of port transfer tasks. Finally, by improving the utilization rate of transfer vehicles and reducing empty runs, energy consumption and labor costs during transportation can be reduced, thereby lowering the overall operating costs of the port.

[0144] This invention constructs a charging scheduling model for transfer vehicles with the objectives of minimizing delays, maximizing transfer vehicle efficiency, and minimizing charging costs. First, through reasonable scheduling, it ensures that electric transfer vehicles can receive timely battery charging when needed, avoiding transportation delays caused by insufficient battery power and thus guaranteeing the completion of port transfer tasks. Second, optimizing the scheduling model ensures that transfer vehicles complete transfer tasks in the shortest possible time, improving transportation efficiency and reducing waiting and idle time, thereby further improving the overall transportation efficiency of the port. Finally, by rationally scheduling charging times and the use of charging stations, charging costs can be reduced, avoiding increased operating costs due to excessive charging fees, thereby lowering the overall transportation costs of the port.

[0145] This invention constructs an energy-transportation collaborative scheduling model by using a vehicle task matching model as the first-stage model and a vehicle charging scheduling model as the second-stage model. First, by constructing and integrating the two models in stages into a complete energy-transportation collaborative scheduling model, global optimization from task allocation to transportation scheduling can be achieved, ensuring the efficiency and coordination of the entire transportation process. Second, building the model in stages makes the decisions at each stage clearer and more efficient, reducing redundancy and duplication in the decision-making process and improving overall decision-making efficiency. Finally, through global optimization and efficient decision-making, the stability of the entire energy-transportation collaborative system can be enhanced, reducing the risk of system collapse due to problems in a single link.

[0146] This invention utilizes a deep Q-network algorithm to solve energy-traffic cooperative scheduling models. Firstly, the deep Q-network algorithm can handle problems with high-dimensional state and action spaces, demonstrating efficient solution capabilities for complex energy-traffic cooperative scheduling models. Secondly, by combining deep learning and reinforcement learning, the deep Q-network algorithm can learn optimal scheduling strategies, providing optimal scheduling schemes for electric transfer vehicles and battery charging in inland waterway ports. Finally, the deep Q-network algorithm possesses real-time learning and environmental adaptability capabilities, dynamically adjusting the scheduling scheme according to actual conditions to ensure the system's real-time performance and robustness. Attached Figure Description

[0147] To make the objectives, technical solutions, and advantages of the invention clearer, the invention will now be described in further detail with reference to the accompanying drawings, wherein:

[0148] Figure 1 A logic diagram of the charging scheduling method for electric transfer vehicles in inland river ports. Detailed Implementation

[0149] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but only to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention.

[0150] It should be noted that similar reference numerals and letters in the following figures indicate similar items. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. In the description of this invention, it should be noted that the terms "center," "upper," "lower," "left," "right," "vertical," "horizontal," "inner," and "outer," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the figures, or the orientation or positional relationship commonly used when the product is in use. They are only for the convenience of describing the invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on the invention. Furthermore, the terms "first," "second," and "third," etc., are only used to distinguish descriptions and should not be construed as indicating or implying relative importance. In addition, the terms "horizontal," "vertical," etc., do not mean that the component is required to be absolutely horizontal or suspended, but can be slightly tilted. For example, "horizontal" only means that its direction is more horizontal than "vertical," and does not mean that the structure must be completely horizontal, but can be slightly tilted. In the description of this invention, it should also be noted that, unless otherwise explicitly specified and limited, the terms "set," "install," "connect," and "link" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art can understand the specific meaning of the above terms in this invention based on the specific circumstances.

[0151] The following detailed explanation illustrates the specific implementation methods:

[0152] Example 1:

[0153] This embodiment discloses a method for scheduling the charging of electric transfer vehicles in inland waterway ports.

[0154] like Figure 1As shown, the charging scheduling method for electric transfer vehicles in inland waterway ports includes:

[0155] S1: Obtain the port's transshipment tasks and the initial status of all transshipment vehicles;

[0156] S2: Construct a transfer vehicle task matching model with the goal of maximizing the utilization rate of transfer vehicles and completing all transfer tasks;

[0157] S3: Construct a charging scheduling model for transfer vehicles with the objectives of minimizing delays, maximizing transfer vehicle efficiency, and minimizing charging costs;

[0158] S4: Use the transfer vehicle task matching model as the first-stage model and the transfer vehicle charging scheduling model as the second-stage model. Construct an energy-transportation collaborative scheduling model through the first-stage model and the second-stage model.

[0159] S5: Take the initial state of the transfer task and all transfer vehicles as input to the energy-traffic cooperative scheduling model, and solve the energy-traffic cooperative scheduling model with the input of the transfer task and the initial state of all transfer vehicles through the deep Q-network algorithm to generate the optimal energy-traffic cooperative scheduling scheme.

[0160] S6: The port's transfer tasks are completed by scheduling transfer vehicles through the optimal energy-transportation coordinated scheduling scheme.

[0161] This invention constructs a transfer vehicle task matching model with the goal of maximizing the utilization rate of transfer vehicles and completing all transfer tasks. First, through reasonable task allocation, it ensures that each electric transfer vehicle is fully utilized, reducing empty runs or waiting time and thus improving the overall transportation efficiency of the port. Second, optimizing the task matching model ensures that all transfer tasks can be completed within the specified time, avoiding delays caused by improper vehicle allocation and guaranteeing the completion rate of port transfer tasks. Finally, by improving the utilization rate of transfer vehicles and reducing empty runs, energy consumption and labor costs during transportation can be reduced, thereby lowering the overall operating costs of the port.

[0162] This invention constructs a charging scheduling model for transfer vehicles with the objectives of minimizing delays, maximizing transfer vehicle efficiency, and minimizing charging costs. First, through reasonable scheduling, it ensures that electric transfer vehicles can receive timely battery charging when needed, avoiding transportation delays caused by insufficient battery power and thus guaranteeing the completion of port transfer tasks. Second, optimizing the scheduling model ensures that transfer vehicles complete transfer tasks in the shortest possible time, improving transportation efficiency and reducing waiting and idle time, thereby further improving the overall transportation efficiency of the port. Finally, by rationally scheduling charging times and the use of charging stations, charging costs can be reduced, avoiding increased operating costs due to excessive charging fees, thereby lowering the overall transportation costs of the port.

[0163] This invention constructs an energy-transportation collaborative scheduling model by using a vehicle task matching model as the first-stage model and a vehicle charging scheduling model as the second-stage model. First, by constructing and integrating the two models in stages into a complete energy-transportation collaborative scheduling model, global optimization from task allocation to transportation scheduling can be achieved, ensuring the efficiency and coordination of the entire transportation process. Second, building the model in stages makes the decisions at each stage clearer and more efficient, reducing redundancy and duplication in the decision-making process and improving overall decision-making efficiency. Finally, through global optimization and efficient decision-making, the stability of the entire energy-transportation collaborative system can be enhanced, reducing the risk of system collapse due to problems in a single link.

[0164] This invention utilizes a deep Q-network algorithm to solve energy-traffic cooperative scheduling models. Firstly, the deep Q-network algorithm can handle problems with high-dimensional state and action spaces, demonstrating efficient solution capabilities for complex energy-traffic cooperative scheduling models. Secondly, by combining deep learning and reinforcement learning, the deep Q-network algorithm can learn optimal scheduling strategies, providing optimal scheduling schemes for electric transfer vehicles and battery charging in inland waterway ports. Finally, the deep Q-network algorithm possesses real-time learning and environmental adaptability capabilities, dynamically adjusting the scheduling scheme according to actual conditions to ensure the system's real-time performance and robustness.

[0165] In the specific implementation process, the objective function of the transfer vehicle task matching model is expressed as:

[0166] ;

[0167] In the formula: Indicates a transfer task With transfer vehicles Matching indicator, when At that time, it indicates a transfer task. Included in the transfer vehicle In the task list, At that time, it indicates a transfer task. Not included in the transfer vehicle In the task list; each transfer task A tuple Description, in which For pickup point, For delivery nodes, The expected completion time of the task; This represents the reward function for the transfer vehicles in the first-stage model.

[0168] The constraints of the transfer vehicle task matching model include the number of tasks.

[0169] The formula for the task quantity constraint is expressed as:

[0170] ;

[0171] ;

[0172] ;

[0173] In the formula: This indicates the maximum number of tasks.

[0174] Each task must be assigned to only one transport vehicle.

[0175] Constraints An upper limit is set for the total number of tasks, denoted as .

[0176] Constraints Defining decision variables as binary values ​​reflects the discrete nature of their allocation.

[0177] In the specific implementation process, the objective function of the transfer vehicle charging scheduling model is expressed as:

[0178] ;

[0179] ;

[0180] ;

[0181] ;

[0182] In the formula: Indicates a charging station; Indicates the path between nodes; Represents the cost function of a charging station; This indicates the empty-load operating cost of the transfer vehicle; This indicates the cost of purchasing electricity from the grid; Indicates the unit battery rating; This represents the cost of no-load operation per unit of time.

[0183] objective function This represents minimizing the energy-transportation system cost of inland river ports.

[0184] Calculate the cost of electricity procurement.

[0185] The aim is to reduce idle time and optimize the utilization of transfer vehicles.

[0186] Constraints of the charging scheduling model for transport vehicles

[0187] 1. To ensure the balance of transfer vehicles between the terminal cranes and the yard nodes, a power constraint for the transfer vehicles was established.

[0188] In this embodiment, the process of generating the transfer vehicle task list relies on an optimal match between transfer vehicles and tasks. Once these task lists are determined, the second phase involves creating a transfer vehicle scheduling and energy management model. Key decisions at this stage include when to visit a specific charging station or proceed to the next task assignment. To formalize this process, we use binary variables. Its instructions for the transfer vehicles Battery status Is it in time? Actively engage in tasks To facilitate such formulation, the State of Charge (SoC) in the model is treated as a discrete value. It should be noted that this assumption has little impact on the results. Therefore, and The relationships between them are shown below, summarizing their basic connections in the model.

[0189] The constraints of the charging scheduling model for transport vehicles include the power constraints of the transport vehicles.

[0190] The power constraint of the transfer vehicle is expressed as follows:

[0191] ;

[0192] ;

[0193] ;

[0194] In the formula: Indicates transfer vehicle Battery status Is it in time? Actively complete the transfer task ; Indicates a transfer task With transfer vehicles Matching indicator, when At that time, it indicates a transfer task. Included in the transfer vehicle In the task list, At that time, it indicates a transfer task. Not included in the transfer vehicle In the task list; each transfer task A tuple Description, in which For pickup point, For delivery nodes, This refers to the expected completion time of the task.

[0195] Indicates time Status of charge (SoC) at time transfer vehicles Have you been assigned a task? This forms the core decision-making point for each transfer vehicle's participation in the mission.

[0196] Defined in time With a specific SoC The transfer vehicles on the road The number of vehicles traveling on the road reveals the spatial distribution of transport vehicles at different energy levels. This information is crucial for optimizing route planning and resource allocation in the operational environment.

[0197] By requiring tasks Within the scheduled timeframe The system is designed to complete tasks within a specified timeframe, imposing significant time constraints. This time constraint ensures that the system efficiently adheres to planned task completion times, thereby improving overall operational efficiency. These equations collectively underpin the model's fundamental aspects, addressing task allocation, transport vehicle energy management, and time-sensitive task completion in dynamic operating environments.

[0198] By constraining the power consumption of transfer vehicles, operational harmony in cargo loading, unloading, and storage was maintained.

[0199] 2. To emphasize the balance of transfer vehicles at charging station nodes, transfer vehicle state constraints were constructed.

[0200] In this embodiment, constraints are proposed regarding the transfer vehicles on the transportation road, with three variables ( , and The variable represents the number of transport vehicles in the loading, charging, and idle states. Due to the limited battery supply at battery charging stations, transport vehicles may need to wait for batteries to become available. To address this issue, we introduce a variable... , indicating time Battery status is The transfer vehicles at the charging station The number of people waiting. Indicates charging station The total number of batteries, covering overall energy resources. Meanwhile, Indicates time charging station Medium battery status The number of batteries helps to achieve precise energy management and distribution within the operational framework.

[0201] Therefore, the constraints of the charging scheduling model for transport vehicles include the state constraints of the transport vehicles.

[0202] The state constraints of the transfer vehicle are represented as follows:

[0203] ;

[0204] ;

[0205] ;

[0206] In the formula: , and This indicates the number of transport vehicles that are in a loading, charging, or idle state. Indicates time Battery status is The transfer vehicles at the charging station The number of people waiting; Indicates charging station The total number of batteries, covering overall energy resources; Indicates time charging station Medium battery status The number of batteries helps to achieve precise energy management and distribution within the operational framework; This indicates the number of transport vehicles that are currently charging. This indicates the number of transfer vehicles that are currently idle. Indicates the node to be rescheduled; Indicates the battery status.

[0207] The fairness of resource allocation and the simplification of operation procedures were ensured by constraining the state of the transport vehicles. 3. Charging power constraints were constructed to maintain the balance of batteries in different charge states.

[0208] In this embodiment, the constraints of the charging scheduling model for transport vehicles include charging power constraints;

[0209] The charging power constraint is expressed as:

[0210] .

[0211] This constraint defines the charging station In time The charging power. State of Charge (SoC) is calculated by subtracting the cumulative SoC required by the departing transfer vehicles (represented as all). Belongs to set of (the sum of all) and the cumulative required SoC (expressed as all) for vehicles waiting to be transferred. of The difference is determined by subtracting the cumulative total SoC (represented by all) arriving at the transfer vehicles. Belongs to set of The sum of and the vehicles waiting for transfer Adjustments were made, among which This represents the energy equivalent to a unit SoC.

[0212] Charging power constraints make charging stations The total number of batteries is kept in line with their availability, thereby optimizing energy management and overall system efficiency.

[0213] These three constraints together promote the smooth operation and effectiveness of the system.

[0214] In practice, the processing steps of the deep Q-network algorithm are as follows:

[0215] S501: Transform the (mixed-integer nonlinear) energy-traffic cooperative scheduling model, which has the input of transfer tasks and the initial states of all transfer vehicles, into a Markov model.

[0216] S502: Generates the optimal energy-traffic coordinated scheduling scheme by combining a two-stage deep Q-network algorithm with a Markov model.

[0217] 1. The resulting Markov models include:

[0218] 1) State space, action space, and reward function in the first stage

[0219] ;

[0220] ;

[0221] ;

[0222] In the formula: , , This represents the state space, the first objective action, and the reward function for the first stage. Indicates a transfer task; Indicates the date; , This indicates the initial location and initial battery status of the transport vehicle; Indicates the first One transfer vehicle; This represents the reward function for the transfer vehicles in the first-stage model; This represents the reward function for the second phase. Indicates a transfer task With transfer vehicles Matching indicator, when At that time, it indicates a transfer task. Included in the transfer vehicle In the task list, At that time, it indicates a transfer task. Not included in the transfer vehicle In the task list;

[0223] 2) The state space, action space, and reward function of the second stage

[0224] ;

[0225] ;

[0226] ;

[0227] ;

[0228] ;

[0229] In the formula: , Represents the first and second state spaces of the second stage; , This indicates the second and third objective actions in the second phase; Indicates time; express Location of the time transfer vehicle; express Battery status of the time-transfer vehicle; Indicates a transfer task; This indicates the cost of purchasing electricity from the grid; express Time Charging Station Battery status of the transfer vehicles; Indicates the first One charging station; Indicates the first The battery status of each transport vehicle; Indicates the unit battery rating; This represents the cost of no-load operation per unit time. , , This indicates the number of transport vehicles that are in an idle, charging, or waiting state. Indicates transfer vehicle Battery status Is it in time? Actively complete the transfer task .

[0230] In the first phase, the matching agent selects a transfer vehicle based on the current state, making a pre-selection for each task from the set of possible actions. This state includes factors such as a specific date. Task characteristics Initial position of the transfer vehicle and initial SoC Information such as...

[0231] In the second phase, the transfer vehicle agent makes decisions from its operational space based on its observable environmental conditions. These conditions include factors such as date, current time, transfer vehicle location, SoC, task list, and electricity price.

[0232] After the vehicle transfer agent makes a decision, the charging station agent determines the optimal power procurement from its operational space based on its perceived environmental conditions. These conditions include factors such as date. ,time Location of the transfer vehicle at that time point The SoC of the instantaneous transfer vehicle Charging stations Battery status (represented as) ) and current price Information such as these. The agent will perform state transitions through a sequential decision-making process and state transition equations.

[0233] In the first phase, the rewards for matching agents include rewards for the utilization rate of transport vehicles.

[0234] In the second phase, the reward function for transport vehicles and charging station agents includes three key elements: electricity procurement costs, penalties for idle driving time, and penalties for task delays. These components collectively shape agent behavior, promote effective energy management, reduce inefficient transport vehicle movement, and emphasize the importance of timely task completion within the operational framework.

[0235] 2. By integrating Q-learning and deep learning, the DQN method demonstrates significant effectiveness in solving discrete action-based scheduling problems. The output of the DQN model uses the current state as input and provides Q-values ​​for all feasible discrete actions. A two-stage DQN system illustrates the interaction of three agents in a port environment. Each agent's actions can potentially affect the state information of the environment, which can also serve as input data for other agents.

[0236] In the first phase, the matching agent can be selected. Each action is associated with an optimal action, and each action is linked to a Q-value embedded in its neural network. Similarly, in the second phase, the transfer vehicle agent and the charging station agent operate within their respective decision spaces. The decision set of the transfer vehicle agent includes... There are several optimal actions, each associated with its Q-value, and the charging station agent uses... An optimal action and its corresponding Q-value. This flexible action selection system enables the agent to make informed decisions and adapt to changing operating conditions and requirements.

[0237] Unlike supervised learning methods, DQN does not provide labeled samples for neural network learning. Therefore, this invention specifically creates two neural networks for the DQN method: a target network and a current network. The target network serves as a reference guiding algorithm updates, similar to the role of ground truth in supervised learning. The current network and the target network share the same parameters and structure, as they are initially initialized with the same values. As training progresses, the current network updates faster than the target network. The mean squared error between the target Q-value and the predicted Q-value is called the loss function. The parameters of the current network are modified based on the gradient of the calculated loss function. The learning process converges when the outputs of the target network and the current network are sufficiently close.

[0238] Specifically, the processing steps of the Q-network method are as follows:

[0239] S5021: Design a DQN (i.e., a deep Q-network, which combines a deep neural network (DNN) and a reinforcement learning algorithm for Q-learning) to approximate the Q-value function and randomly initialize its weights. The DQN is used to predict the Q-value of each action based on the current state.

[0240] S5022: Create an experience replay cache to store the states, actions, rewards, and new states generated by the interaction between the matching agent, the transfer vehicle agent, and the charging station agent and the Markov model;

[0241] S5023: Each time step includes the following two phases:

[0242] Phase 1: The matching agent selects the action space from the current status of the transfer vehicle. Select the first target action and define it in the state space of the first stage. and reward function Perform the first objective action and observe the rewards and new status of the first stage;

[0243] Phase Two: The transfer vehicle agent operates within the second phase's action space based on the current status of the charging station. Select the second target action and place it in the state space of the second phase. and reward function The second objective action is executed, observing the first reward and first new state of the first stage; then the charging station agent moves from the action space of the second stage based on the current state of the transfer vehicle's battery level. Select the third target action and place it in the state space of the second phase. and reward function Execute the third objective action, and observe the second reward and second new state in the first phase;

[0244] S5024: Store the current status, first target action, reward and new status of the transfer vehicle in the first stage, the current status, second target action, first reward and first new status of the charging station in the second stage, and the current status, third target action, second reward and second new status of the transfer vehicle's battery status as a set of experiences in the experience replay cache.

[0245] S5025: Randomly select a batch of experiences from the experience replay cache to train DQN, and update the weights of DQN by minimizing the error between the predicted Q value and the target Q value;

[0246] The loss function for training DQN is as follows:

[0247] ;

[0248] In the formula: Indicates the current reward; Indicates the attenuation coefficient; Indicates the state given and actions The maximum target below value; Indicates the state given and actions The model currently predicts value;

[0249] S5026: Repeat steps S5023 to S5025. Through continuous iteration and optimization, DQN will gradually learn how to make the best decisions in the energy-transportation coordinated scheduling model to maximize long-term cumulative rewards.

[0250] S5027: Input the current state of the transfer vehicles into the trained DQN and output the optimal energy-traffic cooperative scheduling scheme.

[0251] In this embodiment, after the DQN is trained, the current environmental state is input, including the transfer task and the initial state of all vehicles. Then, the DQN outputs a set of action values ​​(Q values) for the current state. Each action value corresponds to a possible action (i.e., an energy-traffic cooperative scheduling scheme), representing the expected cumulative reward obtained by performing the action in this state.

[0252] The following are specific examples of evaluation using the method of the present invention.

[0253] The method of this invention is verified using a real-world example of a port, and compared with traditional mathematical programming methods as follows:

[0254] Table 1 Comparison of computational performance between DQN and mathematical programming

[0255]

[0256] After training, the DQN reinforcement learning agent used a week of real-world data to improve its EV scheduling strategy at the port (electric transshipment). Notably, as shown in Table 1, the DQN strategy outperformed mathematical programming in terms of empty run rate and electricity procurement cost. This highlights the effectiveness of DQN in reducing empty run and minimizing energy procurement expenses, making it a valuable option for system optimization. Mathematical programming faces challenges posed by binary matching variables. The challenge of creating a high-dimensional search space makes finding the optimal solution difficult with limited computational resources. To address this, we employ preprocessing methods to reduce the search space, taking into account job details such as start and end nodes. However, the computational results demonstrate the superior performance of DQN. Furthermore, the outstanding speed of DQN strategies, requiring only 0.06 minutes, is attributed to their efficient neural networks, which rapidly find the optimal solution, contrasting sharply with the inherent complexity and nonlinearity of mathematical programming.

[0257] Example 2:

[0258] This embodiment discloses a charging scheduling system for electric transfer vehicles in inland waterway ports, which is implemented based on the charging scheduling method for electric transfer vehicles in inland waterway ports in Embodiment 1.

[0259] The inland waterway port electric transfer vehicle charging dispatch system includes:

[0260] The acquisition module is used to acquire the port's transshipment tasks and the initial status of all transshipment vehicles;

[0261] The first model building module is used to build a transfer vehicle task matching model with the goal of maximizing the utilization of transfer vehicles and completing all transfer tasks.

[0262] The second model building module is used to build a charging scheduling model for transfer vehicles with the goals of minimizing delays, maximizing transfer vehicle efficiency, and minimizing charging costs.

[0263] The third model construction module is used to use the transfer vehicle task matching model as the first-stage model and the transfer vehicle charging scheduling model as the second-stage model, and to construct an energy-transportation collaborative scheduling model through the first-stage model and the second-stage model.

[0264] The calculation module is used to take the transfer task and the initial state of all transfer vehicles as input to the energy-traffic cooperative scheduling model, and solve the energy-traffic cooperative scheduling model with the input of the transfer task and the initial state of all transfer vehicles through the deep Q-network algorithm to generate the optimal energy-traffic cooperative scheduling scheme.

[0265] The execution module is used to schedule transfer vehicles to complete the port's transfer tasks through the optimal energy-transportation coordinated scheduling scheme.

[0266] In this embodiment, the objective function of the transfer vehicle task matching model in the first model construction module is expressed as:

[0267] ;

[0268] In the formula: Indicates a transfer task With transfer vehicles Matching indicator, when At that time, it indicates a transfer task. Included in the transfer vehicle In the task list, At that time, it indicates a transfer task. Not included in the transfer vehicle In the task list; each transfer task A tuple Description, in which For pickup point, For delivery nodes, The expected completion time of the task; This represents the reward function for the transfer vehicles in the first-stage model.

[0269] In this embodiment, the constraints of the transfer vehicle task matching model in the first model construction module include the task quantity constraint.

[0270] The task quantity constraint is expressed as:

[0271] ;

[0272] ;

[0273] ;

[0274] In the formula: This indicates the maximum number of tasks.

[0275] In this embodiment, the objective function of the transfer vehicle charging scheduling model in the second model construction module is expressed as:

[0276] ;

[0277] ;

[0278] ;

[0279] In the formula: Indicates a charging station; Indicates the path between nodes; Represents the cost function of a charging station; This indicates the empty-load operating cost of the transfer vehicle; This indicates the cost of purchasing electricity from the grid; Indicates the unit battery rating; This represents the cost of no-load operation per unit of time.

[0280] In this embodiment, the constraints of the transfer vehicle charging scheduling model in the second model construction module include the transfer vehicle power constraints.

[0281] The power constraint of the transfer vehicle is expressed as follows:

[0282] ;

[0283] ;

[0284] ;

[0285] In the formula: Indicates transfer vehicle Battery status Is it in time? Actively complete the transfer task ; Indicates a transfer task With transfer vehicles Matching indicator, when At that time, it indicates a transfer task. Included in the transfer vehicle In the task list, At that time, it indicates a transfer task. Not included in the transfer vehicle In the task list; each transfer task A tuple Description, in which For pickup point, For delivery nodes, This refers to the expected completion time of the task.

[0286] In this embodiment, the constraints of the transfer vehicle charging scheduling model in the second model construction module include transfer vehicle state constraints.

[0287] The state constraints of the transfer vehicle are represented as follows:

[0288] ;

[0289] ;

[0290] ;

[0291] In the formula: , and This indicates the number of transport vehicles that are in a loading, charging, or idle state. Indicates time Battery status is The transfer vehicles at the charging station The number of people waiting; Indicates charging station The total number of batteries; Indicates time charging station Medium battery status The number of batteries; This indicates the number of transfer vehicles that are currently being loaded. This indicates the number of transport vehicles that are currently charging. This indicates the number of transfer vehicles that are currently idle. Indicates the node to be rescheduled; Indicates the battery status.

[0292] In this embodiment, the constraints of the transfer vehicle charging scheduling model in the second model construction module include charging power constraints.

[0293] The charging power constraint is expressed as:

[0294] ;

[0295] In the formula: This represents the energy equivalent to a unit SoC.

[0296] In this embodiment, the processing steps of the deep Q-network algorithm in the calculation module are as follows:

[0297] Step 1: Convert the energy-traffic cooperative scheduling model with input transfer tasks and the initial states of all transfer vehicles into a Markov model;

[0298] Step 2: Generate the optimal energy-traffic coordinated scheduling scheme by combining a two-stage deep Q-network algorithm with a Markov model.

[0299] In this embodiment, the Markov model obtained in step one of the calculation module includes:

[0300] 1) State space, action space, and reward function in the first stage

[0301] ;

[0302] ;

[0303] ;

[0304] In the formula: , , This represents the state space, the first objective action, and the reward function for the first stage. Indicates a transfer task; Indicates the date; , This indicates the initial location and initial battery status of the transport vehicle; Indicates the first One transfer vehicle; This represents the reward function for the transfer vehicles in the first-stage model; This represents the reward function for the second phase. Indicates a transfer task With transfer vehicles Matching indicator, when At that time, it indicates a transfer task. Included in the transfer vehicle In the task list, At that time, it indicates a transfer task. Not included in the transfer vehicle In the task list;

[0305] 2) The state space, action space, and reward function of the second stage

[0306] ;

[0307] ;

[0308] ;

[0309] ;

[0310] ;

[0311] In the formula: , Represents the first and second state spaces of the second stage; , This indicates the second and third objective actions in the second phase; Indicates time; express Location of the time transfer vehicle; express Battery status of the time-transfer vehicle; Indicates a transfer task; This indicates the cost of purchasing electricity from the grid; express Time Charging Station Battery status of the transfer vehicles; Indicates the first One charging station; Indicates the first The battery status of each transport vehicle; Indicates the unit battery rating; This represents the cost of no-load operation per unit time. , , This indicates the number of transport vehicles that are in an idle, charging, or waiting state. Indicates transfer vehicle Battery status Is it in time? Actively complete the transfer task .

[0312] In this embodiment, the processing steps of the deep Q-network method in step two of the calculation module are as follows:

[0313] 1) Design a DQN to approximate the Q-value function and randomly initialize its weights. The DQN is used to predict the Q-value of each action based on the current state.

[0314] 2) Create an experience replay cache to store the states, actions, rewards, and new states generated by the interaction between the matching agent, the transfer vehicle agent, and the charging station agent and the Markov model;

[0315] 3) Each time step includes the following two phases:

[0316] Phase 1: The matching agent selects the action space from the current status of the transfer vehicle. Select the first target action and define it in the state space of the first stage. and reward function Perform the first objective action and observe the rewards and new status of the first stage;

[0317] Phase Two: The transfer vehicle agent operates within the second phase's action space based on the current status of the charging station. Select the second target action and place it in the state space of the second phase. and reward function The second objective action is executed, and the first reward and first new state of the second stage are observed; the charging station agent, based on the current state of the transfer vehicle's battery level, moves from the action space of the second stage. Select the third target action and place it in the state space of the second phase. and reward function Execute the third objective action, and observe the second reward and second new state in the second phase;

[0318] 4) Store the current status, first target action, reward and new status of the transfer vehicle in the first stage, as well as the current status, second target action, first reward and first new status of the charging station in the second stage, and the current status, third target action, second reward and second new status of the transfer vehicle's battery status as a set of experiences in the experience replay cache.

[0319] 5) Randomly select a batch of experiences from the experience replay cache to train DQN, and update the weights of DQN by minimizing the error between the predicted Q value and the target Q value;

[0320] The loss function for training DQN is as follows:

[0321] ;

[0322] In the formula: Indicates the current reward; Indicates the attenuation coefficient; Indicates the state given and actions The maximum target below value; Indicates the state given and actions The model currently predicts value;

[0323] 6) Repeat steps 3) to 5), and through continuous iteration and optimization, DQN will gradually learn how to make the best decisions in the energy-transportation cooperative scheduling model to maximize long-term cumulative rewards;

[0324] 7) Input the current state of the transfer vehicles into the trained DQN and output the optimal energy-traffic coordinated scheduling scheme.

[0325] Example 3:

[0326] This embodiment discloses a computer device.

[0327] The computer device includes a processor, a memory, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the steps in the above embodiments of the charging scheduling methods for electric transfer vehicles in inland waterway ports. Alternatively, when the processor executes the computer program, it implements the functions of each module / unit in the above embodiments of the systems.

[0328] For example, the computer program can be divided into one or more modules / units, which are stored in the memory and executed by the processor to complete the present invention. The one or more modules / units can be a series of computer program instruction segments capable of performing specific functions, which describe the execution process of the computer program in the dynamic reactive power compensation device of the power system.

[0329] The computer device may be a desktop computer, laptop, handheld computer, or cloud server, etc. The computer device may include, but is not limited to, a processor and memory.

[0330] Example 4:

[0331] This embodiment discloses a computer-readable storage medium.

[0332] A computer-readable storage medium having a computer program thereon, wherein when the computer program is executed, the steps in the embodiments of the above-described methods for charging and scheduling electric transfer vehicles at inland waterway ports are performed.

[0333] The computer program can be stored in a computer-readable storage medium. When executed by a processor, the computer program can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable file, or some intermediate form. The computer-readable medium can include any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a portable hard drive, a magnetic disk, an optical disk, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium, etc. Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit the technical solutions. Those skilled in the art should understand that any modifications or equivalent substitutions to the technical solutions of the present invention without departing from the spirit and scope of the present invention should be covered within the scope of the claims of the present invention.

Claims

1. A method for scheduling the charging of electric transfer vehicles in inland waterway ports, characterized in that, include: S1: Obtain the port's transshipment tasks and the initial status of all transshipment vehicles; S2: Construct a transfer vehicle task matching model with the goal of maximizing the utilization rate of transfer vehicles and completing all transfer tasks; In step S2, the objective function of the transfer vehicle task matching model is expressed as: ； In the formula: Indicates a transfer task With transfer vehicles Matching indicator, when At that time, it indicates a transfer task. Included in the transfer vehicle In the task list, At that time, it indicates a transfer task. Not included in the transfer vehicle In the task list; each transfer task A tuple Description, in which For pickup point, For delivery nodes, The expected completion time of the task; This represents the reward function for the transfer vehicles in the first-stage model; S3: Construct a charging scheduling model for transfer vehicles with the objectives of minimizing delays, maximizing transfer vehicle efficiency, and minimizing charging costs; In step S3, the objective function of the transfer vehicle charging scheduling model is expressed as: ； ； ； In the formula: Indicates a charging station; Indicates the path between nodes; Represents the cost function of a charging station; This indicates the empty-load operating cost of the transfer vehicle; This indicates the cost of purchasing electricity from the grid; Indicates the unit battery rating; This represents the cost of no-load operation per unit time. and This indicates the number of transport vehicles that are either charging or idle. Indicates time Battery status is The transfer vehicles at the charging station The number of people waiting; S4: Use the transfer vehicle task matching model as the first-stage model and the transfer vehicle charging scheduling model as the second-stage model. Construct an energy-transportation collaborative scheduling model through the first-stage model and the second-stage model. S5: Take the initial state of the transfer task and all transfer vehicles as input to the energy-traffic cooperative scheduling model, and solve the energy-traffic cooperative scheduling model with the input of the transfer task and the initial state of all transfer vehicles through the deep Q-network algorithm to generate the optimal energy-traffic cooperative scheduling scheme. S6: Dispatch transfer vehicles using the optimal energy-transportation coordinated scheduling scheme to complete the port's transfer tasks.

2. The charging scheduling method for electric transfer vehicles in inland waterway ports as described in claim 1, characterized in that: In step S2, the constraints of the transfer vehicle task matching model include the task quantity constraint. The task quantity constraint is expressed as: ； ； ； In the formula: This indicates the maximum number of tasks.

3. The charging scheduling method for electric transfer vehicles in inland waterway ports as described in claim 1, characterized in that: In step S3, the constraints of the transfer vehicle charging scheduling model include the transfer vehicle power constraints. The power constraint of the transfer vehicle is expressed as follows: ； ； ； In the formula: Indicates transfer vehicle Battery status Is it in time? Actively complete the transfer task ; Indicates a transfer task With transfer vehicles Matching indicator, when At that time, it indicates a transfer task. Included in the transfer vehicle In the task list, At that time, it indicates a transfer task. Not included in the transfer vehicle In the task list; each transfer task A tuple Description, in which For pickup point, For delivery nodes, This refers to the expected completion time of the task.

4. The charging scheduling method for electric transfer vehicles in inland waterway ports as described in claim 1, characterized in that: In step S3, the constraints of the charging scheduling model for the transfer vehicles include the state constraints of the transfer vehicles. The state constraints of the transfer vehicle are represented as follows: ； ； ； In the formula: , and This indicates the number of transport vehicles that are in a loading, charging, or idle state. Indicates time Battery status is The transfer vehicles at the charging station The number of people waiting; Indicates charging station The total number of batteries; Indicates time charging station Medium battery status The number of batteries; This indicates the number of transfer vehicles that are currently being loaded. This indicates the number of transport vehicles that are currently charging. This indicates the number of transfer vehicles that are currently idle. 'c' represents the node to be rescheduled; 'c' represents the battery status.

5. The charging scheduling method for electric transfer vehicles in inland waterway ports as described in claim 1, characterized in that: In step S3, the constraints of the charging scheduling model for transport vehicles include charging power constraints. The charging power constraint is expressed as: ； In the formula: This represents the energy equivalent to a unit SoC.

6. The charging scheduling method for electric transfer vehicles in inland waterway ports as described in claim 1, characterized in that: In step S5, the processing steps of the deep Q-network algorithm are as follows: S501: Transform the energy-traffic cooperative scheduling model with inputs of the initial states of the transfer tasks and all transfer vehicles into a Markov model; S502: Generates the optimal energy-traffic coordinated scheduling scheme by combining a two-stage deep Q-network algorithm with a Markov model.

7. The charging scheduling method for electric transfer vehicles in inland waterway ports as described in claim 6, characterized in that: In step S501, the resulting Markov model includes: 1) State space, action space, and reward function in the first stage ； ； ； In the formula: , , This represents the state space, the first objective action, and the reward function for the first stage. Indicates a transfer task; d represents the date; , This indicates the initial location and initial battery status of the transport vehicle; This indicates the Vth transfer vehicle; This represents the reward function for the transfer vehicles in the first-stage model; This represents the reward function for the second phase. Indicates a transfer task With transfer vehicles Matching indicator, when At that time, it indicates a transfer task. Included in the transfer vehicle In the task list, At that time, it indicates a transfer task. Not included in the transfer vehicle In the task list; 2) The state space, action space, and reward function of the second stage ； ； ； ； ； In the formula: , Represents the first and second state spaces of the second stage; , This indicates the second and third objective actions in the second phase; Indicates time; express Location of the time transfer vehicle; express Battery status of the time-transfer vehicle; Indicates a transfer task; This indicates the cost of purchasing electricity from the grid; express Time Charging Station Battery status of the transfer vehicles; Indicates the first One charging station; Indicates the first The battery status of each transport vehicle; Indicates the unit battery rating; This represents the cost of no-load operation per unit time. , , This indicates the number of transport vehicles that are in an idle, charging, or waiting state. Indicates transfer vehicle Battery status Is it in time? Actively complete the transfer task .

8. The charging scheduling method for electric transfer vehicles in inland waterway ports as described in claim 7, characterized in that: In step S502, the processing steps of the deep Q-network method are as follows: S5021: Design a DQN to approximate the Q-value function and randomly initialize its weights. The DQN is used to predict the Q-value of each action based on the current state. S5022: Create an experience replay cache to store the states, actions, rewards, and new states generated by the interaction between the matching agent, the transfer vehicle agent, and the charging station agent and the Markov model; S5023: Each time step includes the following two phases: Phase 1: The matching agent selects the action space from the current status of the transfer vehicle. Select the first target action and define it in the state space of the first stage. and reward function Perform the first objective action and observe the rewards and new status of the first stage; Phase Two: The transfer vehicle agent operates within the second phase's action space based on the current status of the charging station. Select the second target action and place it in the state space of the second phase. and reward function Execute the second objective action and observe the first reward and first new state in the second phase; The charging station agent operates within the second phase of the action space based on the current battery status of the transport vehicle. Select the third target action and place it in the state space of the second phase. and reward function Execute the third objective action, and observe the second reward and second new state in the second phase; S5024: Store the current status, first target action, reward and new status of the transfer vehicle in the first stage, the current status, second target action, first reward and first new status of the charging station in the second stage, and the current status, third target action, second reward and second new status of the transfer vehicle's battery status as a set of experiences in the experience replay cache. S5025: Randomly select a batch of experiences from the experience replay cache to train DQN, and update the weights of DQN by minimizing the error between the predicted Q value and the target Q value; The loss function for training DQN is as follows: ； In the formula: Indicates the current reward; Indicates the attenuation coefficient; Indicates the state given and actions The target maximum Q value; Indicates the state given and actions The current Q value predicted by the model; S5026: Repeat steps S5023 to S5025. Through continuous iteration and optimization, DQN will gradually learn how to make the best decisions in the energy-transportation coordinated scheduling model to maximize long-term cumulative rewards. S5027: Input the current state of the transfer vehicles into the trained DQN and output the optimal energy-traffic cooperative scheduling scheme.

9. A charging and dispatching system for electric transfer vehicles in inland waterway ports, characterized in that: The system is implemented based on the charging and scheduling method for electric transfer vehicles in inland waterway ports according to claim 1; the system includes: The acquisition module is used to acquire the port's transshipment tasks and the initial status of all transshipment vehicles; The first model building module is used to build a transfer vehicle task matching model with the goal of maximizing the utilization of transfer vehicles and completing all transfer tasks. The second model building module is used to build a charging scheduling model for transfer vehicles with the goals of minimizing delays, maximizing transfer vehicle efficiency, and minimizing charging costs. The third model construction module is used to use the transfer vehicle task matching model as the first-stage model and the transfer vehicle charging scheduling model as the second-stage model, and to construct an energy-transportation collaborative scheduling model through the first-stage model and the second-stage model. The calculation module is used to take the transfer task and the initial state of all transfer vehicles as input to the energy-traffic cooperative scheduling model, and solve the energy-traffic cooperative scheduling model with the input of the transfer task and the initial state of all transfer vehicles through the deep Q-network algorithm to generate the optimal energy-traffic cooperative scheduling scheme. The execution module is used to schedule transfer vehicles to complete the port's transfer tasks through the optimal energy-transportation coordinated scheduling scheme.

10. The inland waterway port electric transfer vehicle charging and dispatching system as described in claim 9, characterized in that: In the first model construction module, the objective function of the transfer vehicle task matching model is expressed as: ； In the formula: Indicates a transfer task With transfer vehicles Matching indicator, when At that time, it indicates a transfer task. Included in the transfer vehicle In the task list, At that time, it indicates a transfer task. Not included in the transfer vehicle In the task list; each transfer task A tuple Description, in which For pickup point, For delivery nodes, The expected completion time of the task; This represents the reward function for the transfer vehicles in the first-stage model.

11. The inland waterway port electric transfer vehicle charging and dispatching system as described in claim 10, characterized in that: In the first model construction module, the constraints of the transfer vehicle task matching model include the task quantity constraint; The task quantity constraint is expressed as: ； ； ； In the formula: This indicates the maximum number of tasks.

12. The inland waterway port electric transfer vehicle charging and dispatching system as described in claim 9, characterized in that: In the second model construction module, the objective function of the transfer vehicle charging scheduling model is expressed as: ； ； ； In the formula: Indicates a charging station; Indicates the path between nodes; Represents the cost function of a charging station; This indicates the empty-load operating cost of the transfer vehicle; This indicates the cost of purchasing electricity from the grid; Indicates the unit battery rating; This represents the cost of no-load operation per unit of time.

13. The inland waterway port electric transfer vehicle charging and dispatching system as described in claim 12, characterized in that: In the second model construction module, the constraints of the transfer vehicle charging scheduling model include the transfer vehicle power constraints. The power constraint of the transfer vehicle is expressed as follows: ； ； ； In the formula: Indicates transfer vehicle Battery status Is it in time? Actively complete the transfer task ; Indicates a transfer task With transfer vehicles Matching indicator, when At that time, it indicates a transfer task. Included in the transfer vehicle In the task list, At that time, it indicates a transfer task. Not included in the transfer vehicle In the task list; each transfer task A tuple Description, in which For pickup point, For delivery nodes, This refers to the expected completion time of the task.

14. The inland waterway port electric transfer vehicle charging and dispatching system as described in claim 12, characterized in that: In the second model construction module, the constraints of the transfer vehicle charging scheduling model include transfer vehicle state constraints. The state constraints of the transfer vehicle are represented as follows: ； ； ； In the formula: , and This indicates the number of transport vehicles that are in a loading, charging, or idle state. Indicates time Battery status is The transfer vehicles at the charging station The number of people waiting; Indicates charging station The total number of batteries; Indicates time charging station Medium battery status The number of batteries; This indicates the number of transfer vehicles that are currently being loaded. This indicates the number of transport vehicles that are currently charging. This indicates the number of transfer vehicles that are currently idle. Indicates the node to be rescheduled; Indicates the battery status.

15. The inland waterway port electric transfer vehicle charging and dispatching system as described in claim 12, characterized in that: In the second model construction module, the constraints of the transfer vehicle charging scheduling model include charging power constraints; The charging power constraint is expressed as: ； In the formula: This represents the energy equivalent to a unit SoC.

16. The inland waterway port electric transfer vehicle charging and dispatching system as described in claim 9, characterized in that: In the computation module, the processing steps of the deep Q-network algorithm are as follows: Step 1: Convert the energy-traffic cooperative scheduling model with input transfer tasks and the initial states of all transfer vehicles into a Markov model; Step 2: Generate the optimal energy-traffic coordinated scheduling scheme by combining a two-stage deep Q-network algorithm with a Markov model.

17. The inland waterway port electric transfer vehicle charging and dispatching system as described in claim 16, characterized in that: In step one of the calculation module, the resulting Markov model includes: 1) State space, action space, and reward function in the first stage ； ； ； In the formula: , , This represents the state space, the first objective action, and the reward function for the first stage. Indicates a transfer task; Indicates the date; , This indicates the initial location and initial battery status of the transport vehicle; Indicates the first One transfer vehicle; This represents the reward function for the transfer vehicles in the first-stage model; This represents the reward function for the second phase. Indicates a transfer task With transfer vehicles Matching indicator, when At that time, it indicates a transfer task. Included in the transfer vehicle In the task list, At that time, it indicates a transfer task. Not included in the transfer vehicle In the task list; 2) The state space, action space, and reward function of the second stage ； ； ； ； ； In the formula: , Represents the first and second state spaces of the second stage; , This indicates the second and third objective actions in the second phase; Indicates time; express Location of the time transfer vehicle; express Battery status of the time-transfer vehicle; Indicates a transfer task; This indicates the cost of purchasing electricity from the grid; express Time Charging Station Battery status of the transfer vehicles; Indicates the first One charging station; Indicates the first The battery status of each transport vehicle; Indicates the unit battery rating; This represents the cost of no-load operation per unit time. , , This indicates the number of transport vehicles that are in an idle, charging, or waiting state. Indicates transfer vehicle Battery status Is it in time? Actively complete the transfer task .

18. The inland waterway port electric transfer vehicle charging and dispatching system as described in claim 16, characterized in that: In step two of the computation module, the processing steps of the deep Q-network method are as follows: 1) Design a DQN to approximate the Q-value function and randomly initialize its weights. The DQN is used to predict the Q-value of each action based on the current state. 2) Create an experience replay cache to store the states, actions, rewards, and new states generated by the interaction between the matching agent, the transfer vehicle agent, and the charging station agent and the Markov model; 3) Each time step includes the following two phases: Phase 1: The matching agent selects the action space from the current status of the transfer vehicle. Select the first target action and define it in the state space of the first stage. and reward function Perform the first objective action and observe the rewards and new status of the first stage; Phase Two: The transfer vehicle agent operates within the second phase's action space based on the current status of the charging station. Select the second target action and place it in the state space of the second phase. and reward function Execute the second objective action and observe the first reward and first new state in the second phase; The charging station agent operates within the second phase of the action space based on the current battery status of the transport vehicle. Select the third target action and place it in the state space of the second phase. and reward function Execute the third objective action, and observe the second reward and second new state in the second phase; 4) Store the current status, first target action, reward and new status of the transfer vehicle in the first stage, as well as the current status, second target action, first reward and first new status of the charging station in the second stage, and the current status, third target action, second reward and second new status of the transfer vehicle's battery status as a set of experiences in the experience replay cache. 5) Randomly select a batch of experiences from the experience replay cache to train DQN, and update the weights of DQN by minimizing the error between the predicted Q value and the target Q value; The loss function for training DQN is as follows: ； In the formula: Indicates the current reward; Indicates the attenuation coefficient; Indicates the state given and actions The maximum target below value; Indicates the state given and actions The model currently predicts value; 6) Repeat steps 3) to 5), and through continuous iteration and optimization, DQN will gradually learn how to make the best decisions in the energy-transportation cooperative scheduling model to maximize long-term cumulative rewards; 7) Input the current state of the transfer vehicles into the trained DQN and output the optimal energy-traffic coordinated scheduling scheme.

19. A computer device, characterized in that, include: One or more processors; The processor is used to store one or more programs; When the one or more programs are executed by the one or more processors, the charging scheduling method for electric transfer vehicles in inland waterway ports as described in any one of claims 1 to 8 is implemented.

20. A computer-readable storage medium, characterized in that, It contains a computer program, which, when executed, implements the charging scheduling method for electric transfer vehicles in inland waterway ports as described in any one of claims 1 to 8.

Images