Port multi-carrier collaborative scheduling optimization system based on deep reinforcement learning

Abstract

The invention relates to the technical field of port scheduling, and discloses a port multi-carrier collaborative scheduling optimization system based on deep reinforcement learning, and the system comprises a state sensing module which obtains the position, state and task progress data of multiple carriers; the multi-carrier scheduling optimization model module constructs a collaborative scheduling optimization problem model, wherein the collaborative scheduling optimization problem model comprises a target function and a constraint condition; the scheduling problem solving module is used for solving the collaborative scheduling optimization problem model and generating a collaborative scheduling strategy with the minimum energy consumption and the shortest time consumption; the action execution module converts the collaborative scheduling strategy into an actual port operation instruction; the fault statistics module calculates a carrier fault rate and formulates a carrier maintenance strategy; and the maintenance management module carries out maintenance recording on the fault carrier so as to carry out preventive maintenance management. According to the method, multi-source data are integrated, the collaborative scheduling optimization model is constructed, dual-target solution is realized, and the technical problems of low scheduling efficiency, high energy consumption and maintenance strategy lagging in the traditional technology are solved by combining fault statistics and preventive maintenance.

Description

Technical Field

[0001] The present invention relates to the technical field of port scheduling, and in particular to a port multi-vehicle collaborative scheduling optimization system based on deep reinforcement learning. Background Art

[0002] With the transformation of my country's industrial structure and the introduction of the Belt and Road Initiative, trade volume continues to grow, leading to the rapid development of multimodal transport, such as containerized rail-water transport. Furthermore, the transportation industry is highly energy-dependent, and its rapid development inevitably leads to significant energy consumption, making energy conservation and emission reduction a critical consideration. Ports, as key nodes in sea-rail transport, primarily handle cargo loading and unloading operations. Therefore, in this port context, studying the coordinated scheduling of rail-water transport loading and unloading equipment, while considering both efficiency and energy consumption, is of great value.

[0003] In terms of coordinated scheduling of port equipment, the efficiency of coordinated scheduling of multiple vehicles in port logistics operations directly affects the port throughput and operating costs, while the traditional port scheduling management model has significant shortcomings: on the one hand, there is a lack of real-time data integration mechanism for the location, status and task progress of multiple vehicles, resulting in a lack of comprehensive and real-time data support for scheduling decisions, making it difficult to achieve efficient coordination of multiple vehicles; on the other hand, most existing scheduling models only optimize for a single goal and cannot take into account the dual optimization needs of energy consumption and operation time at the same time. In addition, a fault statistics and preventive maintenance system has not been established. The post-fault repair model of multiple vehicles is prone to cause operation interruptions and increased maintenance costs. Summary of the Invention

[0004] The present invention provides a port multi-vehicle collaborative scheduling optimization system based on deep reinforcement learning. It integrates multi-source data, constructs a collaborative scheduling optimization model, realizes dual-objective solution, and combines fault statistics with preventive maintenance to achieve port multi-vehicle scheduling optimization, solving the technical problems of low scheduling efficiency, high energy consumption, and lagging maintenance strategies in traditional technologies.

[0005] The present invention provides a port multi-vehicle collaborative scheduling optimization system based on deep reinforcement learning, including a state perception module, a multi-vehicle scheduling optimization model module, a scheduling problem solving module, an action execution module, a fault statistics module and a maintenance management module. The state perception module is respectively connected to the multi-vehicle scheduling optimization model module and the fault statistics module, the multi-vehicle scheduling optimization model module is also connected to the scheduling problem solving module, the scheduling problem solving module is also connected to the action execution module, and the fault statistics module is also connected to the maintenance management module.

[0006] The state perception module is used to integrate IoT sensors to obtain the location, status, and task progress data of multiple vehicles; wherein the multiple vehicles include quay cranes, container trucks, and reach stackers;

[0007] The multi-vehicle scheduling optimization model module is used to construct a collaborative scheduling optimization problem model for quay cranes, container trucks, and reach stackers, including objective functions and constraints;

[0008] The scheduling problem solving module is used to solve the collaborative scheduling optimization problem model to generate a collaborative scheduling strategy with minimum energy consumption and shortest time;

[0009] The action execution module is used to convert the collaborative scheduling strategy into actual port operation instructions and control multiple vehicles to perform specific operations;

[0010] The fault statistics module is used to count the number of vehicle failures and operating time, and calculate the vehicle failure rate based on the number of failures and operating time to formulate a vehicle maintenance strategy, and transmit the vehicle failure situation to the status perception module and maintenance management module;

[0011] The maintenance management module is used to record the maintenance of the faulty vehicle according to the vehicle fault condition and calculate the maintenance cost rate to perform preventive maintenance management on multiple vehicles.

[0012] Furthermore, the objective function of the multi-vehicle scheduling optimization model module includes:

[0013] Basic assumptions: The port's quay cranes, container trucks, and reach stackers are of the same model and performance, and container unloading operations within the yard are not considered. During the planning period, only container unloading operations are considered. The container trucks' movement paths within the yard are not considered, and only the transportation of container trucks between the quay cranes and reach stackers is considered.

[0014] The objective function Z1 of minimizing the total operation time of the container unloading on the quay crane, container truck and reach stacker during the statistical period is set as:

[0015]

[0016] The objective function Z2 for minimizing the total energy consumption of the quay crane, container truck, and reach stacker for unloading containers during the statistical period is set as:

[0017]

[0018] Among them, quay cranes use electric engines; container trucks and reach stackers all use diesel engines; N is the container task set, i, i′∈N; Q is the quay crane set, q∈Q; V is the container truck set, v∈V; R is the reach stacker set, r∈R; p1, p2, p3 are the operating efficiencies of quay cranes, container trucks, and reach stackers; μ1, μ2, μ3 are the unit energy consumption indicators of quay cranes, container trucks, and reach stackers; is the lifting height and horizontal travel distance of the quay crane; l v is the average one-way travel distance of the container truck; lr The average one-way travel distance of the reach stacker; The lifting and horizontal operation speed of the quay crane; The speed of container trucks with or without load; is the travel speed of the reach stacker with full load and empty load; α1 and α2 are the conversion coefficients of electricity and diesel energy consumption; ts i,q is the start time of task box i of quay crane q; ts i,v is the start time of the task box i of the container truck v; ts i,r is the start time of task box i of reach stacker r.

[0019] Furthermore, the constraints of the multi-vehicle scheduling optimization model module include:

[0020] The equipment uniqueness constraint means that each task box can only be operated once on a quay crane, a container truck, or a reach stacker:

[0021]

[0022] The continuity constraint means that when the same quay crane, container truck, or reach stacker operates on a task box, there is only one subsequent task after any task box is operated:

[0023]

[0024] Container operation sequence constraint, that is, each container must be handled in the order of quay crane-container truck-reach stacker, that is, the start time of loading and unloading operations on the container truck is later than the completion time of loading and unloading operations on the quay crane; the start time of loading and unloading operations on the reach stacker is later than the completion time of loading and unloading operations on the container truck:

[0025]

[0026] The operating capacity constraint means that the operating time of the equipment at each stage should be less than or equal to the effective operating time of the equipment:

[0027]

[0028]

[0029] The operating time relationship between the two containers that are operated continuously on each device is that the completion time of the operation of the first container is less than the start time of the operation of the second container:

[0030]

[0031] Decision variable constraints:

[0032]

[0033] Xi,q ,X i,v ,X i,r ,X i,i′,q ,X i,i′,v ,X i,i′,r ∈{0,1}

[0034] Among them, k1, k2, k3 are the utilization rates of quay cranes, container trucks, and reach stackers; T is the statistical period;

[0035] X i,q ∈{0,1},X i,q =1 means that the quay crane q operates on the task box i, otherwise, X i,q =0;

[0036] X i,v ∈{0,1},X i,v =1 means that the container truck v performs the operation on the task box i, otherwise, X i,v =0;

[0037] X i,r ∈{0,1},X i,r =1 means that the reach stacker r is working on the task box i, otherwise, X i,r =0;

[0038] X i,i′,q ∈{0,1},X i,i′,q =1 means that the quay crane q operates on the task box i and then operates on the task box i', otherwise, X i,i′,q =0;

[0039] X i,i′,v ∈{0,1},X i,i′,v =1 means that the container truck v performs the operation on the task box i and then performs the operation on the task box i', otherwise, X i,i′,v =0;

[0040] X i,i′,r ∈{0,1},X i,i′,r =1 means that the reach stacker r operates on the task box i and then operates on the task box i', otherwise, X i,i′,r =0.

[0041] Furthermore, the scheduling problem solving module uses the NSGA-II algorithm to solve the collaborative scheduling optimization problem model, and the process is as follows:

[0042] Initialize the population, calculate the non-dominated sorting and crowding distance, select, crossover, and mutate to form the initial generation population, make the evolutionary generation gen+1, and then perform selection, crossover, and mutation again;

[0043] The parent and child populations are merged into a new population 2N, and it is determined whether a new parent population should be generated;

[0044] If a new parent population is not generated, the objective function is calculated, a fast non-dominated sort and crowding calculation is performed, and suitable individuals are selected to form the new parent generation N. The result is returned to determine whether a new parent population is generated.

[0045] If a new parent population is generated, selection, crossover, and mutation are performed to form a child population, and it is determined whether Gen is less than the maximum number of generations;

[0046] If Gen is less than the maximum number of generations, set Gen = Gen + 1, and return to the step of merging the parent and child populations into a new population 2N, and determine whether to generate a new parent population;

[0047] If Gen is greater than or equal to the maximum number of generations, the process ends.

[0048] Furthermore, the fast non-dominated sort is used to classify the solutions in the population, with a complexity of O(MN 2 ), M and N represent the target number and population size respectively; binary tournament selection is adopted, giving priority to individuals with lower orders obtained in the non-dominated sorting process, and the lower the order, the higher the probability of being selected into the next generation; when the orders of individuals are the same, the population diversity is enhanced by measuring the crowding distance, and individuals with larger crowding distance are selected; binary crossover-simulated binary crossover and polynomial mutation based on real number coding are adopted.

[0049] Furthermore, in the failure statistics module, the calculation formula for the vehicle failure rate is:

[0050]

[0051] Among them, FR is the equipment failure rate, times / thousand hours, N f is the number of failures, T op is the equipment operating time, hours; through this formula, the vehicle failure rate can be monitored and preventive measures can be taken for vehicles whose failure frequency exceeds the set value.

[0052] Furthermore, in the maintenance management module, the calculation formula for the maintenance cost rate is:

[0053] Maintenance cost rate = maintenance cost / original value of fixed assets × 100%

[0054] Maintenance costs refer to the total cost of vehicle maintenance over a specific period of time. The original value of a fixed asset includes initial costs such as purchase price, installation fees, and transportation fees. This formula can be used to assess the proportion of maintenance costs to the original value of a fixed asset, thereby determining the rationality and economic benefits of maintenance costs. By analyzing the cost ratio of maintenance costs, it is possible to identify and address issues with excessively high or low maintenance costs.

[0055] Perform preventive maintenance management on multiple vehicles and calculate the effectiveness of preventive maintenance:

[0056]

[0057] Among them, MRR is the ratio of preventive maintenance to reduce failure rate, %, R P is the equipment failure rate after implementing preventive maintenance, R f is the failure rate of equipment without preventive maintenance. This formula can be used to quantify the effectiveness of preventive maintenance and determine its impact on vehicle reliability, so as to plan the frequency and content of regular inspections.

[0058] The beneficial effects of the present invention are:

[0059] The present invention realizes the real-time integration and synchronization of the position, status and task progress data of multiple vehicles through the state perception module, provides accurate data support for scheduling decisions, and significantly improves the real-time and integrity of data collection; the collaborative scheduling model constructed by the multi-vehicle scheduling optimization model module, combined with the dual-objective optimization solution of the scheduling problem solving module, realizes the generation of collaborative scheduling strategies for minimizing energy consumption and shortening operation time, greatly improves port operation efficiency and reduces energy consumption costs; the action execution module ensures that the scheduling strategy is accurately converted into operation instructions, and guarantees the consistency and efficiency of multi-vehicle collaborative operations; the fault statistics module and the maintenance management module effectively reduce the unplanned downtime of multiple vehicles through real-time fault statistics, fault rate calculation and preventive maintenance strategy formulation, and realizes the reasonable allocation of maintenance resources in combination with maintenance cost rate analysis, which significantly reduces vehicle maintenance costs, and finally forms a port multi-vehicle collaborative operation system integrating data perception, intelligent scheduling, fault warning and preventive maintenance, which comprehensively improves the intelligence level and comprehensive benefits of port operations. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] Figure 1 This is a structural diagram of the port multi-vehicle collaborative scheduling optimization system based on deep reinforcement learning in the present invention.

[0061] The purpose, features and advantages of the present invention will be further described with reference to the accompanying drawings and in conjunction with the embodiments. DETAILED DESCRIPTION

[0062] It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0063] like Figure 1As shown, the present invention provides a port multi-vehicle collaborative scheduling optimization system based on deep reinforcement learning, including a state perception module, a multi-vehicle scheduling optimization model module, a scheduling problem solving module, an action execution module, a fault statistics module and a maintenance management module. The state perception module is respectively connected to the multi-vehicle scheduling optimization model module and the fault statistics module, the multi-vehicle scheduling optimization model module is also connected to the scheduling problem solving module, the scheduling problem solving module is also connected to the action execution module, and the fault statistics module is also connected to the maintenance management module.

[0064] (1) State perception module

[0065] The state perception module integrates IoT sensors (RFID, GPS, LiDAR, etc.) to acquire data on the location, status (operating / idle), and task progress of quay cranes, container trucks, reach stackers, and other vehicles. It also uses the Port Management Information System (PMS) to acquire static and dynamic data such as ship arrival plans, container loading and unloading requirements, and yard distribution.

[0066] Vehicle status perception:

[0067] Quay crane / reach stacker: The encoder collects the lifting height, trolley position, and trolley travel coordinates, the current sensor monitors the motor load, and the RFID identifies the container ID.

[0068] Trucks / AGVs: Integrated GPS (positioning accuracy 1-3 meters) + Inertial Measurement Unit (IMU) fusion positioning, LiDAR (Lidar) to scan surrounding obstacles, and millimeter-wave radar to detect vehicle distance.

[0069] Data frequency: The vehicle position data collection frequency is ≥10Hz, and the equipment status data (such as fault signals) adopts an event trigger mechanism (millisecond-level response).

[0070] Yard status: The number of container stacking layers in the yard is identified through cameras and computer vision (YOLO algorithm), and RFID readers are used to obtain container location information in batches.

[0071] Berth status: A laser rangefinder is used to monitor the docking status of ships at the berth, and a tide sensor is used to collect water level data;

[0072] Connect to the port ERP system to obtain the vessel arrival schedule (ETA) and container loading and unloading list (B / L); access the equipment management system (EMS) to obtain vehicle maintenance plans, maintenance cycles and other constraints.

[0073] (2) Multi-vehicle scheduling optimization model module

[0074] The multi-vehicle scheduling optimization model module is used to construct a collaborative scheduling optimization problem model for quay cranes, container trucks, and reach stackers, including objective functions and constraints.

[0075] Transportation and loading and unloading operations mainly include three parts: at the front of the wharf, ships occupy quay crane resources for loading and unloading; between the front of the wharf and the yard, container trucks are responsible for horizontal transportation operations; at the yard and between the yard and the loading and unloading line, front loaders are used for loading and unloading and horizontal transportation operations.

[0076] The loading and unloading efficiency of quay cranes is closely related to the performance of lifting capacity, lifting height, inner and outer reach, track gauge, as well as the number of idle quay cranes and the number of containers. As an intermediate operation link, container trucks affect the operating efficiency of quay cranes and reach stackers. Their operating efficiency is closely related to factors such as the number of idle container trucks, the operating efficiency of quay cranes and reach stackers, and the horizontal transportation distance. Container reach stackers are mobile machines that can realize loading and unloading, stacking and horizontal transportation. Their operating efficiency is related to factors such as lifting capacity, stacking height, working range and speed. Reasonable loading and unloading sequence is the core issue that needs to be solved in port loading and unloading scheduling. An unreasonable loading and unloading sequence will increase the waiting time between equipment, thereby increasing loading and unloading time and energy consumption. Therefore, the loading and unloading operation sequence should be arranged reasonably to reduce operation time and reduce operation energy consumption.

[0077] Based on the hybrid flow shop scheduling model, the task allocation plan and operation sequence of ship loading and unloading equipment, container horizontal transportation equipment and loading and unloading line equipment were determined, namely: the workpiece to be processed is the container that needs to be operated; the three workshops are the front-end operation area of ​​the terminal and the loading and unloading line between the front and the yard, and the workshop equipment is the same type of quay crane, container truck and reach stacker; the operation sequence of each container is the same, and they all pass through the quay crane, container truck and reach stacker in turn for operation.

[0078] Considering that container port loading and unloading operations are affected by many factors, such as the performance of port loading and unloading equipment, the stacking position in the yard, and the distance traveled by container trucks, as well as many uncertainties, in order to make the model easier to solve, the following basic assumptions are made to simplify the model:

[0079] a. The port gantry cranes, container trucks and reach stackers are of the same model and have the same performance;

[0080] b. The unloading operation in the yard is not considered;

[0081] c. During the planning period, only container unloading operations are considered;

[0082] d. The movement path of the container truck in the yard is not considered, only the transportation of the container truck between the quay crane and the reach stacker is considered.

[0083] The objective function includes:

[0084] The objective function Z1 of minimizing the total operation time of the container unloading on the quay crane, container truck and reach stacker during the statistical period is set as:

[0085]

[0086] The objective function Z2 for minimizing the total energy consumption of the quay crane, container truck, and reach stacker for unloading containers during the statistical period is set as:

[0087]

[0088] Among them, quay cranes use electric engines; container trucks and reach stackers all use diesel engines; N is the container task set, i, i′∈N; Q is the quay crane set, q∈Q; V is the container truck set, v∈V; R is the reach stacker set, r∈R; p1, p2, p3 are the operating efficiencies of quay cranes, container trucks, and reach stackers; μ1, μ2, μ3 are the unit energy consumption indicators of quay cranes, container trucks, and reach stackers; is the lifting height and horizontal travel distance of the quay crane; l v is the average one-way travel distance of the container truck; l r The average one-way travel distance of the reach stacker; The lifting and horizontal operation speed of the quay crane; The speed of container trucks with or without load; is the travel speed of the reach stacker with full load and empty load; α1 and α2 are the conversion coefficients of electricity and diesel energy consumption; ts i,q is the start time of task box i of quay crane q; ts i,v is the start time of the task box i of the container truck v; ts i,r is the start time of task box i of reach stacker r.

[0089] Constraints include:

[0090] The equipment uniqueness constraint means that each task box can only be operated once on a quay crane, a container truck, or a reach stacker:

[0091]

[0092] The continuity constraint means that when the same quay crane, container truck, or reach stacker operates on a task box, there is only one subsequent task after any task box is operated:

[0093]

[0094]

[0095] Container operation sequence constraint, that is, each container must be handled in the order of quay crane-container truck-reach stacker, that is, the start time of loading and unloading operations on the container truck is later than the completion time of loading and unloading operations on the quay crane; the start time of loading and unloading operations on the reach stacker is later than the completion time of loading and unloading operations on the container truck:

[0096]

[0097] The operating capacity constraint means that the operating time of the equipment at each stage should be less than or equal to the effective operating time of the equipment:

[0098]

[0099] The operating time relationship between the two containers that are operated continuously on each device is that the completion time of the operation of the first container is less than the start time of the operation of the second container:

[0100]

[0101] Decision variable constraints:

[0102]

[0103] X i,q ,X i,v ,X i,r ,X i,i′,q ,X i,i′,v ,X i,i′,r ∈{0,1}

[0104] Among them, k1, k2, k3 are the utilization rates of quay cranes, container trucks, and reach stackers; T is the statistical period;

[0105] X i,q ∈{0,1},X i,q =1 means that the quay crane q operates on the task box i, otherwise, X i,q =0;

[0106] X i,v ∈{0,1},X i,v =1 means that the container truck v performs the operation on the task box i, otherwise, X i,v =0;

[0107] X i,r ∈{0,1},X i,r =1 means that the reach stacker r is working on the task box i, otherwise, X i,r =0;

[0108] X i,i′,q ∈{0,1},X i,i′,q =1 means that the quay crane q operates on the task box i and then operates on the task box i', otherwise, Xi,i′,q =0;

[0109] X i,i′,v ∈{0,1},X i,i′,v =1 means that the container truck v performs the operation on the task box i and then performs the operation on the task box i', otherwise, X i,i′,v =0;

[0110] X i,i′,r ∈{0,1},X i,i′,r =1 means that the reach stacker r operates on the task box i and then operates on the task box i', otherwise, X i,i′,r =0.

[0111] (3) Scheduling problem solving module

[0112] The scheduling problem solving module is used to solve the collaborative scheduling optimization problem model to generate a collaborative scheduling strategy with the lowest energy consumption and the shortest time.

[0113] The collaborative scheduling optimization problem model is a dual-objective optimization model. When solving it, multiple objectives are usually converted to a single objective, but this is highly subjective and produces a single result. Therefore, the NSGA-II multi-objective optimization algorithm is used. NSGA-II uses an elite retention strategy to facilitate the retention and inheritance of outstanding individuals in the population. By calculating the congestion degree, the solution is distributed as evenly as possible, maintaining a high level of population diversity. The algorithm process is as follows:

[0114] 1) Initialize the population, calculate the non-dominated sorting and crowding distance, select, crossover, and mutate to form the initial generation population, make the evolutionary generation gen+1, and then perform selection, crossover, and mutation again;

[0115] 2) The parent and offspring populations are merged into a new population 2N, and it is determined whether a new parent population is generated;

[0116] 3) If a new parent population is not generated, calculate the objective function, perform fast non-dominated sorting and crowding calculation, select suitable individuals to form the new parent generation N, and return to determine whether a new parent population is generated;

[0117] 4) If a new parent population is generated, selection, crossover, and mutation are performed to form a child population, and it is determined whether Gen is less than the maximum number of generations;

[0118] 5) If Gen is less than the maximum number of generations, set Gen = Gen + 1, and return to the step of merging the parent and child populations into a new population 2N, and determining whether to generate a new parent population;

[0119] 6) If Gen is greater than or equal to the maximum number of generations, the process ends.

[0120] In addition, the present invention adopts a real number encoding method based on the process to express the order relationship between the processes. The fast non-dominated sort is used to classify the solutions in the population, and the complexity is O(MN 2 ), M and N represent the target number and population size respectively; binary tournament selection is adopted, giving priority to individuals with lower orders obtained in the non-dominated sorting process, and the lower the order, the higher the probability of being selected into the next generation; when the orders of individuals are the same, the population diversity is enhanced by measuring the crowding distance, and individuals with larger crowding distance are selected; binary crossover-simulated binary crossover and polynomial mutation based on real number coding are adopted.

[0121] (4) Action execution module

[0122] The action execution module is used to convert the collaborative scheduling strategy into actual port operation instructions and control multiple vehicles to perform specific operations.

[0123] Develop an interface adaptation layer to connect to the port equipment control system (such as the quay crane PLC system and the container truck dispatch terminal), and design action mapping rules: for example, convert the decision of "assigning AGV to quay crane A" into a specific equipment control instruction and add safety constraint checks. That is,

[0124] a. Strategy-action mapping layer: Design a domain-specific language (DSL) to describe scheduling decisions, and use a rule engine to convert the DSL into a vehicle control protocol (such as ROS messages for AGVs and PLC instructions for quay cranes).

[0125] b. Equipment control interface layer: quay cranes / reach stackers: Connect to the PLC system via the OPC UA protocol to send instructions such as gantry travel coordinates, trolley positioning, and spreader movement; container trucks / AGVs: Receive path point sequences (WGS84 coordinates) through the onboard terminal and execute speed planning in conjunction with the onboard controller (MCU); Protocol adaptation: Supports conversion between multiple protocols such as Modbus, CANopen, and MQTT, and is compatible with both new and existing equipment.

[0126] c. Optimize execution instructions: Batch processing of similar actions: Merge the paths of multiple trucks heading to the same yard to reduce the number of path planning times; Add forward-looking control: Send the next action instruction 500ms in advance to avoid waiting for the vehicle.

[0127] (5) Fault statistics module

[0128] The fault statistics module is used to count the number of failures and operating time of multiple vehicles, and calculate the vehicle failure rate based on the number of failures and operating time to formulate a vehicle maintenance strategy, and transmit the vehicle failure situation to the status perception module and maintenance management module.

[0129] Common failure types experienced by port container handling equipment during long, intensive operations include mechanical, electrical, hydraulic, and control system failures. Mechanical failures are the most common, typically manifesting as component wear, wire rope breakage, and pulley failure. These problems are primarily caused by long equipment operating cycles, excessive loads, or insufficient lubrication. Electrical failures frequently occur in control circuits, motors, and other areas, often due to current overload or equipment aging. Hydraulic system failures often manifest as failure of hydraulic pumps or valves, typically caused by abnormal pressure or hydraulic oil contamination. Control system failures include sensor failures and software malfunctions, severely impacting the equipment's automated operation and real-time monitoring capabilities.

[0130] The calculation formula for vehicle failure rate is:

[0131]

[0132] Among them, FR is the equipment failure rate, times / thousand hours, N f is the number of failures, T op is the equipment operating time, in hours. This formula allows us to monitor vehicle failure rates and take preventive measures if the failure frequency exceeds a set value. By analyzing common failure types and frequencies, we can effectively formulate maintenance strategies, reduce equipment failure rates, and ensure the stability and efficiency of port operations.

[0133] (6) Maintenance management module

[0134] The maintenance management module is used to record the maintenance of the faulty vehicle according to the vehicle fault condition and calculate the maintenance cost rate to perform preventive maintenance management on multiple vehicles.

[0135] Maintaining port container handling equipment faces numerous challenges, primarily due to equipment complexity, harsh operating environments, high maintenance costs, and insufficient personnel skills. Equipment complexity is a major obstacle to port maintenance, particularly for automated equipment, which requires proficiency in complex control systems and sensor technology. The harsh natural environment also exacerbates equipment corrosion and wear, leading to increased maintenance frequency. Furthermore, the high maintenance costs of port equipment are a major source of financial pressure, particularly due to the labor costs associated with expensive parts replacement and emergency repairs. Finally, insufficient technical skills, particularly inadequate operational proficiency and troubleshooting capabilities for new intelligent equipment, further complicate equipment maintenance.

[0136] The calculation formula for the maintenance cost rate is:

[0137] Maintenance cost rate = maintenance cost / original value of fixed assets × 100%

[0138] Maintenance costs refer to the total cost of maintaining a vehicle over a specific period of time. The original value of a fixed asset includes initial costs such as purchase price, installation fees, and transportation fees. This formula can be used to evaluate the proportion of maintenance costs in the original value of fixed assets, thereby determining the rationality and economic benefits of maintenance costs. By analyzing the cost rate of maintenance costs, it is possible to identify and address issues such as excessive or insufficient maintenance costs. Through this analysis, port managers can develop more effective maintenance strategies for different equipment and maintenance challenges and improve the reliability and economic efficiency of equipment operations by optimizing maintenance costs.

[0139] Preventive maintenance and regular inspections are crucial components of port container handling equipment maintenance. Regular equipment inspections and maintenance prevent unexpected failures, thereby extending equipment life and reducing operating costs. The core of preventive maintenance lies in continuously monitoring equipment operating data to proactively identify potential problems and implement necessary repairs or replacements. Regular inspections, based on frequency of equipment use and operating environment, comprehensively inspect and maintain key components such as cables, hydraulic systems, and electrical control units.

[0140] Perform preventive maintenance management on multiple vehicles and calculate the effectiveness of preventive maintenance:

[0141]

[0142] Among them, MRR is the ratio of preventive maintenance to reduce failure rate, %, R P is the equipment failure rate after implementing preventive maintenance, R f =( ...

[0143] The present invention realizes the real-time integration and synchronization of the position, status and task progress data of multiple vehicles through the state perception module, provides accurate data support for scheduling decisions, and significantly improves the real-time and integrity of data collection; the collaborative scheduling model constructed by the multi-vehicle scheduling optimization model module, combined with the dual-objective optimization solution of the scheduling problem solving module, realizes the generation of collaborative scheduling strategies for minimizing energy consumption and shortening operation time, greatly improves port operation efficiency and reduces energy consumption costs; the action execution module ensures that the scheduling strategy is accurately converted into operation instructions, and guarantees the consistency and efficiency of multi-vehicle collaborative operations; the fault statistics module and the maintenance management module effectively reduce the unplanned downtime of multiple vehicles through real-time fault statistics, fault rate calculation and preventive maintenance strategy formulation, and realizes the reasonable allocation of maintenance resources in combination with maintenance cost rate analysis, which significantly reduces vehicle maintenance costs, and finally forms a port multi-vehicle collaborative operation system integrating data perception, intelligent scheduling, fault warning and preventive maintenance, which comprehensively improves the intelligence level and comprehensive benefits of port operations.

[0144] It should be noted that, in this document, the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, apparatus, article, or method comprising a series of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, apparatus, article, or method. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, apparatus, article, or method comprising the element.

[0145] The above description is only a preferred embodiment of the present invention and does not limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made by using the contents of the present invention description and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A port multi-vehicle collaborative scheduling optimization system based on deep reinforcement learning, characterized by: It includes a state perception module, a multi-vehicle scheduling optimization model module, a scheduling problem solving module, an action execution module, a fault statistics module and a maintenance management module. The state perception module is connected to the multi-vehicle scheduling optimization model module and the fault statistics module respectively. The multi-vehicle scheduling optimization model module is also connected to the scheduling problem solving module, the scheduling problem solving module is also connected to the action execution module, and the fault statistics module is also connected to the maintenance management module. The state perception module is used to integrate IoT sensors to obtain the location, status, and task progress data of multiple vehicles; wherein the multiple vehicles include quay cranes, container trucks, and reach stackers; The multi-vehicle scheduling optimization model module is used to construct a collaborative scheduling optimization problem model for quay cranes, container trucks, and reach stackers, including objective functions and constraints; The scheduling problem solving module is used to solve the collaborative scheduling optimization problem model to generate a collaborative scheduling strategy with minimum energy consumption and shortest time; The action execution module is used to convert the collaborative scheduling strategy into actual port operation instructions and control multiple vehicles to perform specific operations; The fault statistics module is used to count the number of vehicle failures and operating time, and calculate the vehicle failure rate based on the number of failures and operating time to formulate a vehicle maintenance strategy, and transmit the vehicle failure situation to the status perception module and maintenance management module; The maintenance management module is used to record the maintenance of the faulty vehicle according to the vehicle fault condition and calculate the maintenance cost rate to perform preventive maintenance management on multiple vehicles.

2. The port multi-vehicle collaborative scheduling optimization system based on deep reinforcement learning according to claim 1 is characterized in that: The objective function of the multi-vehicle scheduling optimization model module includes: Basic assumptions: The port's quay cranes, container trucks, and reach stackers are of the same model and performance, and container unloading operations within the yard are not considered. During the planning period, only container unloading operations are considered. The container trucks' movement paths within the yard are not considered, and only the transportation of container trucks between the quay cranes and reach stackers is considered. The objective function Z1 of minimizing the total operation time of the container unloading on the quay crane, container truck and reach stacker during the statistical period is set as: The objective function Z2 for minimizing the total energy consumption of the quay crane, container truck, and reach stacker for unloading containers during the statistical period is set as: Among them, quay cranes use electric engines; container trucks and reach stackers all use diesel engines; N is the container task set, i, i′∈N; Q is the quay crane set, q∈Q; V is the container truck set, v∈V; R is the reach stacker set, r∈R; p1, p2, p3 are the operating efficiencies of quay cranes, container trucks, and reach stackers; μ1, μ2, μ3 are the unit energy consumption indicators of quay cranes, container trucks, and reach stackers; is the lifting height and horizontal travel distance of the quay crane; l v is the average one-way travel distance of the container truck; l r The average one-way travel distance of the reach stacker; The lifting and horizontal operation speed of the quay crane; The speed of container trucks with or without load; is the travel speed of the reach stacker with full load and empty load; α1 and α2 are the conversion coefficients of electricity and diesel energy consumption; ts i,q is the start time of task box i of quay crane q; ts i,v is the start time of the task box i of the container truck v; ts i,r is the start time of task box i of reach stacker r.

3. The port multi-vehicle collaborative scheduling optimization system based on deep reinforcement learning according to claim 2 is characterized in that: The constraints of the multi-vehicle scheduling optimization model module include: The equipment uniqueness constraint means that each task box can only be operated once on a quay crane, a container truck, or a reach stacker: The continuity constraint means that when the same quay crane, container truck, or reach stacker operates on a task box, there is only one subsequent task after any task box is operated: Container operation sequence constraint, that is, each container must be handled in the order of quay crane-container truck-reach stacker, that is, the start time of loading and unloading operations on the container truck is later than the completion time of loading and unloading operations on the quay crane; the start time of loading and unloading operations on the reach stacker is later than the completion time of loading and unloading operations on the container truck: The operating capacity constraint means that the operating time of the equipment at each stage should be less than or equal to the effective operating time of the equipment: The operating time relationship between the two containers that are operated continuously on each device is that the completion time of the operation of the first container is less than the start time of the operation of the second container: Decision variable constraints: X i,q ,X i,v ,X i,r ,X i,i′,q ,X i,i′,v ,X i,i′,r ∈{0,1} Among them, k1, k2, k3 are the utilization rates of quay cranes, container trucks, and reach stackers; T is the statistical period; X i,q ∈{0,1},X i,q =1 means that the quay crane q operates on the task box i, otherwise, X i,q =0; X i,v ∈{0,1},X i,v =1 means that the container truck v performs the operation on the task box i, otherwise, X i,v =0; X i,r ∈{0,1},X i,r =1 means that the reach stacker r is working on the task box i, otherwise, X i,r =0; X i,i′,q ∈{0,1},X i,i′,q =1 means that the quay crane q operates on the task box i and then operates on the task box i', otherwise, X i,i′,q =0; X i,i′,v ∈{0,1},X i,i′,v =1 means that the container truck v performs the operation on the task box i and then performs the operation on the task box i', otherwise, x i,i′,v =0; X i,i′,r ∈{0,1},X i,i′,r =1 means that the reach stacker r operates on the task box i and then operates on the task box i', otherwise, X i,i′,r =0.

4. The port multi-vehicle collaborative scheduling optimization system based on deep reinforcement learning according to claim 1 is characterized in that: The scheduling problem solving module uses the NSGA-II algorithm to solve the collaborative scheduling optimization problem model, and its process is as follows: Initialize the population, calculate the non-dominated sorting and crowding distance, select, crossover, and mutate to form the initial generation population, make the evolutionary generation gen+1, and then perform selection, crossover, and mutation again; The parent and child populations are merged into a new population 2N, and it is determined whether a new parent population should be generated; If a new parent population is not generated, the objective function is calculated, a fast non-dominated sort and crowding calculation is performed, and suitable individuals are selected to form the new parent generation N. The result is returned to determine whether a new parent population is generated. If a new parent population is generated, selection, crossover, and mutation are performed to form a child population, and it is determined whether Gen is less than the maximum number of generations; If Gen is less than the maximum number of generations, set Gen = Gen + 1, and return to the step of merging the parent and child populations into a new population 2N, and determine whether to generate a new parent population; If Gen is greater than or equal to the maximum number of generations, the process ends.

5. The port multi-vehicle collaborative scheduling optimization system based on deep reinforcement learning according to claim 4 is characterized in that: The solutions in the population are classified using fast non-dominated sorting, with a complexity of O(MN 2 ), M and N represent the target number and population size respectively; binary tournament selection is adopted, giving priority to individuals with lower orders obtained in the non-dominated sorting process, and the lower the order, the higher the probability of being selected into the next generation; when the orders of individuals are the same, the population diversity is enhanced by measuring the crowding distance, and individuals with larger crowding distance are selected; binary crossover-simulated binary crossover and polynomial mutation based on real number coding are adopted.

6. The port multi-vehicle collaborative scheduling optimization system based on deep reinforcement learning according to claim 1 is characterized in that: In the fault statistics module, the calculation formula for the vehicle failure rate is: Among them, FR is the equipment failure rate, times / thousand hours, N f is the number of failures, T op is the equipment operating time, hours; through this formula, the vehicle failure rate can be monitored and preventive measures can be taken for vehicles whose failure frequency exceeds the set value.

7. The port multi-vehicle collaborative scheduling optimization system based on deep reinforcement learning according to claim 1 is characterized in that: In the maintenance management module, the calculation formula for the maintenance cost rate is: Maintenance cost rate = maintenance cost / original value of fixed assets × 100% Maintenance costs refer to the total cost of maintaining a vehicle over a specific period of time. The original value of a fixed asset includes initial costs such as purchase price, installation fees, and transportation fees. This formula can be used to evaluate the proportion of maintenance costs in the original value of fixed assets, thereby judging the rationality and economic benefits of maintenance costs. By analyzing the cost rate of maintenance costs, it is possible to discover and solve the problem of excessively high or low maintenance costs. Perform preventive maintenance management on multiple vehicles and calculate the effectiveness of preventive maintenance: Among them, MRR is the ratio of preventive maintenance to reduce failure rate, %, R P is the equipment failure rate after implementing preventive maintenance, R f is the failure rate of equipment without preventive maintenance. This formula can be used to quantify the effectiveness of preventive maintenance and determine its impact on vehicle reliability, so as to plan the frequency and content of regular inspections.

Images