Reinforcement learning driven heuristic solving method for multi-ship co-berthing maintenance scheduling

Abstract

The application discloses a reinforcement learning driven heuristic solving method for multi-ship co-dock maintenance scheduling, and three global search operators are designed for the multi-ship co-dock maintenance scheduling problem. One is a ship crossover operator, which is used to adjust and exchange the maintenance order of the ship. The second is a maintenance task destruction operator, which breaks the original task structure by extracting part of the task from the current maintenance task sequence, providing a larger optimization space for subsequent search. The third is a maintenance task recombination operator, which inserts the extracted maintenance task into the existing maintenance task list to form a new maintenance task order, thereby improving the possibility of obtaining a better scheduling scheme. The execution process of the global search operator is adaptively guided, and a more suitable search operation is dynamically selected according to the state information in the solving process, so as to reduce the invalid search overhead in the meta-heuristic algorithm, improve the utilization efficiency of the computing resources, and further improve the solving efficiency and solving effect of the multi-ship co-dock maintenance scheduling problem.

Description

Technical Field

[0001] This invention relates to the field of ship maintenance scheduling technology, and specifically to a reinforcement learning-driven heuristic solution method for multi-ship dock maintenance scheduling. Background Technology

[0002] Ship maintenance task scheduling typically targets the maintenance service process of shipbuilding companies or maintenance service units. Its core lies in the rational allocation and sequential arrangement of multiple maintenance tasks based on factors such as execution time, task location, personnel allocation, and transportation conditions. This aims to shorten the overall maintenance service span, improve service efficiency, and reduce service costs. Compared to general production scheduling problems, this type of problem, in addition to considering task allocation and sequencing, also requires taking into account the round-trip transportation factors between the port where the maintenance task is located, the spare parts center, and the task location, as well as the deteriorating characteristics of the maintenance task itself due to changes in waiting time.

[0003] In ship maintenance scenarios, maintenance tasks typically exhibit both distributed and deteriorating characteristics. On the one hand, ships often anchor in different ports awaiting maintenance, and these ports are geographically dispersed, requiring maintenance personnel to travel between these ports and spare parts centers to complete the work. On the other hand, when the waiting time for ship maintenance is excessively long, the required time for the corresponding maintenance task will increase, meaning the maintenance task has a deteriorating effect. Due to these characteristics, traditional workshop scheduling algorithms often struggle to provide effective and reasonable scheduling schemes suitable for ship maintenance scenarios, thus hindering the improvement of shipbuilding companies' maintenance service capabilities and market competitiveness. Chinese patent "CN 114936713 A Scheduling Optimization Method for Ship Maintenance Task Scheduling Problem" provides an improved variable neighborhood search algorithm. This method, based on the characteristics of the distributed ship maintenance scheduling problem, designs a locally accurate algorithm and various novel neighborhood structures. It overcomes the shortcomings of classic variable neighborhood search algorithms, such as slow convergence speed and insufficient convergence, further improving the algorithm's solution efficiency and thus achieving efficient solutions to the distributed ship maintenance scheduling problem.

[0004] The variable neighborhood search algorithm proposed in Chinese patent "CN 114936713 A Scheduling Optimization Method for Ship Maintenance Task Scheduling Problem" is an effective intelligent optimization method and has been applied to various scheduling problems. However, its basic neighborhood design is relatively simple, and it is prone to getting trapped in local optima when solving large-scale complex scheduling problems, making it difficult to balance solution efficiency and solution quality. For ship maintenance scheduling problems with distributed tasks, degradation effects, and multi-maintenance personnel collaboration, directly using existing variable neighborhood search frameworks and conventional neighborhood structures often fails to obtain reliable approximate optimal solutions within a reasonable time.

[0005] Chinese patent "CN 116468221 A Method and System for Scheduling Ship Power Equipment Maintenance Tasks" provides a method and system for scheduling ship power equipment maintenance tasks. This patent first constructs a multi-skill, multi-project, and multi-mode resource-constrained project scheduling model, considering the multi-skill characteristics of maintenance personnel, the limited total number of maintenance personnel, and the fatigue recovery effect of maintenance personnel. Then, an improved immune algorithm is used to solve the model to obtain a ship power equipment maintenance task scheduling scheme. Specifically, the improved immune algorithm replaces the single mutation operation in the traditional immune algorithm with crossover and mutation operations, and adds a local search operation after the mutation operation and before randomly removing half of the solutions, thereby improving the solution efficiency and solution quality.

[0006] While the Chinese patent "CN 116468221 A Method and System for Scheduling Ship Power Equipment Maintenance Tasks" considers multiple ship maintenance projects, it does not further address the issue of dock resource constraints in multi-ship dock maintenance scenarios. In reality, during ship maintenance, multiple ships often need to share limited dock resources. However, dock resources are characterized by limited capacity, long occupancy periods, and the inability to be reused simultaneously, leading to significant resource competition between different ships.

[0007] Scheduling ship power equipment maintenance tasks typically requires generating a maintenance scheduling plan that meets the project timeline requirements based on the task sequence, personnel allocation, and resource constraints of multiple ship maintenance projects. This invention employs a heuristic algorithm for employee allocation or an intelligent optimization algorithm to solve this type of problem. Specifically, it first sorts the tasks according to their sequential relationships, then allocates personnel based on the available maintenance staff, and finally generates a scheduling plan with the goal of minimizing the maximum completion time of multiple projects. Summary of the Invention

[0008] In view of this, in order to address the problems of existing technologies not fully considering the competition for dock resources in multi-ship dock maintenance scenarios, resulting in scheduling schemes that fail to accurately reflect the actual maintenance process, and existing algorithms having insufficient solution efficiency and quality in complex scheduling problems, this invention proposes a reinforcement learning-driven heuristic solution method for multi-ship dock maintenance scheduling. By incorporating the characteristics of multi-ship dock maintenance into the scheduling optimization process and adopting a reinforcement learning-driven heuristic solution mechanism, this method improves the rationality of resource allocation, overall maintenance efficiency, and solution performance for complex scheduling problems in multi-ship dock maintenance scenarios.

[0009] The present invention solves the above problems through the following technical means: In a first aspect, the present invention provides a reinforcement learning-driven heuristic solution method for multi-ship dock maintenance scheduling, comprising the following steps: Step 1: Collect ship maintenance task data; Step 2: Design a three-layer coding system for ship maintenance scheduling. The first layer includes a ship maintenance sequence list; the second layer includes a ship maintenance task sequence list; and the third layer includes a list of ship maintenance task support modes. Step 3: Design a dedicated ship maintenance decoding strategy and run the dedicated ship maintenance decoding strategy to obtain an initial maintenance scheduling plan; Step 4: Design three global search operators for optimizing ship maintenance scheduling: the first is the ship crossover operator, the second is the ship task destruction operator, and the third is the ship maintenance task reorganization operator. Step 5: Design reinforcement learning strategies to adaptively guide the global search operator search and balance the exploration and search phases; Step 6: Under the guidance of reinforcement learning strategy, apply the three global search operators appropriately and record the optimized multi-ship dock maintenance scheduling scheme; Step 7: Determine if the algorithm has reached its predetermined termination point. If so, output the scheduling scheme; otherwise, return to step 4 and re-optimize. Step 8: Determine whether the scheduling plan meets the manager's expectations. If it does, the optimization ends; if not, continue optimizing.

[0010] Preferably, in step 1, the ship repair task data includes the number of ships to be repaired, the number of ship repair tasks, the sequence of ship repair tasks, and the type and quantity of resources required for ship repair tasks.

[0011] Preferably, in step 3, the dedicated ship maintenance decoding strategy is described in detail below: 3.1: Calculate the total time domain upper limit of scheduling , ;in Indicates the ship repair task number, set Includes all maintenance tasks. Indicates task The time required to perform maintenance tasks; 3.2: Initialize the task start time, end time, ship docking time, docking time, and the status of various resource occupancy. 3.3: Dispatch each ship in sequence according to the given ship maintenance order; 3.4: Select the earliest available dock berth for the current vessel; 3.5: Determine the earliest docking time for the current vessel; 3.6: Based on the task location attribute, divide the current ship's maintenance tasks into dock-in-dock tasks and dock-out-of-dock tasks; 3.7: Sequentially schedule the dockside tasks of the current vessel; 3.8: Traverse the currently scheduled dockside tasks; 3.9: Select all schedulable tasks whose prerequisite tasks have been completed to form a candidate task set; 3.10: Calculate the earliest start time of the candidate task, taking into account the completion time of the preceding task and the ship's docking time; 3.11: Select an executable maintenance mode for the current task; 3.12: Determine if the resources meet the task execution conditions; 3.13: If resources are sufficient, the task will be scheduled for execution at the earliest start time; 3.14: If resources are insufficient, delay the task start time until resources are sufficient; 3.15: Update resource usage status and add scheduled tasks to the completed task set; 3.16: Update the current docking and undocking time intervals of the vessels based on the task scheduling results; 3.17: Update the available time of dock berths; 3.18: Schedule the current vessels' off-dock tasks sequentially; 3.19: Select all schedulable off-dock tasks whose prerequisite tasks have been completed to form a candidate task set; 3.20: Calculate the earliest start time of candidate off-docking tasks, taking into account the completion time of prerequisite tasks and ship undocking constraints; 3.21: Select an executable maintenance mode for the current off-dock mission; 3.22: Under the condition of satisfying resource constraints, schedule the task to be executed at the earliest feasible time; 3.23: Update resource utilization status and task scheduling status; 3.24: Calculate the current completion time of the vessel; 3.25: Calculate the average completion time of all ships to obtain the final scheduling result.

[0012] Secondly, the present invention provides a reinforcement learning-driven heuristic solution system for multi-ship dock maintenance scheduling, comprising: The maintenance task data collection module is used to collect ship maintenance task data; The maintenance scheduling coding design module is used to design a three-layer coding method for ship maintenance scheduling. The first layer includes a ship maintenance sequence list; the second layer includes a ship maintenance task sequence list; and the third layer includes a ship maintenance task support mode list. The dedicated maintenance decoding strategy design module is used to design dedicated ship maintenance decoding strategies and run the dedicated ship maintenance decoding strategies to obtain an initial maintenance scheduling plan; The global search operator optimization module is used to design three global search operators for optimizing ship maintenance scheduling: the first is the ship crossover operator, the second is the ship task destruction operator, and the third is the ship maintenance task reorganization operator. The reinforcement learning strategy design module is used to design reinforcement learning strategies that adaptively guide the global search operator search and balance the exploration and search phases. The maintenance scheduling plan recording module is used to rationally apply three global search operators under the guidance of reinforcement learning strategies and record the optimized multi-ship dock maintenance scheduling plan. The algorithm's predetermined termination judgment module is used to determine whether the algorithm's predetermined termination has been reached. If it has, the scheduling scheme is output; otherwise, it returns to the global search operator design optimization module for re-optimization. The manager's expectation judgment module is used to determine whether the scheduling plan meets the manager's expectations. If it does, the optimization ends; if it does not, the optimization continues.

[0013] Preferably, in the maintenance task data collection module, the ship maintenance task data includes the number of ships undergoing maintenance, the number of ship maintenance tasks, the sequence of ship maintenance tasks, and the type and quantity of resource requirements for ship maintenance tasks.

[0014] Preferably, in the dedicated maintenance decoding strategy design module, the dedicated ship maintenance decoding strategy is described in detail as follows: Calculate the total time domain upper limit of the scheduling , ;in Indicates the ship repair task number, set Includes all maintenance tasks. Indicates task The time required to perform maintenance tasks; Initialize the task start time, end time, ship docking time, docking time, and the status of various resource occupancy. Each ship is dispatched in sequence according to the given ship maintenance order; Select the earliest available dock berth for the current vessel; Determine the earliest docking time for the current vessel; Based on the mission location attributes, the current ship maintenance mission is divided into dock-in-dock missions and dock-out-of-dock missions; The dockside tasks of the current vessels are scheduled sequentially; Iterate through the currently scheduled tasks within the dock; Select all schedulable tasks whose prerequisite tasks have been completed to form a candidate task set; Calculate the earliest start time of the candidate task, taking into account the completion time of the preceding task and the ship's docking time; The current task selects an executable maintenance mode; Determine whether the resources meet the conditions for task execution; If resources are sufficient, the task will be scheduled for execution at the earliest start time. If resources are insufficient, the task start time will be delayed until resources are available. Update resource usage status and add tasks that have been scheduled to the completed task set; Update the current docking and undocking time intervals of the vessels based on the task scheduling results; Update the availability of dock berths; The current vessels' off-dock tasks are scheduled sequentially; Select all schedulable off-dock tasks that have completed their preceding tasks to form a candidate task set; Calculate the earliest start time of candidate off-dock tasks, taking into account the completion time of prerequisite tasks and ship undocking constraints; Select an executable maintenance mode for the current off-dock mission; Under the condition of satisfying resource constraints, schedule the task to be executed at the earliest feasible time; Update resource utilization status and task scheduling status; Calculate the current completion time of the vessel; The final scheduling result is obtained by calculating the average completion time of all ships.

[0015] Thirdly, the present invention provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of the reinforcement learning-driven heuristic solution method for multi-ship dock maintenance scheduling as described in the first aspect of the present invention.

[0016] Fourthly, the present invention provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the reinforcement learning-driven heuristic solution method for multi-ship dock maintenance scheduling as described in the first aspect of the present invention.

[0017] Compared with the prior art, the beneficial effects of the present invention include at least the following: This invention considers the dry dock as a special critical resource in the multi-ship maintenance process and proposes a multi-ship dry dock maintenance scheduling problem. Furthermore, to improve the modeling and solution of this problem, it is abstracted into a multi-mode resource-constrained multi-project scheduling problem that considers resource competition relationships between projects, thus more accurately reflecting the actual scheduling characteristics in the multi-ship dry dock maintenance scenario.

[0018] To address the multi-ship docking maintenance scheduling problem, this invention designs three global search operators. The first is a ship crossover operator, used to adjust and exchange the maintenance order of ships; the second is a maintenance task disruption operator, which extracts some tasks from the current maintenance task sequence, breaking the original task structure and providing greater optimization space for subsequent searches; the third is a maintenance task reorganization operator, which re-inserts the extracted maintenance tasks into the existing maintenance task list, forming a new maintenance task order, thereby increasing the possibility of obtaining a better scheduling scheme.

[0019] This invention proposes a reinforcement learning-driven solution mechanism that adaptively guides the execution process of the global search operator and dynamically selects a more suitable search operation based on the state information during the solution process. This reduces the ineffective search overhead in the metaheuristic algorithm, improves the efficiency of computing resource utilization, and further enhances the solution efficiency and effectiveness of the multi-ship dock maintenance scheduling problem.

[0020] This invention can not only more accurately depict the actual situation of dock resource competition in multi-ship dock maintenance scenarios and improve the consistency between scheduling schemes and actual maintenance operations, but also enhance the search capability of complex scheduling problems through the synergistic effect of multiple global search operators, and improve the algorithm's solution efficiency and solution quality by leveraging a reinforcement learning-driven adaptive search mechanism, thereby providing a more efficient and reasonable solution method for multi-ship dock maintenance scheduling. Attached Figure Description

[0021] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0022] Figure 1 This is a schematic diagram of the multi-ship dock maintenance scheduling of the present invention, wherein... Figure 1 (a) illustrates the process priority constraints of the maintenance tasks for the two ships. Figure 1 (b) Showing dockworks that move dockworks to the dry dock for repairs. Figure 1 (c) The statement indicates that the remaining tasks will be carried out outside the dock for maintenance; Figure 2 This is a flowchart of the reinforcement learning-driven heuristic solution method for multi-ship dock maintenance scheduling according to the present invention; Figure 3 This is a schematic diagram of the encoding and initial scheduling scheme of the present invention, wherein... Figure 3 (a) is a diagram of the encoding. Figure 3 (b) is a schematic diagram of the initial scheduling scheme; Figure 4These are schematic diagrams of the three heuristic operators of this invention; Figure 5 This is a schematic diagram of the reinforcement learning-driven heuristic solution system for multi-ship dock maintenance scheduling according to the present invention; Figure 6 This is a block diagram of the electronic device structure of the present invention. Detailed Implementation

[0023] To make the above-mentioned objects, features, and advantages of the present invention more apparent and understandable, the technical solutions of the present invention will be described in detail below with reference to the accompanying drawings and specific embodiments. It should be noted that the described embodiments are merely some embodiments of the present invention, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0024] Example 1 Figure 1 A single disjunction diagram is used to describe the maintenance scheduling problem of multiple ships in a shared dock. Figure 1 (a) illustrates the process priority constraints of the maintenance tasks for the two ships. Figure 1 (b) Showing dockworks that move dockworks to the dry dock for repairs. Figure 1 (c) The statement indicates that the remaining tasks will be carried out outside the dry dock for maintenance. The multi-ship dry dock maintenance scheduling problem aims to make reasonable use of the dry dock and allocate resources to minimize the ship maintenance period, taking into account the dry dock capacity and maintenance support resources.

[0025] After introducing the multi-ship dock maintenance scheduling problem, this invention proposes a reinforcement learning-driven heuristic solution method for multi-ship dock maintenance scheduling. The overall steps of the proposed method are as follows. Furthermore, the overall framework of the reinforcement learning-driven heuristic solution is presented in [the document / section]. Figure 2 .

[0026] like Figure 2 As shown, the present invention proposes a reinforcement learning-driven heuristic solution method for multi-ship dock maintenance scheduling, which includes the following steps: Step 1: Collect ship repair task data, including the number of ships repaired, the number of ship repair tasks, the sequence of ship repair tasks, and the types and quantities of resources required for ship repair tasks.

[0027] Step 2: Design a three-layer coding system for ship maintenance scheduling. The first layer includes a ship maintenance sequence list; the second layer includes a ship maintenance task sequence list; and the third layer includes a list of ship maintenance task support modes. See the example below. Figure 3 (a).

[0028] Step 3: Design a dedicated ship maintenance decoding strategy and run the strategy to obtain an initial maintenance scheduling plan. See the example below. Figure 3 (b)

[0029] Step 4: Design three global search operators for optimizing ship maintenance scheduling. The first is the ship crossover operator, the second is the ship task disruption operator, and the third is the ship maintenance task reorganization operator. See the schematic diagram below. Figure 4 As shown.

[0030] Step 5: Design a reinforcement learning strategy to adaptively guide the global search operator search and balance the exploration and search phases.

[0031] Step 6: Under the guidance of reinforcement learning strategy, apply the three global search operators appropriately and record the optimized multi-ship dock maintenance scheduling scheme.

[0032] Step 7: Determine if the algorithm has reached its predetermined termination point. If so, output the scheduling scheme. Otherwise, return to step 4 and re-optimize.

[0033] Step 8: Determine if the scheduling plan meets the manager's expectations. If it does, the optimization ends. If not, continue optimizing.

[0034] In step 3 above, the specific decoding strategy is described in detail below: 3.1: Calculate the total time domain upper limit of scheduling , ;in Indicates the ship repair task number, set Includes all maintenance tasks. Indicates task The time required to perform maintenance tasks; 3.2: Initialize the task start time, end time, ship docking time, docking time, and the status of various resource occupancy.

[0035] 3.3: Schedule each ship in sequence according to the given ship maintenance order.

[0036] 3.4: Select the earliest available dock berth for the current vessel.

[0037] 3.5: Determine the earliest docking time for the current vessel.

[0038] 3.6: Based on the task location attribute, the current ship maintenance task is divided into dock-in-dock tasks and dock-out-of-dock tasks.

[0039] 3.7: Schedule the dockside tasks of the current vessel in sequence.

[0040] 3.8: Traverse the currently scheduled docking tasks.

[0041] 3.9: Select all schedulable tasks whose prerequisite tasks have been completed to form a candidate task set.

[0042] 3.10: Calculate the earliest start time of the candidate task, taking into account the completion time of the preceding task and the time of the ship entering the dock.

[0043] 3.11: Select an executable maintenance mode for the current task.

[0044] 3.12: Determine whether the resources meet the conditions for task execution.

[0045] 3.13: If resources are sufficient, the task will be scheduled to be executed at the earliest start time.

[0046] 3.14: If resources are insufficient, delay the task start time until resources are sufficient.

[0047] 3.15: Update resource usage status and add scheduled tasks to the completed task set.

[0048] 3.16: Update the current docking and undocking time intervals of the vessels based on the task scheduling results.

[0049] 3.17: Update the availability of dock berths.

[0050] 3.18: Schedule the current ship's off-dock tasks in sequence.

[0051] 3.19: Select all schedulable off-dock tasks whose prerequisite tasks have been completed to form a candidate task set.

[0052] 3.20: Calculate the earliest start time of candidate off-docking tasks, taking into account the completion time of prerequisite tasks and ship undocking constraints.

[0053] 3.21: Select an executable maintenance mode for the current off-dock mission.

[0054] 3.22: Under the condition of satisfying resource constraints, schedule the task to be executed at the earliest feasible time.

[0055] 3.23: Update resource utilization status and task scheduling status.

[0056] 3.24: Calculate the current completion time of the vessel.

[0057] 3.25: Calculate the average completion time of all ships to obtain the final scheduling result.

[0058] This invention considers the dry dock as a special critical resource in the multi-ship maintenance process and proposes a multi-ship dry dock maintenance scheduling problem. Furthermore, to improve the modeling and solution of this problem, it is abstracted into a multi-mode resource-constrained multi-project scheduling problem that considers resource competition relationships between projects, thus more accurately reflecting the actual scheduling characteristics in the multi-ship dry dock maintenance scenario.

[0059] To address the multi-ship docking maintenance scheduling problem, this invention designs three global search operators. The first is a ship crossover operator, used to adjust and exchange the maintenance order of ships; the second is a maintenance task disruption operator, which extracts some tasks from the current maintenance task sequence, breaking the original task structure and providing greater optimization space for subsequent searches; the third is a maintenance task reorganization operator, which re-inserts the extracted maintenance tasks into the existing maintenance task list, forming a new maintenance task order, thereby increasing the possibility of obtaining a better scheduling scheme.

[0060] This invention proposes a reinforcement learning-driven solution mechanism that adaptively guides the execution process of the global search operator and dynamically selects a more suitable search operation based on the state information during the solution process. This reduces the ineffective search overhead in the metaheuristic algorithm, improves the efficiency of computing resource utilization, and further enhances the solution efficiency and effectiveness of the multi-ship dock maintenance scheduling problem.

[0061] This invention can not only more accurately depict the actual situation of dock resource competition in multi-ship dock maintenance scenarios and improve the consistency between scheduling schemes and actual maintenance operations, but also enhance the search capability of complex scheduling problems through the synergistic effect of multiple global search operators, and improve the algorithm's solution efficiency and solution quality by leveraging a reinforcement learning-driven adaptive search mechanism, thereby providing a more efficient and reasonable solution method for multi-ship dock maintenance scheduling.

[0062] The following section presents a specific application example of a reinforcement learning-driven heuristic solution method for multi-ship dock maintenance scheduling.

[0063] The reinforcement learning-driven heuristic solution method of this invention is adapted to ship maintenance scheduling scenarios where multiple ships share limited dock resources, maintenance tasks are divided into in-dock and out-of-dock areas, and resource competition and task degradation effects coexist. It can be widely applied to different types of ship maintenance scenarios, such as coastal port ship repair bases, ocean shipping companies' own ship repair docks, inland waterway shipping hub ship repair centers, and special-purpose marine vessel repair docks. The following are specific application examples in four typical scenarios, all of which demonstrate the implementation of the core technologies of this invention: three-layer encoding, dedicated decoding, and reinforcement learning-driven global search operator optimization.

[0064] Application 1: Multi-ship docking maintenance scheduling at integrated ship repair bases in core coastal ports Application Scenarios Background A major coastal hub port has three large dry docks and eight repair berths, serving as a comprehensive ship repair base in the region. It routinely undertakes repair services for various ship types, including bulk carriers, container ships, and product oil tankers, with an average of 10-15 ships awaiting repair daily. The port is characterized by significant differences in ship types, complex repair processes, clear division of tasks between in-dock and out-of-dock operations, and limited repair resources (skilled workers, specialized equipment, and spare parts). Furthermore, different ship owners have different requirements for completion time. Traditional scheduling methods often result in both idle dry dock resources and ships waiting for repair, with some ship repair tasks experiencing extended lead times due to worsening waiting conditions. It is difficult to balance dry dock utilization and ship completion efficiency.

[0065] Specific implementation of the technical solution Data collection: Collect maintenance data for 12 vessels awaiting repair (6 bulk carriers, 4 container ships, and 2 product tankers), including the number of maintenance tasks per vessel (a total of 186 items), the process sequence of each task (e.g., hull rust removal must be completed before painting), the dock-in / dock-out task attributes (e.g., bottom repair is a dock-in task, deck equipment replacement is an dock-out task), and the resource requirements for each task (e.g., the number of special welding equipment and senior technicians).

[0066] The three-layer coding design consists of: the first layer being a maintenance sequence list for 12 ships, initially sorted by ship type and maintenance period; the second layer being an execution sequence list for 186 maintenance tasks within a single ship, following process sequence constraints; and the third layer being a maintenance support mode list for each task (e.g., deck welding can be performed using either "2 technicians + 1 welding machine" or "3 technicians + 2 welding machines").

[0067] The initial scheduling plan is generated through dedicated decoding: the total time domain limit of scheduling is calculated, the resource occupancy status of the dock, equipment, and personnel is initialized, the earliest available dock berth is allocated to each ship, and the dock-in-docking tasks (such as hull repair and hull structure reinforcement) are scheduled first, followed by the dock-out-of-docking tasks (such as navigation equipment maintenance and cargo loading and unloading equipment debugging). During the process, the task start time is dynamically adjusted according to the resource occupancy status to generate the initial scheduling plan. In the initial plan, the average completion time of 12 ships is 18.5 days, and the average dock utilization rate is 72%.

[0068] Reinforcement learning drives global operator optimization: a ship crossover operator is designed to adjust the maintenance sequence of different ship types (such as advancing the maintenance of container ships with shorter construction periods to improve dock turnaround efficiency), a task disruption operator extracts some off-dock tasks with weak process correlation (such as deck cleaning and label refurbishment), and a task reorganization operator re-inserts the extracted tasks into the idle time period of dock resources. The reinforcement learning strategy adaptively selects operator combinations based on real-time resource occupancy status and task completion progress to avoid invalid search.

[0069] Termination and Verification: After the algorithm iterates to a predetermined number of times (200 times), it outputs an optimized solution. After verification by the base scheduling manager, the solution meets the completion time requirements of each ship owner and the final scheduling is completed.

[0070] Application effect After optimization, the average completion time of the 12 ships was shortened to 13.2 days, a reduction of 28.6% compared to the initial plan; the average utilization rate of the dock increased to 89%, and the idle rate of maintenance resources (technicians, special equipment) was reduced to below 15%, effectively solving the resource competition problem of multiple ship types sharing the dock, while avoiding the deterioration of maintenance tasks due to ship waiting.

[0071] Application 2: Centralized maintenance and dispatching of fleets in shipyards owned by ocean shipping companies Application Scenarios Background A certain ocean shipping group owns two large-scale ship repair docks. Its 30 container ships need to return to the docks for annual maintenance at the end of the year. The maintenance tasks are characterized by batch processing, standardized processes, and significant task degradation effects (if a ship is docked and waiting for maintenance for more than 3 days, the time required for tasks such as hull seawater corrosion detection and rust removal will increase by 10%-20%). Moreover, the maintenance team consists of the company's own personnel, with a fixed number of personnel and a division of skills. Traditional variable neighborhood search algorithms are prone to getting stuck in local optima, resulting in some docks being overloaded and others being idle, leading to an overall excessively long maintenance period.

[0072] Specific implementation of the technical solution Data collection: Collect standardized maintenance task data for 30 container ships, including 15 fixed maintenance tasks for each ship, the sequence of tasks (e.g., seawater corrosion detection → rust removal → painting is a fixed sequence), dockside / dockside task attributes (e.g., hull structure inspection is a dockside task, and equipment maintenance inside the ship's hold is an off-dock task), and the resource requirements for each task (e.g., corrosion detection requires 2 professional inspection personnel, and rust removal requires 4 technicians).

[0073] The three-layer coding design consists of three layers: the first layer is a list of the order of docking maintenance for 30 ships, initially sorted by the time the ships return to port; the second layer is a list of the execution order of 15 maintenance tasks for each ship, strictly following standardized process constraints; and the third layer is a list of maintenance support modes for each task (for example, for rust removal tasks, there are two modes: "manual rust removal" or "mechanical rust removal", with different resource requirements and timelines).

[0074] The initial scheduling plan is generated by a dedicated decoding process: the total time domain limit of the scheduling is calculated, the available berth time of the two docks is initialized, the working status of the in-house maintenance personnel is initialized, the earliest dock entry time is determined for each ship, the core maintenance tasks in the dock are completed first, and then the dock-outside tasks are scheduled by utilizing the berth gap after the ships leave the dock, thus generating an initial scheduling plan. In the initial plan, the overall maintenance period for 30 ships is 45 days. For some ships, the maintenance tasks are deteriorated due to waiting to enter the dock, and the maintenance period is extended.

[0075] Reinforcement learning drives global operator optimization: The ship crossover operator is used to adjust the order of ships entering the dock, and ships with similar return times and complementary maintenance tasks are grouped and scheduled together (e.g., when ship A is painting in the dock, ship B is performing maintenance on equipment outside the dock, sharing technical resources); the task destruction operator is used to extract tasks that are deteriorating due to waiting, breaking the original task sequence; the task reorganization operator is used to redistribute deteriorating tasks to time periods with sufficient resources. The reinforcement learning strategy dynamically selects the operator execution probability based on the degree of task deterioration and the dock resource occupancy status, and prioritizes the scheduling optimization of tasks that are about to deteriorate.

[0076] Termination and Verification: After the algorithm iterates until the overall maintenance period no longer shortens, it outputs an optimized solution. After verification by the group's operations and maintenance department managers, the solution meets the requirement that the fleet can resume all operations before the Spring Festival, and the scheduling is successfully implemented.

[0077] Application effect After optimization, the overall maintenance period for 30 ships was shortened to 32 days, a reduction of 28.9% compared to the initial plan; no ships experienced a deterioration in their maintenance tasks due to waiting to enter the dry dock; the work efficiency of the company's own maintenance personnel increased by 35%; the resource utilization of the two dry docks became more balanced; and efficient scheduling of centralized maintenance for batch of ships was achieved.

[0078] Application 3: Co-dock maintenance scheduling for small and medium-sized vessels at inland waterway shipping hubs Application Scenarios Background A certain inland waterway shipping hub has two small and medium-sized dry docks, which mainly undertake the daily maintenance and emergency maintenance of small vessels such as inland barges, inland cargo ships, and tourist passenger ships. The vessels waiting for repair are characterized by a large number, small size, simple maintenance tasks, and high dispatch frequency. Moreover, maintenance resources (small maintenance equipment and local technicians) are limited. Emergency repair vessels (such as those with hull leaks or power failures) need to be dispatched first. Traditional dispatch methods are prone to the problem of "emergency vessels waiting while ordinary vessels occupy the dry docks", resulting in poor dispatch flexibility.

[0079] Specific implementation of the technical solution Data collection: Collect maintenance data for 20 inland waterway vessels awaiting repair on the same day (15 general cargo ships / barges and 5 emergency repair vessels), including the number of maintenance tasks per vessel (2-5 items / vessel, totaling 68 items), the sequence of tasks (e.g., emergency leak repair is a single core task with no process sequence), the dock-in / dock-out task attributes (e.g., hull repair is a dock-in task, and simple engine overhaul is an off-dock task), and the resource requirements for each task (e.g., leak repair requires 2 welders, and engine overhaul requires 1 mechanic).

[0080] The three-layer coding design consists of: the first layer being a maintenance sequence list for 20 vessels, with emergency vessels initially prioritized; the second layer being a sequence list for maintenance tasks within a single vessel, with emergency tasks as the core priority; and the third layer being a list of maintenance support modes for each task (e.g., for simple welding tasks, the "1 welder" or "2 welders" mode can be selected, with a work period of 4 hours and 2 hours respectively).

[0081] Dedicated decoding generates an initial plan: Calculate the total time domain limit for scheduling (based on a 24-hour daily schedule), initialize the berth status and available time of maintenance resources for the two docks, allocate the earliest available dock to emergency vessels, schedule emergency tasks within the dock first, then schedule maintenance tasks for ordinary vessels, and generate an initial scheduling plan. In the initial plan, the average maintenance completion time for the 5 emergency vessels is 6.5 hours, and the average waiting time for the 15 ordinary vessels is 8 hours.

[0082] Reinforcement learning drives global operator optimization: The ship crossover operator dynamically adjusts the maintenance sequence of emergency ships and ordinary ships, prioritizing the dry dock resources of emergency ships; the task destruction operator extracts deferred off-dock tasks (such as hull cleaning) from ordinary ships; and the task reorganization operator re-inserts the extracted off-dock tasks into the execution gaps of emergency ship dry dock tasks. The reinforcement learning strategy adaptively selects operators based on the urgency of emergency tasks and the real-time idle status of the dry dock, achieving a scheduling of "emergency priority and efficient utilization of gaps by ordinary ships".

[0083] Termination and Verification: When the algorithm iterates to complete the repair of all ships awaiting repair within a single day's scheduling cycle (24 hours), it outputs a solution. After verification by the dispatcher at the hub ship repair center, the solution meets the requirements for rapid repair of emergency ships, and the scheduling is completed.

[0084] Application effect After optimization, the average repair completion time of the 5 emergency vessels was shortened to 2.8 hours, meeting the timeliness requirements of emergency repairs on inland waterways; all 15 ordinary vessels completed repairs within 24 hours, with no vessels experiencing delays due to waiting; the daily utilization rate of the 2 docks increased to 95%, and the idle rate of repair resources decreased to below 5%, adapting to the high-frequency, high-flexibility shared-dock repair needs of small and medium-sized vessels on inland waterways.

[0085] Application 4: Cluster scheduling of dedicated repair docks for offshore wind power maintenance vessels Application Scenarios Background A certain offshore wind power industrial park has one dedicated wind power operation and maintenance vessel repair dock and three supporting maintenance berths. It undertakes maintenance services for special vessels such as operation and maintenance vessels and transport vessels from surrounding offshore wind farms. The maintenance of this type of vessel is characterized by high professionalism, clear boundaries between dock and off-dock tasks, and high requirements for vessel reactivation (the shutdown of operation and maintenance vessels will directly affect the daily operation and maintenance of offshore wind farms). Maintenance tasks include hull anti-corrosion repair and power system upgrades inside the dock, as well as the inspection and maintenance of wind power operation and maintenance equipment (cranes, boarding ladders) outside the dock. Moreover, maintenance resources are specialized equipment and technical personnel, which are scarce. Traditional dispatching methods are difficult to balance the needs of maintenance professionalism and rapid vessel reactivation.

[0086] Specific implementation of the technical solution Data Collection: Collect maintenance data for the 8 wind turbine maintenance vessels awaiting repair, including the number of maintenance tasks per vessel (8-12 items / vessel, totaling 76 items), the sequence of tasks (e.g., power system upgrades must be completed after hull anti-corrosion repairs, crane overhauls must be carried out after the vessel leaves the dock), the nature of the tasks inside / outside the dock (e.g., hull anti-corrosion and power system upgrades are core tasks inside the dock, crane and boarding ladder overhauls are special tasks outside the dock), and the special resource requirements for each task (e.g., marine anti-corrosion coating construction requires specialized equipment and 3 certified technicians, crane overhauls require 2 special equipment inspection personnel).

[0087] The three-layer coding design consists of three layers: the first layer is a list of the docking and maintenance sequence for the eight maintenance vessels, initially sorted by the urgency of the wind farm's maintenance needs; the second layer is a list of the execution sequence for the 76 maintenance tasks within a single vessel, following the process constraint of "prioritizing core tasks within the dock and connecting with special tasks outside the dock"; and the third layer is a list of maintenance support modes for each task (for example, anti-corrosion coating construction can choose between "conventional construction" or "high-efficiency construction" modes, with different resource requirements and schedules; high-efficiency construction requires an additional piece of specialized equipment).

[0088] The initial scheduling plan is generated by a dedicated decoding process: the total time domain limit of the scheduling is calculated, the available time of the dedicated dock and supporting berths is initialized, and the occupancy status of special maintenance equipment and personnel is determined. The earliest dock entry time is determined for each maintenance vessel, and the core power system and hull anti-corrosion tasks within the dock are prioritized. After the vessel leaves the dock, the maintenance equipment repair tasks outside the dock are scheduled at the supporting berths. The initial scheduling plan is generated. In the initial plan, the average return time of the 8 maintenance vessels is 7 days, and the utilization efficiency of special resources is 65%.

[0089] Reinforcement learning drives global operator optimization: the ship crossover operator is used to adjust the docking order of maintenance vessels, prioritizing maintenance vessels in the core area of ​​the wind farm; the task destruction operator is used to extract special tasks that are unrelated to docking processes (such as boarding ladder debugging); the task reorganization operator is used to redistribute the extracted tasks to the idle time periods of special resources. The reinforcement learning strategy adaptively selects operator combinations based on the urgency of the wind farm's maintenance needs and the real-time status of special resources, avoiding the idleness and waste of special resources.

[0090] Termination and Verification: After the algorithm iterates until the maintenance vessel's return time meets the wind farm's requirements, it outputs an optimized solution. The solution is verified by the industrial park's maintenance management department and ensures the daily operation and maintenance needs of the offshore wind farm, thus completing the scheduling implementation.

[0091] Application effect After optimization, the average return time of the eight wind power maintenance vessels was shortened to 4.2 days, a 40% reduction compared to the initial plan; the utilization efficiency of special maintenance resources was increased to 90%, the first-time completion rate of core tasks in the dock reached 100%, and there were no delays in vessel return due to improper scheduling. This achieved efficient scheduling of multiple vessels in the dock for special maintenance scenarios of offshore wind power maintenance vessels, ensuring the continuity of operation and maintenance of offshore wind farms.

[0092] Example 2 like Figure 5 As shown, this invention provides a reinforcement learning-driven heuristic solution system for multi-ship dock maintenance scheduling, comprising: The maintenance task data collection module is used to collect ship maintenance task data; the ship maintenance task data includes the number of ships to be maintained, the number of ship maintenance tasks, the process sequence of ship maintenance tasks, and the type and quantity of resource requirements for ship maintenance tasks.

[0093] The maintenance scheduling coding design module is used to design a three-layer coding method for ship maintenance scheduling. The first layer includes a ship maintenance sequence list; the second layer includes a ship maintenance task sequence list; and the third layer includes a ship maintenance task support mode list. The dedicated maintenance decoding strategy design module is used to design dedicated ship maintenance decoding strategies and run the dedicated ship maintenance decoding strategies to obtain an initial maintenance scheduling plan; The global search operator optimization module is used to design three global search operators for optimizing ship maintenance scheduling: the first is the ship crossover operator, the second is the ship task destruction operator, and the third is the ship maintenance task reorganization operator. The reinforcement learning strategy design module is used to design reinforcement learning strategies that adaptively guide the global search operator search and balance the exploration and search phases. The maintenance scheduling plan recording module is used to rationally apply three global search operators under the guidance of reinforcement learning strategies and record the optimized multi-ship dock maintenance scheduling plan. The algorithm's predetermined termination judgment module is used to determine whether the algorithm's predetermined termination has been reached. If it has, the scheduling scheme is output; otherwise, it returns to the global search operator design optimization module for re-optimization. The manager's expectation judgment module is used to determine whether the scheduling plan meets the manager's expectations. If it does, the optimization ends; if it does not, the optimization continues.

[0094] Specifically, in the dedicated maintenance decoding strategy design module, the dedicated ship maintenance decoding strategy is described in detail as follows: Calculate the total time domain upper limit of the scheduling , ;in Indicates the ship repair task number, set Includes all maintenance tasks. Indicates task The time required to perform maintenance tasks; Initialize the task start time, end time, ship docking time, docking time, and the status of various resource occupancy. Each ship is dispatched in sequence according to the given ship maintenance order; Select the earliest available dock berth for the current vessel; Determine the earliest docking time for the current vessel; Based on the mission location attributes, the current ship maintenance mission is divided into dock-in-dock missions and dock-out-of-dock missions; The dockside tasks of the current vessels are scheduled sequentially; Iterate through the currently scheduled tasks within the dock; Select all schedulable tasks whose prerequisite tasks have been completed to form a candidate task set; Calculate the earliest start time of the candidate task, taking into account the completion time of the preceding task and the ship's docking time; The current task selects an executable maintenance mode; Determine whether the resources meet the conditions for task execution; If resources are sufficient, the task will be scheduled for execution at the earliest start time. If resources are insufficient, the task start time will be delayed until resources are available. Update resource usage status and add tasks that have been scheduled to the completed task set; Update the current docking and undocking time intervals of the vessels based on the task scheduling results; Update the availability of dock berths; The current vessels' off-dock tasks are scheduled sequentially; Select all schedulable off-dock tasks that have completed their preceding tasks to form a candidate task set; Calculate the earliest start time of candidate off-dock tasks, taking into account the completion time of prerequisite tasks and ship undocking constraints; Select an executable maintenance mode for the current off-dock mission; Under the condition of satisfying resource constraints, schedule the task to be executed at the earliest feasible time; Update resource utilization status and task scheduling status; Calculate the current completion time of the vessel; The final scheduling result is obtained by calculating the average completion time of all ships.

[0095] Example 3 Based on the same concept, the present invention also provides a schematic diagram of a physical structure, such as... Figure 6 As shown, the server may include a processor 810, a communications interface 820, a memory 830, and a communication bus 840. The processor 810, communications interface 820, and memory 830 communicate with each other via the communication bus 840. The processor 810 can call logical instructions in the memory 830 to execute the steps of the reinforcement learning-driven heuristic solution method for multi-ship dock maintenance scheduling.

[0096] Furthermore, the logical instructions in the aforementioned memory 830 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0097] Example 4 Based on the same concept, the present invention also provides a non-transitory computer-readable storage medium storing a computer program containing at least one piece of code that can be executed by a master control device to control the master control device to implement the steps of the reinforcement learning-driven heuristic solution method for multi-ship dock maintenance scheduling.

[0098] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium accessible to a computer or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state drive).

[0099] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. This program can be stored in a computer-readable storage medium, and when executed, it can include the processes described in the above method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as ROM or random access memory (RAM), magnetic disks, or optical disks.

[0100] The embodiments described above are merely illustrative of several implementations of the present invention, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these modifications and improvements all fall within the scope of protection of the present invention. Therefore, the scope of protection of this patent should be determined by the appended claims.

Claims

1. A reinforcement learning-driven heuristic solution method for multi-ship dock maintenance scheduling, characterized in that, Includes the following steps: Step 1: Collect ship maintenance task data; Step 2: Design a three-layer coding system for ship maintenance scheduling. The first layer includes a ship maintenance sequence list; the second layer includes a ship maintenance task sequence list; and the third layer includes a list of ship maintenance task support modes. Step 3: Design a dedicated ship maintenance decoding strategy and run the dedicated ship maintenance decoding strategy to obtain an initial maintenance scheduling plan; Step 4: Design three global search operators for optimizing ship maintenance scheduling: the first is the ship crossover operator, the second is the ship task destruction operator, and the third is the ship maintenance task reorganization operator. Step 5: Design reinforcement learning strategies to adaptively guide the global search operator search and balance the exploration and search phases; Step 6: Under the guidance of reinforcement learning strategy, apply the three global search operators appropriately and record the optimized multi-ship dock maintenance scheduling scheme; Step 7: Determine whether the algorithm has reached its predetermined termination point. If so, output the scheduling scheme. Otherwise, return to step 4 and optimize again; Step 8: Determine whether the scheduling plan meets the manager's expectations. If it does, the optimization ends; if not, continue optimizing.

2. The reinforcement learning-driven heuristic solution method for multi-ship dock maintenance scheduling according to claim 1, characterized in that, In step 1, the ship repair task data includes the number of ships to be repaired, the number of ship repair tasks, the sequence of ship repair tasks, and the type and quantity of resources required for ship repair tasks.

3. The reinforcement learning-driven heuristic solution method for multi-ship dock maintenance scheduling according to claim 1, characterized in that, In step 3, the dedicated ship maintenance decoding strategy is described in detail below: 3.1: Calculate the total time domain upper limit of scheduling , ;in Indicates the ship repair task number, set Includes all maintenance tasks. Indicates task The time required to perform maintenance tasks; 3.2: Initialize the task start time, end time, ship docking time, docking time, and the status of various resource occupancy. 3.3: Dispatch each ship in sequence according to the given ship maintenance order; 3.4: Select the earliest available dock berth for the current vessel; 3.5: Determine the earliest docking time for the current vessel; 3.6: Based on the task location attribute, divide the current ship's maintenance tasks into dock-in-dock tasks and dock-out-of-dock tasks; 3.7: Sequentially schedule the dockside tasks of the current vessel; 3.8: Traverse the currently scheduled dockside tasks; 3.9: Select all schedulable tasks whose prerequisite tasks have been completed to form a candidate task set; 3.10: Calculate the earliest start time of the candidate task, taking into account the completion time of the preceding task and the ship's docking time; 3.11: Select an executable maintenance mode for the current task; 3.12: Determine if the resources meet the task execution conditions; 3.13: If resources are sufficient, the task will be scheduled for execution at the earliest start time; 3.14: If resources are insufficient, delay the task start time until resources are sufficient; 3.15: Update resource usage status and add scheduled tasks to the completed task set; 3.16: Update the current docking and undocking time intervals of the vessels based on the task scheduling results; 3.17: Update the available time of dock berths; 3.18: Schedule the current vessels' off-dock tasks sequentially; 3.19: Select all schedulable off-dock tasks whose prerequisite tasks have been completed to form a candidate task set; 3.20: Calculate the earliest start time of candidate off-docking tasks, taking into account the completion time of prerequisite tasks and ship undocking constraints; 3.21: Select an executable maintenance mode for the current off-dock mission; 3.22: Under the condition of satisfying resource constraints, schedule the task to be executed at the earliest feasible time; 3.23: Update resource utilization status and task scheduling status; 3.24: Calculate the current completion time of the vessel; 3.25: Calculate the average completion time of all ships to obtain the final scheduling result.

4. A reinforcement learning-driven heuristic solution system for multi-ship dock maintenance scheduling, characterized in that, include: The maintenance task data collection module is used to collect ship maintenance task data; The maintenance scheduling coding design module is used to design a three-layer coding method for ship maintenance scheduling. The first layer includes a ship maintenance sequence list; the second layer includes a ship maintenance task sequence list; and the third layer includes a ship maintenance task support mode list. The dedicated maintenance decoding strategy design module is used to design dedicated ship maintenance decoding strategies and run the dedicated ship maintenance decoding strategies to obtain an initial maintenance scheduling plan; The global search operator optimization module is used to design three global search operators for optimizing ship maintenance scheduling: the first is the ship crossover operator, the second is the ship task destruction operator, and the third is the ship maintenance task reorganization operator. The reinforcement learning strategy design module is used to design reinforcement learning strategies that adaptively guide the global search operator search and balance the exploration and search phases. The maintenance scheduling plan recording module is used to rationally apply three global search operators under the guidance of reinforcement learning strategies and record the optimized multi-ship dock maintenance scheduling plan. The algorithm's predetermined termination judgment module is used to determine whether the algorithm's predetermined termination has been reached. If it has, the scheduling scheme is output. Otherwise, return to the global search operator design optimization module for re-optimization; The manager's expectation judgment module is used to determine whether the scheduling plan meets the manager's expectations. If it does, the optimization ends; if it does not, the optimization continues.

5. The reinforcement learning-driven heuristic solution system for multi-ship dock maintenance scheduling according to claim 4, characterized in that, In the maintenance task data collection module, the ship maintenance task data includes the number of ships undergoing maintenance, the number of ship maintenance tasks, the sequence of ship maintenance tasks, and the type and quantity of resource requirements for ship maintenance tasks.

6. The reinforcement learning-driven heuristic solution system for multi-ship dock maintenance scheduling according to claim 4, characterized in that, The dedicated ship maintenance decoding strategy is described in detail in the dedicated maintenance decoding strategy design module as follows: Calculate the total time domain upper limit of the scheduling , ;in Indicates the ship repair task number, set Includes all maintenance tasks. Indicates task The time required to perform maintenance tasks; Initialize the task start time, end time, ship docking time, docking time, and the status of various resource occupancy. Each ship is dispatched in sequence according to the given ship maintenance order; Select the earliest available dock berth for the current vessel; Determine the earliest docking time for the current vessel; Based on the mission location attributes, the current ship maintenance mission is divided into dock-in-dock missions and dock-out-of-dock missions; The dockside tasks of the current vessels are scheduled sequentially; Iterate through the currently scheduled tasks within the dock; Select all schedulable tasks whose prerequisite tasks have been completed to form a candidate task set; Calculate the earliest start time of the candidate task, taking into account the completion time of the preceding task and the ship's docking time; The current task selects an executable maintenance mode; Determine whether the resources meet the conditions for task execution; If resources are sufficient, the task will be scheduled for execution at the earliest start time. If resources are insufficient, the task start time will be delayed until resources are available. Update resource usage status and add tasks that have been scheduled to the completed task set; Update the current docking and undocking time intervals of the vessels based on the task scheduling results; Update the availability of dock berths; The current vessels' off-dock tasks are scheduled sequentially; Select all schedulable off-dock tasks that have completed their preceding tasks to form a candidate task set; Calculate the earliest start time of candidate off-dock tasks, taking into account the completion time of prerequisite tasks and ship undocking constraints; Select an executable maintenance mode for the current off-dock mission; Under the condition of satisfying resource constraints, schedule the task to be executed at the earliest feasible time; Update resource utilization status and task scheduling status; Calculate the current completion time of the vessel; The final scheduling result is obtained by calculating the average completion time of all ships.

7. An electronic device, characterized in that, It includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of the reinforcement learning-driven heuristic solution method for multi-ship dock maintenance scheduling as described in claims 1-3.

8. A non-transitory computer-readable storage medium, characterized in that, It stores a computer program that, when executed by a processor, implements the steps of the reinforcement learning-driven heuristic solution method for multi-ship dock maintenance scheduling as described in claims 1-3.

Images