Optimization method for port ship emission reduction scheduling driven by deep Q-network and knowledge combination

Abstract

This invention discloses a port ship emission reduction scheduling optimization method driven by a deep Q-network and knowledge combination, relating to the field of port ship traffic organization. The method comprises the following steps: establishing a ship emission reduction scheduling optimization model, focusing on all ships scheduled to enter the port within a port arrival cycle, with the goal of minimizing total fuel consumption and total port arrival time; and designing a collaborative metaheuristic algorithm driven by a deep Q-network and knowledge combination to solve the ship emission reduction scheduling optimization model and obtain an optimized ship emission reduction scheduling solution. By simultaneously optimizing the total port arrival and departure time and port arrival fuel consumption, the method achieves the optimal solution for port ship scheduling.

Description

Technical Field

[0001] The present invention relates to the field of port ship traffic organization, and in particular to a method for optimizing ship emission reduction scheduling in port waters driven jointly by a deep Q-network and knowledge. Background Art

[0002] With increasing global demands for environmental protection and energy efficiency, reducing emissions from ships in port waters is gaining increasing attention. Existing scheduling solutions are unable to meet port vessel scheduling needs, particularly due to factors such as the variety of ship types, varying speeds, port arrival windows, and berth utilization efficiency. This leads to low operational efficiency, which directly impacts fuel consumption and carbon emissions. Improvements are urgently needed from both an economic and environmental perspective. Summary of the Invention

[0003] The purpose of this invention is to provide a deep Q-network and knowledge-based optimization method for ship emission reduction scheduling in port waters. By simultaneously optimizing the total time of ship entry and exit and fuel consumption in port, the optimization of port ship scheduling is achieved.

[0004] The technical means adopted in the present invention are as follows:

[0005] A deep Q-network and knowledge-based scheduling optimization method for ship emission reduction in port waters is proposed, comprising the following steps:

[0006] S1: Taking all ships scheduled to enter the port within a port entry cycle as the research objects, and taking the minimum total fuel consumption (TFC) and the shortest total time to enter the port (TTEP) as the goals, a ship emission reduction scheduling optimization model is established;

[0007] S2: A collaborative metaheuristic algorithm based on deep Q-network and knowledge-driven joint is used to solve the ship emission reduction scheduling optimization model, thereby obtaining the optimized ship emission reduction scheduling plan.

[0008] Furthermore, the constraints of the ship emission reduction scheduling optimization model include ship time and space allocation constraints, ship speed maintenance constraints in stages, and ship safety time distance constraints, where:

[0009] The ship spatiotemporal allocation constraint means that each incoming ship can only be assigned to a specific position in the port arrival sequence, and each position can only be occupied by one ship;

[0010] The ship speed maintenance constraint in stages means that each ship entering the port must maintain a specified speed in each port entry stage, where the port entry stages are divided into: anchorage-channel entrance stage, channel entrance-channel exit stage, and channel exit-berth stage;

[0011] The ship safety time interval constraint indicates that a safety time interval must be maintained between two consecutively sailing ships.

[0012] Furthermore, the ship emission reduction scheduling optimization model calculates the fuel consumption of the ship during different approach stages based on the ship fuel consumption heterogeneity characterization model.

[0013] Furthermore, the ship emission reduction scheduling optimization model is solved based on the collaborative metaheuristic algorithm driven by the deep Q-network and knowledge, including:

[0014] S2.1: Generate an initial emission reduction scheduling plan based on the point-by-point insertion principle. The initial emission reduction plan includes the ship sequence with the minimum TTEP in the scheduling plan and the ship speeds during each approach stage.

[0015] S2.2: Decompose the ship emission reduction scheduling optimization problem into two sub-problems: determining the scheduling order of ships entering the port and selecting the sailing speed for each stage of the ship. Generate neighborhood solutions for the two sub-problems based on the constructed neighborhood search mechanisms NS1 and NS2 driven by the deep Q-network; NS1 reduces the TFC value by adjusting the ship's sailing speed, while NS2 reduces the TTEP by adjusting the ship's approach order.

[0016] S2.3: Based on the relationship between the ship's designed speed and fuel consumption, abstract the knowledge of reducing both TTEP and TFC by reducing the ship's sailing speed, and optimize the neighborhood solutions; the knowledge of reducing both TTEP and TFC by reducing the ship's sailing speed includes:

[0017] [[ID=e16]]Theorem 1:

[0018] Suppose ship B k-1 and B k have only one adjacent ship between them and ship B k and B k+1 have only one adjacent ship between them If the stage index u satisfies a < u < b (i.e., a < b - 1), then the time T k+1,u,l decreases as the sailing time increases,

[0019] Theorem 2:

[0020] Suppose ship B k-1 and B k have only one adjacent ship between them and ship B k and B k+1 have only one adjacent ship between them If the stage index u satisfies a < u < b, then the maximum increment Δ of satisfies:

[0021] where, and Indicates two adjacent ships on the same route. and There is no idle time between them (i.e. only safe time slots), which are called adjacent and are recorded as: Among them, a, b, u are the indices of different port entry stages, a, b, u ∈ I, and I represents the set of port entry stages.

[0022] Furthermore, the step of generating the initial emission reduction scheduling plan includes:

[0023] First, an initial scheduling plan is randomly generated, including the initial ship sequence and the ship's speed at each port entry stage. The total sailing time of each ship is calculated, and a new ship sequence is generated in ascending order based on the sailing time.

[0024] Then, the ships are taken out one by one and inserted into the remaining ship sequence. By trying all possible positions, the optimal insertion point is found and the complete sequence is gradually constructed.

[0025] Next, for each ship in the complete sequence, it is removed in order and reinserted into the remaining positions to find the optimal position that can further reduce TTEP. The sequence is continuously updated. This optimization process is repeated until the sequence can no longer be optimized. Finally, the ship sequence with the smallest TTEP is output as the initial emission reduction scheduling plan.

[0026] Furthermore, neighborhood solutions to the two subproblems are generated based on the neighborhood search mechanisms NS1 and NS2, including:

[0027] All schemes are divided into two sub-schemes Pop1 and Pop2 according to the size of TTEP, among which Pop1 has better TTEP than Pop2;

[0028] Pop1 and Pop2 generate neighborhood solutions through search mechanisms NS1 and NS2, respectively, and use deep Q-networks to select search operators to generate new neighborhood solutions. This involves autonomously selecting the optimal action based on the current state through the deep Q-network. The state includes the number of ships, the number of stages in the ship's port voyage, the ship's sailing time, the port completion time, and the ship's fuel consumption. The action is the neighborhood search operator in search mechanisms NS1 and NS2. The optimal action is the action with the largest reward value, which is calculated based on the previous state, the current state, and the optimal solution.

[0029] Furthermore, the operator of the neighborhood search mechanism NS1 is designed to adjust the velocity matrix V as follows:

[0030] ①NS1-1: Randomly swap the navigation speeds of two different positions in the same phase;

[0031] ②NS1-2: Randomly swap the sailing speeds of two ships at different stages;

[0032] ③NS1-3: Randomly select two different positions in the same stage and reverse the order of the sailing speeds of all ships between these two positions;

[0033] ④NS1-4: Randomly select two different locations in the same stage, and then randomly rearrange the sailing speeds of all ships between these two locations.

[0034] The operator of the neighborhood search mechanism NS2 is designed to adjust the ship's port entry sequence B as follows:

[0035] ①NS2-1: Randomly select a ship, mark it as "Ship2-1", and then place it in each position of the path in turn, find the position Pos1 that can minimize TTEP, and place "Ship2-1" in this position;

[0036] ②NS2-2: Randomly select a ship and mark it as "Ship2-2-1", then swap its position with each ship in the sequence in turn, find the ship that can minimize TTEP and mark it as "Ship2-2-2", then swap "Ship2-2-1" and "Ship2-2-2".

[0037] Compared with the prior art, the present invention has the following advantages:

[0038] 1. The present invention provides a method for optimizing ship emission reduction scheduling in port waters, which is jointly driven by a deep Q-network and knowledge. The core principle of this method is to optimize port ship scheduling by simultaneously optimizing the total time for ships to enter and exit the port and their fuel consumption in the port. Specifically, this method comprehensively considers multiple constraints such as ship time and space allocation, speed adjustment, ship safety time interval, and heterogeneous ship fuel consumption to formulate an optimal ship scheduling plan. By jointly driving a deep Q-network and knowledge, the scheduling strategy can be adaptively adjusted to achieve a dynamic balance between port ship scheduling efficiency and fuel consumption, thereby effectively reducing carbon emissions and improving the overall operational efficiency of the port.

[0039] 2. This method not only focuses on improving ship arrival efficiency but also emphasizes striking a balance between efficiency and emissions reduction. By optimizing ship arrival scheduling and sailing speeds throughout the port, it not only improves ship arrival efficiency but also minimizes fuel consumption, ensuring the port's green development. Therefore, this method can provide port authorities with a scientific and rational ship scheduling optimization solution, improve port transportation efficiency, reduce energy consumption, and promote the development of ports in a green and intelligent direction, with significant economic and social value. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0041] Figure 1 This is a process flow of a method for optimizing ship emission reduction scheduling in port waters driven by a deep Q-network and knowledge in an embodiment of the present invention.

[0042] Figure 2 This is the driven collaborative meta-heuristic algorithm process driven by deep Q-network and knowledge in an embodiment of the present invention.

[0043] Figure 3 This is a schematic diagram of the Tianjin Port area in an embodiment of the present invention.

[0044] Figure 4 Comparison of TTEP obtained by different methods in the examples of the present invention.

[0045] Figure 5 Comparison of TFCs obtained by different methods in the embodiments of the present invention. DETAILED DESCRIPTION

[0046] In order to enable those skilled in the art to better understand the solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the embodiments described are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of the present invention.

[0047] like Figure 1 The method for optimizing ship emission reduction scheduling in port waters driven by a deep Q-network and knowledge is shown, and specifically includes the following steps:

[0048] S1: Analyze the attributes of the port's ship traffic organization based on the port's ship entry organization and scheduling rules. The port ship entry organization process is as follows: within the port entry period, ships planning to enter the port wait for port entry notification at the anchorage. Once berths, waterways, and other resources are ready, the ships weigh anchor and enter the port in order. The method of this embodiment includes: establishing a ship emission reduction scheduling optimization model;

[0049] Step 1.1: Define the relevant parameters and variables as shown in Table 1.

[0050] Table 1 Parameters and variables

[0051]

[0052] Step 1.2: Establish a dual objective function:

[0053] Minimize(TTEP,TFC)(1)

[0054]

[0055] TFC=F outside +F in-port (3)

[0056] Formula (1) represents the method of minimizing the total time and total fuel consumption of a ship entering the port, Formula (2) represents the calculation method of TTEP, and Formula (3) represents the calculation method of TFC.

[0057] Step 1.3: Characterize the space-time allocation constraints of ships

[0058]

[0059] Formulas (4) and (5) indicate that each incoming ship can only be assigned to a specific position in the port arrival sequence, and each position can only be occupied by one ship.

[0060] Step 1.4: Characterize the ship speed maintenance constraints in stages

[0061]

[0062] Formulas (6) and (7) indicate that each ship entering the port must maintain a specified speed in each port entry phase, and formula (8) specifies the time when the ship departs from the first position in the initial phase.

[0063] Step 1.5: Characterize the ship safety time constraint

[0064]

[0065] Formulas (9) and (10) indicate that a safe time interval must be maintained between two consecutively sailing ships to prevent collision accidents.

[0066] Step 1.6: Develop a model to characterize the heterogeneity of ship fuel consumption

[0067]

[0068] Formula (11) represents the fuel consumption of a ship during the navigation from anchorage to channel entrance and from channel entrance to channel exit. Formula (12) represents the fuel consumption of a ship during the navigation from channel exit to berth.

[0069] S2: Design a driven collaborative metaheuristic algorithm driven by deep Q-network and knowledge combination. Based on the emission reduction scheduling optimization model described in S1, the optimized ship emission reduction scheduling plan is solved according to the basic information of ports and ships. The process of the driven collaborative metaheuristic algorithm driven by deep Q-network and knowledge combination is as follows: Figure 2 The corresponding experimental simulation data of Tianjin Port waterway was obtained to verify the present invention. Figure 3 Schematic diagram of the Tianjin Port area in the embodiment, including information on anchorages, waterways, and berths.

[0070] Step 2.1: Initial emission reduction scheduling plan generation strategy based on the point-by-point insertion principle

[0071] The input of this strategy is a randomly generated feasible scheduling solution B O , the output is an optimized scheduling plan B, which aims to minimize TTEP. First, use the formula Calculate the sailing time T of each ship j j , where d i is the distance of each stage, is the sailing speed of ship j at each stage. Then, according to the calculated sailing time T j Arrange the ships in ascending order to generate a new sequence For the sequence Each ship in Insert it into all possible positions of the partial sequence B and find the position that best minimizes TTEP. For each ship, continue the insertion operation until the sequence All ships in are inserted, generating a complete sequence B. Next, the ship originally in the first position in the complete sequence B is removed and reinserted into all empty slots in the remaining sequence, and the optimal reinsertion position is determined to generate a new sequence B. * If the sequence B * If there is an improvement in TTEP compared to B, then update B to B * This iterative process of removal and reinsertion is repeated for each vessel in sequence B, starting with the second vessel and continuing until the last vessel. The algorithm continues this cycle as long as it improves the TTEP until no further improvement is possible. At this point, sequence B becomes the initial emission reduction schedule. This initial solution serves as a comparison with the optimal solution generated through subsequent evolutionary and knowledge-driven steps, and also serves to validate the algorithm's results.

[0072] Step 2.2: The ship emission reduction scheduling optimization problem is decomposed into two sub-problems: determining the scheduling order of ships entering the port and selecting the sailing speed for each stage. The neighborhood search mechanisms NS1 and NS2 driven by the deep Q-network are used to generate neighborhood solutions for the two sub-problems. Specifically, they include:

[0073] 2.2.1 Constructing a co-evolutionary mechanism based on meta-heuristic algorithms

[0074] Ship emission reduction scheduling optimization consists of two subproblems: determining the order in which ships enter a port and selecting the appropriate speeds for each phase. The solution spaces for these two subproblems are extensive and interconnected, necessitating simultaneous optimization of both subproblems to achieve near-optimality.

[0075] (1) According to the size of TTEP, all schemes (including the initial emission reduction scheduling scheme and other schemes generated by the initial emission reduction scheduling scheme) are divided into two sub-schemes Pop1 and Pop2, among which the TTEP of Pop1 is better than that of Pop2.

[0076] (2) Pop1 and Pop2 generate neighborhood solutions through NS1 and NS2, respectively. NS1 and NS2 are two neighborhood search mechanisms driven by deep Q-networks. NS1 achieves a lower TFC value by adjusting the ship’s sailing speed, while NS2 achieves a lower TTEP by adjusting the ship’s port entry order.

[0077] The operator of NS1 is designed to adjust the velocity matrix V as follows:

[0078] ①NS1-1: Randomly swap the navigation speeds of two different positions in the same phase;

[0079] ②NS1-2: Randomly swap the sailing speeds of two ships at different stages;

[0080] ③NS1-3: Randomly select two different positions in the same stage and reverse the order of the sailing speeds of all ships between these two positions;

[0081] ④NS1-4: Randomly select two different locations in the same stage, and then randomly rearrange the sailing speeds of all ships between these two locations.

[0082] The NS2 operator is designed to adjust the ship's port entry sequence B as follows:

[0083] ①NS2-1: Randomly select a ship, mark it as "Ship2-1", and then place it in each position of the path in turn, find the position Pos1 that can minimize TTEP, and place "Ship2-1" in this position;

[0084] ②NS2-2: Randomly select a ship and mark it as "Ship2-2-1", then swap its position with each ship in the sequence in turn, find the ship that can minimize TTEP and mark it as "Ship2-2-2", then swap "Ship2-2-1" and "Ship2-2-2".

[0085] Step 2.2.2: Neighborhood Search Operator Selection Mechanism Driven by Deep Q-Network

[0086] NS1 and NS2 are designed to optimize solutions within a scenario. Each strategy employs a unique search operator to generate new neighborhood solutions. At each stage of the search process, a deep Q-network is used to select the most effective search operator. The deep Q-network autonomously selects the optimal action based on the current state, refactoring the selection of neighborhood search operators into a deep Q-network recommendation of the operator most appropriate for the current situation. The operator with the highest Q value is considered most likely to produce a better solution. The core elements of this process include state, action, and reward.

[0087] Status: Considering the relationship between the ship emission reduction scheduling optimization problem and time and fuel consumption, five key indicators are proposed, including two indicators related to the problem attributes (the number of ships and the number of stages of the ship's port voyage), two indicators related to the current operation status of the port voyage (the ship's sailing time and the port completion time), and one indicator related to energy consumption (the ship's fuel consumption).

[0088] Action: The neighborhood search operators (NS1-1 to NS1-4 and NS2-1 to NS2-2) in NS1 and NS2 as described in S2 are defined as actions.

[0089] Reward: The reward is obtained by calculating the hypervolume rate, which is calculated based on the previous state (TTEP′, TFC′), the current state (TTEP, TFC) and the optimal solution (TTEP * ,TFC * ) is calculated. The optimal solution is obtained through step 2.2.2. After the deep Q-network selects the most effective search operator, it obtains the optimal solution by executing this search mechanism.

[0090] The specific steps for calculating the reward are as follows:

[0091] (1) Calculate rewards

[0092] If the current TTEP and TFC are both lower than the TTEP′ and TFC′ of the previous state, the reward calculation method is: Reward = (TTEP * -TTEP)-(TFC * -TFC)-(TTEP *-TTEP′)-(TFC * -TFC′)

[0093] Among them, TTEP * and TFC * are the TTEP and TFC of the optimal solution, TTEP and TFC are the TTEP and TFC of the current state, and TTEP′ and TFC′ are the TTEP and TFC of the previous state.

[0094] If the current TTEP and TFC are both higher than the TTEP′ and TFC′ of the previous state, the reward is calculated as: Reward = (TTEP * -TTEP′)-(TFC * -TFC′)-(TTEP * -TTEP)-(TFC * -TFC)

[0095] If neither of the above two conditions is met, that is, TTEP and TFC change differently between the current state and the previous state, then the reward value is 0.

[0096] (2) Return reward value: After the calculation is completed, the reward value is returned.

[0097] Step 2.3: Build a knowledge-driven energy-saving strategy

[0098] Based on the cubic relationship between the ship's design speed and fuel consumption, we abstract the knowledge that reducing the ship's sailing speed can simultaneously reduce TTEP and TFC, and reduce TFC by reducing the ship's sailing speed at the corresponding stage. The cubic relationship is:

[0099] F=k*v 3 , where F is the fuel consumption, k is the proportional constant, and v is the ship speed

[0100] The specific theorem and proof are as follows:

[0101] definition: and Indicates two adjacent ships on the same route. and There is no idle time between them (i.e. only safe time slots), which are called adjacent and are recorded as: Among them, a, b, u are the indices of different port entry stages, and a, b, u ∈ I. I represents the set of port entry stages.

[0102] Theorem 1:

[0103] Assume that ship B k-1 and B k There is only one adjacent ship between them And vessel B k and B k+1 There is only one adjacent vessel If the stage index u satisfies a < u < b (i.e. a < b - 1), then the time T k+1,u,l With the sailing time decreases as it increases.

[0104] Proof 1:

[0105] In this problem, T k+1,u,l and the starting time are calculated by the following formulas (16) and (17):

[0106]

[0107] In the presence of an adjacent case, remains unchanged, and T k+1,u,l decreases as increases.

[0108] Theorem 2:

[0109] Assume that there is only one adjacent vessel k-1 between vessel B k and B and there is only one adjacent vessel k between vessel B k+1 and B If the stage index u satisfies a < u < b, then [[ID=5​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​ at this time,

[0116]

[0117] When i ≥ u, according to Proof 2.1 Can get At this time, B k and B k+1 The relative position of From this we can get The maximum increment Δ satisfies:

[0118] The above Theorems 1 and 2 are introduced to further improve the quality of the solution and thus achieve the global optimization goal.

[0119] The parameter settings for the ship emission reduction scheduling optimization model in this example are as follows: the number of scheduled ships is 14, and the standard sailing speed of all ships follows a discrete uniform distribution U{3,10} knots. The safety interval between adjacent ships is set at 15 minutes. The distance between the anchorage and the channel entrance is 5 nautical miles; the distance between the channel entrance and the channel exit is 16 nautical miles; and the distance between the channel exit and the berth is 3 nautical miles. Based on the tonnage distribution of ships in Tianjin Port and data from Lloyd's Register, the ship's main engine power is assumed to be 11,000 kW. According to the "Technical Specifications for the Construction of Shore Power Facilities for Ships at Berth," each ship is equipped with three auxiliary power units, each with a rated power of 1,960 kW, for a total power of 5,880 kW. The specific fuel consumption of the main engine during the anchorage-channel entrance and channel entrance-channel exit phases is 0.206 g / kWh, and the specific fuel consumption of the auxiliary engine is 0.211 g / kWh. The load factor for the main engine during navigation from anchorage to channel entrance and channel entrance to channel exit is 0.8, and for navigation from channel exit to berth is 0.85. The load factor for the auxiliary engine is always 0.5.

[0120] The ship emission reduction traffic scheduling optimization model was instantiated. The optimization model of the present invention was repeatedly solved 100 times with 14 ships and the average value was taken. The ship entry sequence obtained after optimization is shown in Table 2. The final average values ​​of the total time for ships to enter and leave the port and the total waiting time of ships are shown in Table 2. Figure 4 、 5 As shown in Figure 2, it can be seen that with the increase in the number of ships in the port entry and exit queues, the advantages of the multi-objective genetic algorithm based on heuristic screening rules become increasingly obvious and its stability is relatively high.

[0121] Table 2 Optimized ship entry and exit sequence and speed

[0122]

[0123]

[0124] This paper proposes a method for optimizing ship emission reduction scheduling in port waters, driven by a deep Q-network and knowledge. This method comprehensively considers multiple constraints, including ship temporal and spatial allocation, speed adjustment, ship safety headway, and heterogeneous fuel consumption, to develop an optimal ship scheduling plan. By combining a deep Q-network with knowledge, this method adaptively adjusts scheduling strategies to achieve a dynamic balance between port ship scheduling efficiency and fuel consumption, effectively reducing carbon emissions and improving overall port operational efficiency.

[0125] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the above embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A deep Q-network and knowledge-driven optimization method for ship emission reduction scheduling in port waters, characterized by: The following steps are involved: S1: Taking all ships scheduled to enter the port within a port entry cycle as the research objects, and taking the minimum total fuel consumption and the shortest total port entry time as the goals, a ship emission reduction scheduling optimization model is established; S2: Solve the ship emission reduction scheduling optimization model based on the deep Q-network and knowledge-driven collaborative metaheuristic algorithm to obtain the optimized ship emission reduction scheduling plan, including: S2.1: Generate an initial emission reduction scheduling plan based on the point-by-point insertion principle, which includes the ship sequence with the shortest total port entry time in the scheduling plan and the ship's speed at each port entry stage; S2.2: The ship emission reduction scheduling optimization problem is decomposed into two sub-problems: determining the scheduling order for ships to enter the port and selecting the sailing speed for each stage. Neighborhood solutions for these two sub-problems are generated based on the constructed neighborhood search mechanisms NS1 and NS2 driven by deep Q-networks. NS1 reduces the total fuel consumption of ships entering the port by adjusting the sailing speed, while NS2 reduces the total port entry time by adjusting the order in which ships enter the port. S2.3: Based on the relationship between the ship's design speed and fuel consumption, abstract the knowledge that reducing the ship's sailing speed can simultaneously reduce the total port arrival time and total fuel consumption of the ship entering the port, and optimize the neighborhood solution. The knowledge of reducing the total port arrival time and total fuel consumption by reducing the ship's sailing speed includes: Theorem 1: Assume ship B k-1 and B k There is only one adjacent ship between them And ship B k and B k+1 There is only one adjacent ship between them If the stage index u satisfies a < u < b (i.e. a < b - 1), then the time T k+1,u，l decreases as the navigation time increases Theorem 2: Suppose vessel B k-1 and B k There is only one adjacent vessel between them And vessel B k and B k+1 There is only one adjacent vessel between them If the stage index u satisfies a < u < b, then The maximum increment Δ of in, and Indicates two adjacent ships on the same route. If and There is no idle time between them, so they are called adjacent and are written as: Among them, a, b, u are the indices of different port entry stages, a, b, u ∈ I, I represents the set of port entry stages.

2. The method for optimizing ship emission reduction scheduling in port waters driven by a deep Q-network and knowledge combination according to claim 1 is characterized in that: The constraints of the ship emission reduction scheduling optimization model include ship time and space allocation constraints, ship speed maintenance constraints in stages, and ship safety time distance constraints, among which: The ship spatiotemporal allocation constraint means that each incoming ship can only be assigned to a specific position in the port arrival sequence, and each position can only be occupied by one ship; The ship speed maintenance constraint in stages means that each ship entering the port must maintain a specified speed in each port entry stage, where the port entry stage is divided into: anchorage to channel entrance stage, channel entrance to channel exit stage, and channel exit to berth stage; The ship safety time interval constraint indicates that a safety time interval must be maintained between two consecutively sailing ships.

3. The method for optimizing ship emission reduction scheduling in port waters driven by a deep Q-network and knowledge combination according to claim 2 is characterized in that: The ship emission reduction scheduling optimization model calculates the fuel consumption of ships at different port entry stages based on a ship fuel consumption heterogeneity characterization model.

4. The method for optimizing ship emission reduction scheduling in port waters driven by a deep Q-network and knowledge combination according to claim 1 is characterized in that: The steps of generating the initial emission reduction scheduling plan include: First, an initial scheduling plan is randomly generated, including the initial ship sequence and the ship's speed at each port entry stage. The total sailing time of each ship is calculated, and a new ship sequence is generated in ascending order based on the sailing time. Then, the ships are taken out one by one and inserted into the remaining ship sequence. By trying all possible positions, the optimal insertion point is found and the complete sequence is gradually constructed. Next, for each ship in the complete sequence, it is removed in order and tried to be reinserted into the remaining positions to find the optimal position that can further reduce the total port arrival time. The sequence is continuously updated. This optimization process is repeated until the sequence can no longer be optimized. Finally, the ship sequence with the smallest total port arrival time is output as the initial emission reduction scheduling plan.

5. The method for optimizing ship emission reduction scheduling in port waters driven by a deep Q-network and knowledge combination according to claim 1 is characterized in that: Based on the neighborhood search mechanisms NS1 and NS2, neighborhood solutions for the two subproblems are generated, including: All the schemes are divided into two sub-schemes Pop1 and Pop2 according to the total arrival time. The total arrival time of Pop1 is better than that of Pop2. Pop1 and Pop2 generate neighborhood solutions through search mechanisms NS1 and NS2, respectively, and use deep Q-networks to select search operators to generate new neighborhood solutions. This involves autonomously selecting the optimal action based on the current state through the deep Q-network. The state includes the number of ships, the number of stages in the ship's port voyage, the ship's sailing time, the port completion time, and the ship's fuel consumption. The action is the neighborhood search operator in search mechanisms NS1 and NS2. The optimal action is the action with the largest reward value, which is calculated based on the previous state, the current state, and the optimal solution.

6. The method for optimizing ship emission reduction scheduling in port waters driven by a deep Q-network and knowledge combination according to claim 5 is characterized in that: The operator of the neighborhood search mechanism NS1 is designed to adjust the velocity matrix V as follows: ①NS1-1: Randomly swap the navigation speeds of two different positions in the same phase; ②NS1-2: Randomly swap the sailing speeds of two ships at different stages; ③NS1-3: Randomly select two different positions in the same stage and reverse the order of the sailing speeds of all ships between these two positions; ④NS1-4: Randomly select two different locations in the same phase, and then randomly rearrange the sailing speeds of all ships between these two locations; The operator of the neighborhood search mechanism NS2 is designed to adjust the ship's port entry sequence B as follows: ①NS2-1: Randomly select a ship, mark it as "Ship2-1", and then place it in each position of the path in turn, find the position Pos1 that minimizes the total time to enter the port, and place "Ship2-1" in this position; ②NS2-2: Randomly select a ship and mark it as "Ship2-2-1", then swap its position with each ship in the sequence in turn, find the ship that can minimize the total port arrival time and mark it as "Ship2-2-2", then swap "Ship2-2-1" and "Ship2-2-2".

Images