Deep Q-network and knowledge combined driven port ship emission reduction scheduling optimization method

Abstract

The invention discloses a deep Q-network and knowledge combined driven port ship emission reduction scheduling optimization method, and relates to the field of port ship traffic organization. Comprising the following steps: establishing a ship emission reduction scheduling optimization model by taking all ships planned to enter a port in a port entering period as research objects and taking the minimum total oil consumption of the ships entering the port and the shortest total time of the ships entering the port as targets; and designing a collaborative meta-heuristic algorithm jointly driven by a deep Q network and knowledge to solve the ship emission reduction scheduling optimization model, and obtaining an optimized ship emission reduction scheduling scheme. According to the invention, the optimal scheme of port ship scheduling is realized by optimizing the total time of entering and leaving the port and the fuel oil consumption of entering the port at the same time.

Description

Technical Field

[0001] The present invention relates to the field of port ship traffic organization, and particularly to a method for optimizing port water area ship emission reduction scheduling driven by a combination of deep Q-network and knowledge. Background Art

[0002] With the continuous improvement of global requirements for environmental protection and energy efficiency, the issue of ship emission reduction in port waters has attracted increasing attention. Especially under the influence of various factors such as multiple ship types, speed changes, ship arrival windows, and berth utilization efficiency, the existing scheduling schemes cannot meet the port ship scheduling requirements, resulting in low operating efficiency, directly affecting fuel consumption and carbon emission levels, and urgently needing improvement in both economic and environmental aspects. Summary of the Invention

[0003] The object of the present invention is to provide a method for optimizing port water area ship emission reduction scheduling driven by a combination of deep Q-network and knowledge. By simultaneously optimizing the total time for ships to enter and leave the port and the fuel consumption in the port, the optimization of port ship scheduling is achieved.

[0004] The technical means adopted by the present invention are as follows:

[0005] A method for optimizing port water area ship emission reduction scheduling driven by a combination of deep Q-network and knowledge, comprising the following steps:

[0006] S1: Taking all the planned inbound ships within an inbound cycle as the research object, with the goal of minimizing the total fuel consumption (TFC) and the shortest total inbound time (TTEP) of the inbound ships, a ship emission reduction scheduling optimization model is established;

[0007] S2: Solving the ship emission reduction scheduling optimization model based on a collaborative meta-heuristic algorithm driven by a combination of deep Q-network and knowledge, so as to obtain an optimized ship emission reduction scheduling scheme.

[0008] Further, the constraint conditions of the ship emission reduction scheduling optimization model include ship spatio-temporal allocation constraints, ship speed stage-keeping constraints, and ship safety time interval constraints, where:

[0009] The ship spatio-temporal allocation constraint means that each inbound ship can only be assigned to a specific position in the inbound sequence, and each position can only be occupied by one ship;

[0010] The ship speed stage-keeping constraint means that each inbound ship must maintain a specified speed in each inbound stage, where the inbound stage is divided into: anchorage - channel entrance stage, channel entrance - channel exit stage, channel exit - berth stage;

[0011] The ship safety time interval constraint means that a safe time interval needs to be maintained between two continuously sailing ships.

[0012] Furthermore, the ship emission reduction scheduling optimization model calculates the fuel consumption of the ship during different approach phases based on the ship fuel consumption heterogeneity characterization model.

[0013] Furthermore, the ship emission reduction scheduling optimization model is solved based on the collaborative meta-heuristic algorithm driven by the deep Q-network and knowledge, including:

[0014] S2.1: Generate an initial emission reduction scheduling plan based on the point-by-point insertion principle. The initial emission reduction plan includes the ship sequence with the minimum TTEP in the scheduling plan and the ship speeds during each approach phase.

[0015] S2.2: Decompose the ship emission reduction scheduling optimization problem into two sub-problems: determining the scheduling order of ships entering the port and selecting the sailing speed for each ship in each phase. Generate neighborhood solutions for the two sub-problems based on the constructed neighborhood search mechanisms NS1 and NS2 driven by the deep Q-network; NS1 reduces the TFC value by adjusting the ship's sailing speed, while NS2 reduces the TTEP by adjusting the ship's approach order.

[0016] S2.3: Based on the relationship between the ship's designed speed and fuel consumption, abstract the knowledge of reducing both TTEP and TFC by reducing the ship's sailing speed, and optimize the neighborhood solutions; the knowledge of reducing both TTEP and TFC by reducing the ship's sailing speed includes:

[0017] Theorem 1:

[0018] Suppose there is only one adjacent ship between ship B k-1 and B k and there is only one adjacent ship between ship B and B k and B k+1 If the stage index u satisfies a < u < b (i.e., a < b - 1), then the time T decreases as the sailing time k+1,u,l increases.

[0019] Theorem 2:

[0020] Suppose there is only one adjacent ship between ship B k-1 and B k and there is only one adjacent ship between ship B and B k and B k+1 and there is only one adjacent ship between ship B If the stage index u satisfies a < u < b, then the maximum increment Δ of satisfies:

[0021] where and represent two adjacent vessels on the same leg. If and have no idle time between them (i.e., only safety time slots), they are called adjacent, denoted as: where a, b, u are indices of different inbound sailing phases, a, b, u ∈ I, and I represents the set of inbound phases.

[0022] Furthermore, the steps for generating the initial emission reduction scheduling plan include:

[0023] First, randomly generate an initial scheduling plan, including the initial sequence of vessels and the speeds of the vessels sailing in each inbound phase, calculate the total sailing time of each vessel, and generate a new sequence of vessels in ascending order of sailing time;

[0024] Then, successively take out the vessels and insert them into the remaining sequence of vessels, find the optimal insertion point by trying all possible positions, and gradually construct a complete sequence;

[0025] Next, for each vessel in the complete sequence, remove it in order and try to re-insert it into other positions to find the optimal position that can further reduce TTEP, continuously update the sequence, and repeat this optimization process until the sequence can no longer be optimized. Finally, output the sequence of vessels with the minimum TTEP as the initial emission reduction scheduling plan.

[0026] Furthermore, generate neighborhood solutions for two sub-problems based on the neighborhood search mechanisms NS1 and NS2, including:

[0027] Divide all the plans into two sub-plans Pop1 and Pop2 according to the magnitude relationship of TTEP, where the TTEP of Pop1 is better than that of Pop2;

[0028] Pop1 and Pop2 generate neighborhood solutions through the search mechanisms NS1 and NS2 respectively, and use the deep Q-network to select search operators to generate new neighborhood solutions, including autonomously selecting the optimal action according to the current state through the deep Q-network. The state includes the number of vessels, the number of phases of the inbound voyage of the vessels, the sailing time of the vessels, the inbound completion time, and the fuel consumption of the vessels. The action is the neighborhood search operator in the search mechanisms NS1 and NS2, and the optimal action is the action with the largest reward value. The reward value is calculated based on the previous state, the current state, and the optimal solution.

[0029] Furthermore, the operator of the neighborhood search mechanism NS1 is designed to adjust the speed matrix V, specifically as follows:

[0030] ① NS1-1: Randomly exchange the sailing speeds at two different positions in the same phase;

[0031] ② NS1-2: Randomly exchange the sailing speeds of two ships at different stages;

[0032] ③ NS1-3: Randomly select two different positions in the same stage and reverse the sailing speed order of all ships between these two positions;

[0033] ④ NS1-4: Randomly select two different positions in the same stage and then randomly rearrange the sailing speeds of all ships between these two positions.

[0034] The operator of the neighborhood search mechanism NS2 is designed to adjust the inbound order B of ships, specifically as follows:

[0035] ① NS2-1: Randomly select a ship, mark it as "Ship2-1", and then place it successively in each position of this path to find the position Pos1 that can minimize TTEP, and place "Ship2-1" in this position;

[0036] ② NS2-2: Randomly select a ship, mark it as "Ship2-2-1", and then exchange its position successively with each ship in the sequence to find the ship that can minimize TTEP and mark it as "Ship2-2-2", and then exchange "Ship2-2-1" and "Ship2-2-2".

[0037] Compared with the prior art, the present invention has the following advantages:

[0038] 1. A method for optimizing the emission reduction scheduling of ships in port waters jointly driven by deep Q-network and knowledge provided by the present invention. The core principle of this method is to optimize the total time of ships entering and leaving the port and the fuel consumption in the port simultaneously to achieve the optimization of port ship scheduling. Specifically, this method comprehensively considers multiple constraints such as ship spatio-temporal allocation, speed adjustment, ship safety time interval, and heterogeneous ship fuel consumption, and formulates the optimal ship scheduling plan. Driven jointly by the deep Q-network and knowledge, it can adaptively adjust the scheduling strategy to achieve the dynamic balance between the scheduling efficiency of port ships and fuel consumption, thereby effectively reducing carbon emissions and improving the overall operation efficiency of the port.

[0039] 2. This method not only focuses on improving the inbound efficiency of ships, but also particularly emphasizes achieving the balance between efficiency and emission reduction. By optimizing the inbound scheduling plan and sailing speed of ships in the entire port, it can not only improve the inbound efficiency of ships, but also minimize fuel consumption to the greatest extent, ensuring the green development of the port. Therefore, this method can provide a scientific and reasonable ship scheduling optimization plan for port management departments, improve port transportation efficiency, reduce energy consumption, and promote the development of the port towards the green and intelligent direction, with significant economic and social value. Description of the Drawings

[0040] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the accompanying drawings required in the description of the embodiments or the prior art. Obviously, the accompanying drawings in the following description are some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other accompanying drawings can be obtained based on these drawings.

[0041] Figure 1 This is the flow of a port water area ship emission reduction scheduling optimization method jointly driven by a deep Q-network and knowledge in the embodiments of the present invention.

[0042] Figure 2 This is the flow of a drive collaborative meta-heuristic algorithm jointly driven by a deep Q-network and knowledge in the embodiments of the present invention.

[0043] Figure 3 This is a schematic diagram of the Tianjin Port area in the embodiments of the present invention.

[0044] Figure 4 This is the comparison of TTEP obtained by different methods in the embodiments of the present invention.

[0045] Figure 5 This is the comparison of TFC obtained by different methods in the embodiments of the present invention. Detailed implementation manners

[0046] In order to enable those skilled in the art of this technology to better understand the solution of the present invention, the following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0047] As Figure 1 shown, a port water area ship emission reduction scheduling optimization method jointly driven by a deep Q-network and knowledge specifically includes the following steps:

[0048] S1: Analyze the attributes of port ship traffic organization according to the ship arrival organization and scheduling rules of the port; the port ship arrival organization process is: during the arrival cycle, the ships to enter the port wait at the anchorage for the arrival notice. After the resources such as berths and channels are ready, the ships weigh anchor and enter the port in sequence. The method of this embodiment includes: establishing a ship emission reduction scheduling optimization model;

[0049] Step 1.1: Define the relevant parameters and variables as shown in Table 1.

[0050] Table 1 Parameters and variables

[0051]

[0052] Step 1.2: Establish a bi-objective function:

[0053] Minimize(TTEP,TFC)(1)

[0054]

[0055] TFC = F outside +F in-port (3)

[0056] Equation (1) represents minimizing the total time and total fuel consumption of ships entering the port. Equation (2) represents the calculation method of TTEP, and Equation (3) represents the calculation method of TFC.

[0057] Step 1.3: Characterize the spatio-temporal allocation constraints of ships

[0058]

[0059] Equations (4) and (5) indicate that each ship entering the port can only be assigned to a specific position in the entering sequence, and each position can only be occupied by one ship.

[0060] Step 1.4: Characterize the constraint of maintaining ship speed in stages

[0061]

[0062] Equations (6) and (7) indicate that each ship entering the port must maintain a specified speed in each entering stage, and Equation (8) specifies the departure time of the ship from the first position in the initial stage.

[0063] Step 1.5: Characterize the safe time interval constraint of ships

[0064]

[0065] Equations (9) and (10) indicate that a safe time interval must be maintained between two continuously sailing ships to prevent collision accidents.

[0066] Step 1.6: Establish a characterization model for ship fuel consumption heterogeneity

[0067]

[0068] Equation (11) represents the fuel consumption of the ship during the voyage from the anchorage to the channel entrance and from the channel entrance to the channel exit; Equation (12) represents the fuel consumption of the ship during the voyage from the channel exit to the berth.

[0069] S2: Design a drive collaborative meta - heuristic algorithm driven by deep Q - network and knowledge. Based on the emission reduction scheduling optimization model described in S1, solve the optimized ship emission reduction scheduling plan according to the basic information of ports and ships. The process of the drive collaborative meta - heuristic algorithm driven by deep Q - network and knowledge is as Figure 2 shown. Obtain the corresponding experimental simulation data according to the Tianjin Port waterway for example verification of the present invention, Figure 3 which is the schematic diagram of Tianjin Port area in the embodiment, including information on anchorage, waterway and berths.

[0070] Step 2.1: Initial emission reduction scheduling plan generation strategy based on the principle of point - by - point insertion

[0071] The input of this strategy is a randomly generated feasible scheduling plan B O , and the output is an optimized scheduling plan B, aiming to minimize TTEP. First, use the formula to calculate the sailing time T of each ship j j , where d i is the distance of each stage, and j is the sailing speed of ship j at each stage. Then, sort the ships in ascending order according to the calculated sailing time T to generate a new sequence For each ship in the sequence insert it into all possible positions in the partial sequence B to find the position that can minimize TTEP the most. For each ship, continue the insertion operation until all the ships in the sequence are inserted to generate the complete sequence B. Next, remove the ship initially in the first position of the complete sequence B and re - insert it into all empty slots in the remaining sequence to determine the best re - insertion position and generate a new sequence B * . If this sequence B * has an improvement in TTEP compared to B, update B to B * . This iterative process of removal and re - insertion is carried out for each ship in the sequence B, starting from the second position and continuing until the last ship. As long as TTEP can be improved, the algorithm continues to loop and execute this operation until no further improvement can be made. At this time, the current sequence B is the initial emission reduction scheduling plan. The role of the initial solution is to compare with the optimal solution generated after subsequent evolution, knowledge - driven steps, etc., and it is also effective in verifying the results produced by the algorithm.

[0072] Step 2.2: The ship emission reduction scheduling optimization problem is decomposed into two sub-problems: determining the scheduling order of ships entering the port and selecting the sailing speed for each stage of the ship. Based on the constructed neighborhood search mechanisms NS1 and NS2 driven by the deep Q-network, neighborhood solutions for the two sub-problems are generated. Specifically, it includes:

[0073] 2.2.1 Construct a co-evolution mechanism based on meta-heuristic algorithms

[0074] The ship emission reduction scheduling optimization includes two sub-problems, namely determining the scheduling order of ships entering the port and selecting the sailing speed for each stage of the ship. The solution spaces of these two sub-problems are extensive and interrelated. Therefore, it is necessary to optimize the two sub-problems simultaneously to achieve the approximate optimality of the final solution.

[0075] (1) According to the magnitude relationship of TTEP, all solutions (including the initial emission reduction scheduling solution and other solutions generated by generating the initial emission reduction scheduling solution) are divided into two sub-solutions Pop1 and Pop2, where the TTEP of Pop1 is better than that of Pop2.

[0076] (2) Pop1 and Pop2 generate neighborhood solutions through NS1 and NS2 respectively. NS1 and NS2 are two neighborhood search mechanisms driven by the deep Q-network. NS1 achieves a lower TFC value by adjusting the sailing speed of the ship, while NS2 obtains a lower TTEP by adjusting the ship's arrival order at the port.

[0077] The operator of NS1 is designed to adjust the speed matrix V, specifically as follows:

[0078] ① NS1-1: Randomly exchange the sailing speeds at two different positions in the same stage;

[0079] ② NS1-2: Randomly exchange the sailing speeds of two ships in different stages;

[0080] ③ NS1-3: Randomly select two different positions in the same stage and reverse the sailing speed order of all ships between these two positions;

[0081] ④ NS1-4: Randomly select two different positions in the same stage, and then randomly rearrange the sailing speeds of all ships between these two positions.

[0082] The operator of NS2 is designed to adjust the ship's arrival order B at the port, specifically as follows:

[0083] ① NS2-1: Randomly select a ship, marked as "Ship2-1", and then place it in each position of this path in turn. Find the position Pos1 that can minimize TTEP and place "Ship2-1" in this position;

[0084] ② NS2-2: Randomly select a ship, label it as "Ship2-2-1", then swap its position with each ship in the sequence in turn. Find the ship that can minimize TTEP and label it as "Ship2-2-2". Then swap "Ship2-2-1" and "Ship2-2-2".

[0085] Step 2.2.2: Deep Q-Network Driven Neighborhood Search Operator Selection Mechanism

[0086] NS1 and NS2 have been designed to optimize the solutions in the scheme. Each strategy adopts a unique search operator to generate new neighborhood solutions. At each stage of the search process, a deep Q-network is used to select the most effective search operator. The deep Q-network autonomously selects the optimal action according to the current state, reconstructing the selection of the neighborhood search operator as the deep Q-network recommending the operator most suitable for the current situation. The operator with the highest Q-value is considered the most likely to produce a better solution. The core elements of this process include state, action, and reward.

[0087] State: Considering the relationship between the ship emission reduction scheduling optimization problem and time and fuel consumption, five key indicators are proposed, including two indicators related to the problem attributes (the number of ships, the number of stages of the ship's inbound voyage), two indicators related to the current operating state of the inbound voyage (the sailing time of the ship, the inbound completion time), and one indicator related to energy consumption (the fuel consumption of the ship).

[0088] Action: The neighborhood search operators in NS1 and NS2 as described in S2 (NS1-1 to NS1-4 and NS2-1 to NS2-2) are defined as actions.

[0089] Reward: The reward is obtained by calculating the hypervolume rate, and is calculated based on the previous state (TTEP′, TFC′), the current state (TTEP, TFC), and the optimal solution (TTEP * , TFC * ). The optimal solution is obtained through Step 2.2.2. After the deep Q-network selects the most effective search operator, the optimal solution is obtained by executing this search mechanism.

[0090] The specific calculation steps of the reward are as follows:

[0091] (1) Calculate the reward

[0092] If both the current TTEP and TFC are lower than the TTEP′ and TFC′ of the previous state, then the reward calculation method is: Reward = (TTEP * - TTEP) - (TFC * - TFC) - (TTEP *-(TTEP′)-(TFC * -TFC′)

[0093] Among them, TTEP * and TFC * are the TTEP and TFC of the optimal solution. TTEP and TFC are the TTEP and TFC of the current state, and TTEP′ and TFC′ are the TTEP and TFC of the previous state.

[0094] If both the current TTEP and TFC are higher than the TTEP′ and TFC′ of the previous state, then the reward is calculated as: Reward = (TTEP * -TTEP′)-(TFC * -TFC′)-(TTEP * -TTEP)-(TFC * -TFC)

[0095] If neither of the above two cases is satisfied, that is, TTEP and TFC vary between the current state and the previous state, then the reward value is 0.

[0096] (2) Return the reward value: After the calculation is completed, return this reward value.

[0097] Step 2.3: Construct a knowledge-driven energy-saving strategy

[0098] Based on the cubic relationship between the designed ship speed and fuel consumption, abstract the knowledge of reducing both TTEP and TFC by reducing the ship's sailing speed, and drive the reduction of TFC by reducing the ship's sailing speed at the corresponding stage. This cubic relationship is:

[0099] F = k * v 3 , where F is the fuel consumption, k is the proportionality constant, and v is the ship speed

[0100] The specific theorem and proof are as follows:

[0101] Definition: and represent two adjacent ships on the same voyage segment. If there is no idle time (i.e., only safety time slots) between and , then they are called adjacent, denoted as: Among them, a, b, u are the indices of different inbound sailing stages, and a, b, u ∈ I. I represents the set of inbound stages.

[0102] Theorem 1:

[0103] Suppose there is only one adjacent ship between ship B k-1 and B k ​And vessel B k and B k+1 There is only one adjacent vessel If the stage index u satisfies a < u < b (i.e. a < b - 1), then the time T k+1,u,l decreases with the increase of the navigation time .

[0104] Proof 1:

[0105] In this problem, T k+1,u,l and the starting time are calculated by the following formulas (16) and (17):

[0106]

[0107] In the case of the existence of an adjacent , remains unchanged, and T k+1,u,l decreases with the increase of .

[0108] Theorem 2:

[0109] Suppose there is only one adjacent vessel between vessel B k-1 and B k and there is only one adjacent vessel between vessel B and B k and B k+1 If the stage index u satisfies a < u < b, then the maximum increment Δ of satisfies:

[0110] Proof 2.1:

[0111] When increases by Δ and remains unchanged, the new starting time and the new departure time are calculated by formulas (18) and (19):

[0112]

[0113] According to (18) and (19), if i ≤ u, the new starting time is advanced. If the new starting time satisfies i.e. then the proof holds.

[0114] Proof 2.2:

[0115] When increases by Δ and remains unchanged, the new starting time satisfies: At this time,

[0116]

[0117] When i ≥ u, according to in Proof 2.1, At this time, B k and B k+1 change their relative positions, and the new Thus, it can be obtained that the maximum increment Δ of

[0118] satisfies:

[0119] The parameter settings of the ship emission reduction scheduling optimization model in this embodiment are as follows: the number of scheduled ships is 14, and the standard sailing speeds of all ships follow a discrete uniform distribution U{3, 10} knots. The safety interval between adjacent ships is set to 15 minutes. The distance from the anchorage to the channel entrance is 5 nautical miles; the distance from the channel entrance to the channel exit is 16 nautical miles; the distance from the channel exit to the berth is 3 nautical miles. According to the tonnage distribution of ships in Tianjin Port and the data of Lloyd's Register, the main engine power of the ship is taken as 11000 kW. According to the Technical Specification for the Construction of Shore Power Facilities for Berthing Ships, each ship is equipped with three sets of auxiliary power devices, each with a rated power of 1960 kW and a total power of 5880 kW. When the ship is sailing in the anchorage-channel entrance and channel entrance-channel exit stages, the unit fuel consumption rate of the main engine is 0.206 g / kWh, and that of the auxiliary engine is 0.211 g / kWh. The load factor of the main engine is 0.8 when sailing in the anchorage-channel entrance and channel entrance-channel exit stages, and 0.85 when sailing in the channel exit-berth stage. The load factor of the auxiliary engine is always 0.5.

[0120] The ship emission reduction traffic scheduling optimization model is instantiated. The optimization model described in the present invention is solved 100 times repeatedly in the case of 14 ships and the average value is taken. The ship arrival sequence after optimization is shown in Table 2; the average values of the total ship arrival and departure time and the total ship waiting time finally obtained are respectively as Figure 4 and 5 shown. It can be seen that as the number of ships in the arrival and departure queue increases, the advantages of the multi-objective genetic algorithm based on the heuristic screening rule become gradually obvious and the stability is relatively high.

[0121] Table 2 Ship arrival and departure sequence and sailing speed after optimization

[0122]

[0123]

[0124] A method for optimizing the ship emission reduction scheduling in port waters driven by deep Q-network and knowledge is provided by the present invention. This method comprehensively considers multiple constraint conditions such as ship spatio-temporal allocation, speed adjustment, ship safety time interval, and heterogeneous ship fuel consumption, and formulates an optimal ship scheduling plan. Driven by the combination of deep Q-network and knowledge, it can adaptively adjust the scheduling strategy, achieve the dynamic balance between the port ship scheduling efficiency and fuel consumption, thereby effectively reducing carbon emissions and improving the overall operation efficiency of the port.

[0125] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements on some or all of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A deep Q-network and knowledge-driven optimization method for ship emission reduction scheduling in port waters, characterized in that: The following steps are involved: S1: Taking all ships that are scheduled to enter the port within a port entry cycle as the research object, and taking the minimum total fuel consumption and the shortest total port entry time as the goal, a ship emission reduction scheduling optimization model is established; S2: The ship emission reduction scheduling optimization model is solved based on the collaborative meta-heuristic algorithm driven by deep Q network and knowledge union, so as to obtain the optimized ship emission reduction scheduling plan.

2. According to claim 1, a deep Q-network and knowledge-driven optimization method for ship emission reduction scheduling in port waters is characterized in that: The constraints of the ship emission reduction scheduling optimization model include ship time and space allocation constraints, ship speed maintenance constraints in stages, and ship safety time distance constraints, among which: The ship time-space allocation constraint means that each incoming ship can only be assigned to a specific position in the port entry sequence, and each position can only be occupied by one ship; The ship speed maintenance constraint in stages means that each ship entering the port must maintain a specified speed in each port entry stage, wherein the port entry stage is divided into: anchorage-channel entrance stage, channel entrance-channel exit stage, channel exit-berth stage; The ship safety time interval constraint indicates that a safety time interval must be maintained between two consecutively sailing ships.

3. According to claim 2, a deep Q-network and knowledge-driven optimization method for ship emission reduction scheduling in port waters is characterized in that: The ship emission reduction scheduling optimization model calculates the fuel consumption of ships at different port entry stages based on a ship fuel consumption heterogeneity characterization model.

4. According to claim 1, a deep Q-network and knowledge-driven optimization method for ship emission reduction scheduling in port waters is characterized in that: The ship emission reduction scheduling optimization model is solved based on a collaborative meta-heuristic algorithm driven by deep Q-network and knowledge union, including: S2.1: Generate an initial emission reduction scheduling plan based on the point-by-point insertion principle, the initial emission reduction plan includes the ship sequence with the smallest TTEP in the scheduling plan and the speed of the ship at each port entry stage; S2.2: The ship emission reduction scheduling optimization problem is decomposed into two sub-problems: determining the scheduling order of ships entering the port and selecting the sailing speed for the ships in each stage. The neighborhood solutions of the two sub-problems are generated based on the constructed neighborhood search mechanisms NS1 and NS2 driven by the deep Q-network; NS1 reduces the TFC value by adjusting the sailing speed of the ship, while NS2 reduces the TTEP by adjusting the order of ships entering the port; S2.3: Based on the relationship between the ship's design speed and fuel consumption, the knowledge of reducing TTEP and TFC by reducing the ship's speed is abstracted, and the neighborhood solution is optimized; the knowledge of reducing TTEP and TFC by reducing the ship's speed includes: Theorem 1: Assume ship B k-1 and B k There is only one adjacent ship and ship B k and B k+1 There is only one adjacent ship If the phase index u satisfies a < u < b (i.e. a < b - 1), then the time T k+1,u,l decreases with the increase of the navigation time ​ Theorem 2: Assume ship B k-1 and B k There is only one adjacent ship between them And ship B k and B k+1 There is only one adjacent ship between them If the stage index u satisfies a < u < b, then The maximum increment Δ satisfies: in, and Indicates two adjacent ships on the same route. and There is no idle time between them (i.e. only safe time slots), they are called adjacent and are recorded as: Among them, a, b, u are the indices of different port entry navigation stages, a, b, u ∈ I, and I represents the set of port entry stages.

5. According to claim 4, a deep Q-network and knowledge-driven optimization method for ship emission reduction scheduling in port waters is characterized in that: The steps of generating the initial emission reduction scheduling plan include: First, an initial scheduling plan is randomly generated, including the initial sequence of ships and the speed of ships at each port entry stage, the total sailing time of each ship is calculated, and a new ship sequence is generated in ascending order according to the sailing time; Then, the ships are taken out one by one and inserted into the remaining ship sequence. The optimal insertion point is found by trying all possible positions, and the complete sequence is gradually constructed. Next, for each ship in the complete sequence, it is removed in order and tried to be reinserted into the remaining positions to find the optimal position that can further reduce TTEP. The sequence is continuously updated and this optimization process is repeated until the sequence can no longer be optimized. Finally, the ship sequence with the smallest TTEP is output as the initial emission reduction scheduling plan.

6. According to claim 4, a deep Q-network and knowledge-driven optimization method for ship emission reduction scheduling in port waters is characterized in that: Based on the neighborhood search mechanisms NS1 and NS2, neighborhood solutions for the two sub-problems are generated, including: All schemes are divided into two sub-schemes Pop1 and Pop2 according to the size of TTEP, among which Pop1 has better TTEP than Pop2; Pop1 and Pop2 generate neighborhood solutions through search mechanisms NS1 and NS2 respectively, and use deep Q-network to select search operators to generate new neighborhood solutions, including autonomously selecting the optimal action according to the current state through the deep Q-network, the state includes the number of ships, the number of stages of the ship's port voyage, the ship's sailing time, the port completion time and the ship's fuel consumption, the action is the neighborhood search operator in the search mechanisms NS1 and NS2, the optimal action is the action with the largest reward value, and the reward value is calculated based on the previous state, the current state and the optimal solution.

7. The method for optimizing ship emission reduction scheduling in port waters driven by deep Q-network and knowledge combination according to claim 6 is characterized in that: The operator design of the neighborhood search mechanism NS1 is used to adjust the velocity matrix V, as follows: ①NS1-1: Randomly exchange the navigation speeds of two different positions in the same stage; ②NS1-2: Randomly exchange the sailing speeds of two ships at different stages; ③NS1-3: Randomly select two different positions in the same stage and reverse the order of the sailing speeds of all ships between these two positions; ④NS1-4: Randomly select two different positions in the same stage, and then randomly rearrange the sailing speeds of all ships between these two positions. The operator design of the neighborhood search mechanism NS2 is used to adjust the ship's port entry sequence B, as follows: ①NS2-1: Randomly select a ship, mark it as "Ship2-1", and then put it into each position of the path in turn, find the position Pos1 that can minimize TTEP, and put "Ship2-1" into this position; ②NS2-2: Randomly select a ship and mark it as "Ship2-2-1", then swap its position with each ship in the sequence in turn, find the ship that can minimize TTEP and mark it as "Ship2-2-2", then swap "Ship2-2-1" and "Ship2-2-2".

Images