A multi-objective train scheduling method and system integrated with Q learning

Abstract

This invention discloses a multi-objective train scheduling method and system integrating Q-learning, aiming to solve the problems of low efficiency and insufficient multi-objective coordination in existing technologies for large-scale solutions. Its core is to transform train scheduling into a work-shop scheduling problem with congestion constraints, establishing a mixed-integer programming model with the objectives of minimizing delay penalties, passenger dissatisfaction, and the total number of operating vehicles. A two-layer vector encoding of scheduling ordering and vehicle allocation is adopted, combined with multi-block left insertion heuristic decoding adaptation constraints. A multi-objective evolutionary framework is constructed by improving the NSGA-II algorithm, incorporating integrated population selection and customized crossover operators. Q-learning is introduced to drive variable neighborhood search, and adaptive selection of neighborhood operators enhances search intelligence. Experimental verification shows that this method outperforms mainstream algorithms in both solution efficiency and solution quality in large-scale examples and real subway data. It can efficiently balance multiple objectives, adapt to dynamic train formations and passenger flow fluctuations, and has engineering application value.

Description

Technical Field

[0001] This invention relates to the field of subway or train scheduling, specifically to a multi-objective train scheduling method and system integrating Q-learning. Background Technology

[0002] As a core component of public transportation, urban rail transit systems directly impact operating cost control, train punctuality, and passenger travel experience through the scientific nature of their train scheduling, making it a key research area in transportation. In recent years, dynamic train formation technology, with its advantages of flexible vehicle formation, dynamic grouping, and decoupling, has demonstrated significant value in improving passenger satisfaction and enhancing fleet operational flexibility, and has gradually become a core research direction for train scheduling optimization.

[0003] The Train Timetable Problem (TTP) is a core component of train scheduling, with the core objective of developing an optimal periodic timetable while satisfying track capacity constraints and operational limitations. Existing research has extensively explored TTP and related issues: Cacchiani et al. (2016) proposed a solution to minimize the conflict between ideal timetables of different operators for TTP at railway nodes; Xu et al. (2018) used a Lagrange relaxation heuristic to solve the integrated optimization problem of train timetables and locomotive allocation; Li and Ni (2022) applied multi-agent deep reinforcement learning (DRL) to TTP solving; Tian et al. (2024) jointly optimized train timetables and skip-stop decisions in a high-speed railway corridor scenario; Yue et al. (2024) and Dong et al. (2024) respectively employed reinforcement learning methods to solve the problems of timetable rescheduling and timetable-train trajectory collaborative optimization.

[0004] In scenarios involving dynamic demand and multi-objective optimization, existing technologies have further expanded the research dimensions: Barrena et al. (2014) used the branch-cut (B&C) algorithm to study TTP under dynamic demand with the goal of minimizing the average passenger waiting time; Robenek et al. (2016) incorporated passenger satisfaction into railway scheduling optimization through a mixed integer linear programming (MILP) model; and Gong et al. (2024) addressed the problem of uneven spatiotemporal distribution of passenger demand by using the adaptive large neighborhood search (ALNS) algorithm to realize TTP and vehicle circulation. The plan involves comprehensive optimization; Zhong et al. (2024) used the Variable Neighborhood Search (VNS) method to solve the TTP with demand fluctuations and energy efficiency requirements; Wen et al. (2025) used the Soft Actor-Critic (DRL) method to extract dynamic passenger travel behavior and train operation event features to optimize TTP; Feng et al. (2023) and Liu et al. (2024) explored the integrated optimization problem of TTP and train formation in high-speed railway networks using the lower bound algorithm and the hybrid genetic algorithm (GA), respectively.

[0005] However, existing technologies still have significant shortcomings and deficiencies, making it difficult to meet the complex needs of actual urban rail transit scheduling:

[0006] Large-scale problem solving is inefficient: Most existing technologies rely on deterministic optimization methods such as branch and bound and branch-cut algorithm (Chai et al., 2024). These methods are based on mathematical programming models. When the scale of the scheduling problem increases (such as the number of trains and stations), the computation time increases exponentially and cannot be applied to the real-time scheduling needs of large-scale rail transit networks.

[0007] Insufficient synergy in multi-objective optimization: Some studies focus only on a single optimization objective (such as passenger waiting time and operating costs), failing to effectively balance the synergistic optimization of multiple objectives such as train delay penalties, passenger dissatisfaction and the total number of operating vehicles. Furthermore, there are conflicting relationships among multiple objectives (such as increasing the number of vehicles can reduce capacity shortages but increase costs), and existing algorithms lack an efficient balancing mechanism.

[0008] Poor adaptability of congestion constraints: There is a congestion constraint in urban rail transit networks where "only one train is allowed to pass through the same section at the same time". However, existing decoding mechanisms (such as "first come, first served") fail to fully consider this constraint, resulting in low track utilization and difficulty in balancing the feasibility and efficiency of the solution.

[0009] The search strategy lacks adaptability: the neighborhood search operators in existing algorithms such as VNS and GA mostly adopt random selection or fixed rules, and fail to adaptively adjust the selection strategy according to the dynamic characteristics of the optimization process, resulting in insufficient exploration of the search space and the algorithm being prone to getting trapped in local optima.

[0010] The integration of dynamic grouping and scheduling is not in-depth: Although some studies have considered dynamic grouping constraints, they have failed to deeply integrate them with multi-objective scheduling and efficient optimization algorithms, resulting in the fleet's flexibility advantage not being fully utilized and making it difficult to adapt to complex and ever-changing passenger flow demands. Summary of the Invention

[0011] To address the aforementioned problems, this invention provides solutions to the technical deficiencies mentioned above. Focusing on the dynamic formation and multi-objective scheduling optimization problem of urban rail transit, this invention designs a multi-objective evolutionary optimization algorithm integrating Q-learning. It aims to solve key technical problems such as low efficiency in solving large-scale problems, insufficient multi-objective coordination, and poor adaptability to congestion constraints. This provides an efficient and feasible optimization solution for train scheduling, possessing significant engineering application value and academic significance.

[0012] Specifically, one aspect of this invention discloses a multi-objective train scheduling method integrating Q-learning, comprising: considering the actual constraints of dynamic formation and continuous locomotive operation, transforming the train scheduling problem into a job shop scheduling problem with congestion constraints, and clarifying the constraints; defining optimization indices based on the number of times train capacity is insufficient, the total delay value, and the total number of vehicles, and constructing a mixed integer programming mathematical model based on the job shop scheduling problem with congestion constraints through key parameters including section passage priority, train running time, and passenger arrival volume, and decision variables including scheduling order, number of vehicles, and arrival time, which can obtain feasible solutions for small-scale problems through the CPLEX solver; establishing a two-layer vector encoding mechanism including scheduling order and vehicle allocation, where the first dimension represents the train scheduling order and the second dimension defines the number of vehicles allocated, realizing synchronous optimization of scheduling and vehicle configuration; proposing and proving a congestion-related lemma, and designing a multi-block left insertion decoding strategy based on the lemma. Under the premise of satisfying congestion constraints, the algorithm aims to reduce train idle time, improve track utilization, and enhance solution quality. A multi-objective processing framework is established, based on the NSGA-II algorithm, introducing an ensemble population tournament selection mechanism. This mechanism integrates historical high-quality solutions to guide the search direction, and employs three crossover operators—sequential crossover, task crossover, and double-point crossover—to achieve population evolution. Based on critical path theory, various neighborhood structures are designed to cover the dual-layer coding space of scheduling and vehicle allocation. A variable neighborhood search structure is established and used as the action space. A reward function is designed based on the multi-objective optimization results, and the variable neighborhood search is driven by the Q-learning method. The Q-table update mechanism trains the operator selection probability matrix, enabling adaptive selection of neighborhood search operators and improving algorithm optimization efficiency. Finally, the parent and offspring populations are merged, and the next generation is selected using non-dominated sorting and congestion distance functions. The multi-objective evolutionary optimization and variable neighborhood search steps are repeated until the termination condition is met, outputting a Pareto-optimal train scheduling scheme set.

[0013] In another aspect, this invention discloses a multi-objective train scheduling system integrating Q-learning, comprising: a data input module for acquiring train operation data, network topology data, passenger demand data, and operational constraint parameters, wherein the train operation data includes train passenger capacity, running time of each segment, and planned arrival time, and the operational constraint parameters include minimum safe interval time and segment congestion rules; a mathematical modeling module for establishing a mixed integer programming mathematical model of the job scheduling problem with congestion constraints based on the data acquired by the data input module, wherein the model aims to minimize train delay penalties, passenger dissatisfaction, and the total number of operating vehicles, and includes congestion constraints, segment passage priority constraints, train arrival time constraints, passenger boarding and alighting constraints, and train delay constraints; an encoding and decoding module, comprising an encoding unit and a decoding unit, wherein the encoding unit is used to encode the scheduling problem using a two-layer vector encoding mechanism of scheduling order and vehicle allocation, wherein the first dimension vector represents the train scheduling order, and the second dimension vector represents the number of vehicles allocated to each train; the decoding unit is used to decode based on the congestion constraint-related lemma and through a multi-block left insertion heuristic rule, calculating the start time and completion time of trains in each segment; and a population initialization module for... The NEH heuristic algorithm is used to generate initial solutions for the scheduling part, and a random heuristic method is used to generate initial solutions for the vehicle allocation part. After performing non-dominated sorting on the initial population, high-quality individuals are selected to form a ensemble subpopulation. A multi-objective evolutionary optimization module is used to select individuals based on a tournament selection mechanism of the ensemble population, applying crossover and mutation operators to generate the offspring population. The crossover operators include order-based crossover, task-based crossover, and two-point crossover. A Q-learning-driven variable neighborhood search module is used to use ten neighborhood structures as the action space for Q-learning, with neighborhood structure numbers as states, based on multi-objective optimization... The algorithm designs a reward function based on the optimization results, trains the operator selection probability matrix through a Q-table update mechanism, and adaptively selects a neighborhood operator to search the offspring population. An iterative screening module merges the parent population with the new solution set output by the Q-learning-driven variable neighborhood search module, filters the next generation population through non-dominated sorting and crowding distance functions, and repeatedly triggers the multi-objective evolutionary optimization module and the Q-learning-driven variable neighborhood search module until the termination condition is met. A scheduling scheme output module outputs a Pareto-optimal train scheduling scheme set, which includes the train scheduling order, the number of vehicles, and the running time for each section.

[0014] The working principle of this invention: In subway train scheduling, the flexibility of vehicles plays an important role and has become the focus of more and more research. In this invention, the train scheduling problem of urban rail transit network is studied and modeled as a special job shop scheduling problem with congestion constraints. This invention considers three evaluation indicators: minimizing train delay penalty, passenger dissatisfaction and the total number of running vehicles. Based on these constraints and optimization objectives, this paper proposes a detailed mathematical model of the problem. Subsequently, a multi-objective evolutionary optimization algorithm based on Q learning is designed and combined with the variable neighborhood search (VNS) strategy for solving. In order to solve the congestion constraint problem, a lemma for a specific problem is proposed to design an efficient decoding heuristic algorithm. Next, in order to fully explore the search space, eight variable neighborhood search operators are designed based on critical path theory. In addition, an integrated Q learning method is used to train the operator selection probability matrix to achieve adaptive operator selection: the specific scheme includes: (1) Considering the actual constraints such as dynamic grouping and continuous locomotive operation, a mixed integer programming mathematical model based on the job shop scheduling problem with congestion constraints is established. The model is implemented using the CPLEX solver to provide feasible solutions for small-scale optimization problems. (2) To design an efficient multi-objective optimization algorithm, a two-layer vector coding mechanism including scheduling and vehicle allocation is proposed. Based on the consideration of the congestion constraints of subway locomotives, a lemma related to congestion is proposed and proved. Based on this lemma, a multi-block left insertion heuristic rule is innovatively designed, which can effectively improve the utilization rate of subway tracks and significantly improve the quality of the algorithm solution under the premise of satisfying the congestion constraints. (3) In order to fully explore the search space, eight variable neighborhood search operators are designed based on the critical path theory for train scheduling and vehicle allocation in the two-layer coding space. The Q-learning method is used to train the operator selection probability matrix, thereby effectively improving the selection performance of the neighborhood search operator and enhancing the optimization capability of the algorithm. (4) In order to further enhance the global search capability of the algorithm, based on the results of the multi-objective optimization algorithm, an improved Q-learning action space, reward value calculation and Q table update mechanism are designed. At the same time, combined with the characteristics of the problem, three effective crossover operators are designed. (5) To verify the efficiency and effectiveness of the proposed algorithm, three sets of test cases were conducted. The first set of small-scale test cases was used to compare with CPLEX to verify the effectiveness of the mathematical model. The second set of test cases was used to verify the solution efficiency of the algorithm in medium-sized instances. The third set of test cases was conducted based on real data from the Shenzhen Metro system. To ensure a fair comparison, six state-of-the-art multi-objective optimization algorithms were selected as control algorithms.The core logic of this invention is: "Mathematical model defines the boundary → Encoding and decoding build the bridge → Evolutionary algorithm expands the search → Q-learning improves accuracy → Iterative selection finds the optimal solution": Mathematical model ensures that the solution conforms to actual constraints, encoding and decoding achieve the adaptation of the problem and the algorithm, evolutionary algorithm enables extensive exploration of multi-objective solutions, Q-learning improves the search targeting and efficiency, and finally, under the premise of "satisfying blocking constraints and operational rules", it achieves the synergistic optimization of three conflicting objectives, solving the core pain points of low efficiency in solving large-scale scheduling problems and poor balance of multi-objectives.

[0015] Compared with existing technologies, and the beneficial effects:

[0016] 1. Significantly improved efficiency in solving large-scale scheduling problems.

[0017] The proposed EQVNS algorithm addresses the pain point of exponentially increasing computation time in large-scale scenarios for traditional deterministic optimization methods (such as branch and bound). In a medium-sized case study with 58 stations and 80 trains, the proposed EQVNS algorithm improves the HV value and reduces the IGD value compared to algorithms such as QVNS-NSGA-II. In real-world cases of the Shenzhen Metro (50-150 trains), the HV and IGD indicators of 10 cases are comprehensively superior to 6 mainstream multi-objective optimization algorithms, demonstrating its efficient adaptability to the real-time scheduling requirements of complex rail transit networks.

[0018] 2. Multi-objective optimization synergy and optimal solution quality

[0019] Achieving a precise balance between the three conflicting objectives of minimizing train delay penalties, passenger dissatisfaction, and the total number of operating vehicles, the algorithm exhibits a more uniform Pareto front distribution and stronger convergence. Experiments show that in small-scale cases, the algorithm outperforms CPLEX in 10 out of 12 instances, and reduces the IGD value. In medium-scale cases, the number of times train capacity is insufficient is reduced, the total delay value is lowered, and the total number of operating vehicles is optimized, effectively resolving the contradiction between increasing trains to reduce dissatisfaction and delays to increase capacity.

[0020] 3. Improved adaptability to congestion constraints and enhanced track utilization

[0021] By employing a multi-block left-insertion heuristic decoding rule, and strictly adhering to the constraint that "only one train is allowed to pass through the same section at the same time," invalid idle time between sections is eliminated. Compared to the "first-come, first-served" decoding method, the algorithm improves HV value and track utilization, reduces IGD value, and avoids capacity waste caused by train congestion.

[0022] 4. Enhanced search intelligence and algorithm robustness

[0023] The Q-learning-driven adaptive operator selection mechanism improves the targeting of neighborhood search. Compared with the traditional VNS method, the average IGD of the algorithm is reduced, and better solutions are obtained in 11 out of 15 medium-sized cases. By combining ensemble population tournament selection and customized crossover operators, the algorithm maintains stable performance under different route layouts and passenger flow fluctuation scenarios. The results of multifactor ANOVA verification are significant.

[0024] 5. Excellent practical operational adaptability and outstanding engineering value.

[0025] Adaptable to dynamic train formation and uneven spatial and temporal distribution of passenger flow, the train can dynamically adjust the number of carriages and scheduling order according to changes in passenger flow in real data tests on Shenzhen Metro Lines 4 and 5 (including busy transfer stations such as Shenzhen North Station). This reduces the average waiting time for passengers, improves the on-time rate of trains, and reduces the investment cost of operating vehicles, demonstrating its direct engineering implementation capability. Attached Figure Description

[0026] Figure 1 This is a schematic diagram of the layout of six sections of a simple urban rail transit network in the mathematical modeling problem description of Example 1;

[0027] Figure 2 This is a schematic diagram of the encoding method for the first dimension vector used in train scheduling in Example 1;

[0028] Figure 3 This is a schematic diagram of the Gantt chart for the left insertion heuristic in Example 1;

[0029] Figure 4 This is a schematic diagram of the Q-table update OX-I crossover operator in Example 1;

[0030] Figure 5 This is a schematic diagram of the Q-table update JBX-I crossover operator in Example 1;

[0031] Figure 6 This is a schematic diagram comparing the ANOVA of EQVNS and CPLEX in Example 2;

[0032] Figure 7 This is a schematic diagram showing the ANOVA comparison results of the EQVNS-NQ and EQVNS algorithms in Example 2;

[0033] Figure 8 This is a schematic diagram showing the ANOVA comparison results between the EQVNS algorithm and EQVNS-NV without the VNS part in Example 2;

[0034] Figure 9 This is a schematic diagram showing the comparison results of ANOVA (analysis of variance) for the two methods in Example 2;

[0035] Figure 10 This is a schematic diagram showing the ANOVA comparison results of the seven multi-objective optimization algorithms in Example 2;

[0036] Figure 11 This is a Gantt chart of one of the optimal solutions for example "52-58" in Example 2;

[0037] Figure 12 This is a network layout diagram of Shenzhen Metro Lines 4 and 5 in Example 2;

[0038] Figure 13 This is a schematic diagram of the ANOVA comparison results of six multi-objective algorithms (including CMOEACD, DSPCMDE, DAEA, ARMOEA, HpaEA, and NSGA-III) in Example 2.

[0039] Figure 14 This is a comparison plot of the Pareto front for example "50-1" in Example 2. Detailed Implementation

[0040] The present invention will now be described in further detail with reference to specific embodiments and accompanying drawings.

[0041] Example 1: A Multi-Objective Train Scheduling Method Integrating Q-Learning

[0042] Step 1: Modeling

[0043] 1. Mathematical Modeling

[0044] 1.1 Problem Description

[0045] In this study, we consider an urban rail transit network comprising a set of train lines. Each line consists of several stations and track sections. Each train is assigned to a known line and must pass through all the stations on that line in sequence. A set of depots (or rolling stock) is set up in the network for train departures and arrivals. Each train can select a certain number of vehicles, which is an optimization variable used to improve the system's flexibility.

[0046] Figure 1 illustrates the layout of a simple urban rail transit network comprising four stations and two depots. To model the problem, we divide the network into six segments, which form connections between depots and stations, and between stations themselves. For each segment, for safety reasons, only one train can pass through at any given time. Therefore, there is a mutual exclusion constraint between any pair of trains within each segment. Furthermore, each train must not interrupt its operation while passing through any segment, except for the time spent picking up and dropping off passengers.

[0047] 1.2 Indicators and Parameters

[0048] (1) Indicators

[0049] Table 1 Key Indicators

[0050]

[0051] (2) Parameters

[0052] Table 2 Key Parameters

[0053]

[0054] (3) Auxiliary decision variables

[0055] Table 3 Variables for Assisting Decision Making

[0056]

[0057] (4) Decision variables

[0058] Table 4 Decision Variables

[0059]

[0060] 1.3 Mathematical Model

[0061] (1) Objective function of the mathematical model

[0062] The optimization objective of the problem under study is shown in equation (1), where the first objective function is... The second objective function is used to calculate the total number of operations performed by the train when its carrying capacity is insufficient. Used to calculate the total train delay; the third objective function Used to calculate the total number of vehicles used.

[0063] These three objectives are usually in conflict. For example, when the number of vehicles... When there are more passengers, the train can carry more passengers, thus enabling... The value decreases; while delaying departure allows trains to carry more passengers, but this results in a delay penalty.

[0064] Therefore, the optimization objective of this study is:

[0065] (1)

[0066] (2)

[0067] (2) Job Shop Scheduling (JSP) Constraints with Blocking Constraints

[0068] To avoid two trains being too close together, constraints (3) and (4) ensure that when two trains enter the same section, the latter train must wait for an additional period of time after the former train enters its next section. Constraint (5) guarantees the congestion condition between two adjacent sections for each train, that is, each train should enter the next section immediately after completing the previous section. Constraints (6) and (7) ensure that two trains arriving at the same section should have a unique passing order and should not overlap. Constraint (8) is used to calculate the completion time of each train in each section. Constraint (9) ensures that all variables are positive values.

[0069]

[0070] (3) Section traffic priority constraints

[0071] Constraint (10) ensures that a train is assigned a unique passage priority when it passes through a pair of sections. Constraint (11) ensures that at most one train can be assigned a passage priority in each pair of sections. That is, no two trains are allowed to overlap on the same passage priority. Constraint (12) ensures that there should be no gaps in the passage priority allocation for each pair of sections, i.e., passage priorities should be allocated consecutively from left to right. Constraint (13) ensures that the start time of the next priority should be greater than the completion time of the previous consecutive priority, i.e., the processing between any two adjacent passage priorities is not allowed to overlap. Constraint (14) is used to calculate the relationship between each train and its corresponding passage priority in each section. Constraint (15) ensures that all variables are positive values.

[0072]

[0073] (4) Arrival time constraints for each train

[0074] Constraint (16) ensures that each train arrives at its assigned section within a single time period for each operation. Constraint (17) is used to calculate the arrival time period for each train operation. Constraint (18) ensures that each train can only arrive at any section within a single time period. Constraint (19) is used to establish the relationship between two decision variables.

[0075]

[0076] (5) Passenger boarding and alighting constraints

[0077] Constraints (20) and (21) are used to calculate the number of passengers boarding when the first train arrives and simultaneously serves sections u and v. Constraints (22) and (23) are used to calculate the number of passengers boarding when subsequent trains arrive in the following sequence.

[0078] Constraint (24) ensures that each train should be assigned a unique train size or number of cars. Constraint (25) ensures that if a train consists of n cars, its initial passenger capacity should be expanded to n times. Constraint (26) is used to calculate the passenger capacity of train k when it first arrives at segment u. Constraint (27) is used to calculate the passenger capacity of train k in subsequent segments. Constraint (28) is used to calculate the total number of passengers waiting to board at segment u who wish to take train k. Constraints (29) to (30) are used to calculate the number of passengers disembarking at segment u. Constraints (31) to (34) ensure that all variables are positive.

[0079]

[0080] (6) Train delay constraints

[0081] Equation (35) is used to calculate the delay value of train k. The calculation method is as follows: first, find the maximum value between the completion time of train k and the planned arrival time (deadline). Then, the difference between the maximum value and the planned arrival time must be a non-negative number.

[0082] (35)

[0083] Step 2: Encoding

[0084] To simultaneously accomplish the tasks of train scheduling and vehicle allocation, we introduce a two-dimensional vector as the encoding representation. Consider an example involving 7 subway trains and 6 sections. Assume the system has two depots, labeled "d1" and "d2," from which trains depart and terminate. Assume there are four types of routes: {1, 2, 3}, {1, 2, 3, 4, 5, 6}, {4, 5, 6}, and {4, 5, 6, 1, 2, 3}. Let the routes chosen by the 7 trains be {1, 2, 1, 1, 1, 4, 2}, meaning the first train chooses the first route {1, 2, 3}, and the seventh train chooses the second route {1, 2, 3, 4, 5, 6}.

[0085] The first dimension vector is used for train scheduling. For example, in Figure 2 In (a), the first train to be dispatched is train number 4, while the last train waiting to be assembled is train number 6. The second dimension vector is used for vehicle assignment. Figure 2 (b) It can be seen that the first train is assigned 1 vehicle, while the seventh train is assigned 4 vehicles.

[0086] Step 3: Decoding

[0087] (1) Lemma for a specific problem

[0088] The decoding process for urban train formation scheduling is crucial. To generate feasible and efficient solutions, this paper proposes a lemma considering congestion constraints.

[0089] Lemma 1: Let For the j-th operation of train k, and Assignments The start time and the finish time, Let be the available time for segment u. Define the time interval range of the i-th blocking segment u. .like ,but:

[0090] (36)

[0091] Therefore, the length of the left insertion is equal to .

[0092] Proof: This condition ensures that only when The left insertion heuristic is executed only when the end time of train k is later than its deadline. If the end time is earlier than the deadline, the left insertion heuristic for that train is ignored. Ensure that free blocks can meet the operation requirements. The processing requirements. Then, if Since it's the first operation, i.e., j=0, we can shift this operation to the left. One unit; otherwise, the movement length should be considered as the minimum of the two values, i.e., the current movement length and . The length of is set because the blocking constraint must be satisfied. Therefore, the lemma is proved.

[0093] (2) Left insertion decoding heuristic method

[0094] Based on the relevant lemma, the main steps of the decoding heuristic using the left insertion method can be described as follows: First, calculate the possible start and finish times for each train on its assigned segments. Next, adjust each operation according to the blocking constraints, i.e., fix the last operation of each train and adjust the start times of its preceding operations from right to left to eliminate idle time between them. Then, re-mark all idle blocks on each segment and find possible left insertion positions for each train on each segment.

[0095] The Gantt charts for the solutions with and without the left insertion heuristic are shown in Figure 3(a) and (b), respectively. From the Gantt charts, we can observe that: (1) in both (a) and (b), the decoding heuristic yields feasible solutions; (2) the Gantt chart for the solution with the left insertion heuristic achieves better results in terms of maximum completion time. Therefore, with the left insertion heuristic, all trains can reach their destination as quickly as possible; (3) the main difference between the two Gantt charts is that the first three operations of train T6 are adjusted to be before T2 and T7 without violating the congestion constraint.

[0096] Step 4: Initialization Method

[0097] The encoding consists of two parts, and therefore the initialization also consists of two parts.

[0098] (1) Initialization of the scheduling section

[0099] The NEH heuristic is an efficient heuristic algorithm used to generate an initial solution to the scheduling problem. In this embodiment, the NEH heuristic is embedded to generate an efficient solution, while the remaining solutions are generated randomly to cover a larger search space. Detailed steps of the scheduling initialization heuristic are given in lines 1–12 of the above calculation formulas.

[0100] (2) Initialization of the vehicle section

[0101] For the initialization of the vehicle part, a simple random heuristic method is used, the description of which can be found in lines 13–17 of the above calculation formula.

[0102] (3) Initialization of the integrated subpopulation

[0103] After obtaining the initial population, a non-dominated sorting function is applied to it. High-quality individuals are selected and stored in the ensemble subpopulation according to the code in lines 20–30 of the above calculation formula.

[0104] Step 5: Multi-objective processing method

[0105] In this section, we propose an NSGA-II method based on integrated populations, named ENSGA-II.

[0106] (1) Tournament selection method based on ensemble population

[0107] In the classic NSGA-II, selecting suitable individuals for the mating pool to complete the crossover operation is crucial. Traditional tournament selection processes typically select individuals based on the current population. However, the high-quality individuals with diversity and convergence found to date often possess the ability to guide the search into the potential prime space. Therefore, we propose a tournament selection method based on an integrated population, as described below.

[0108] Step 1. Let MP represent the mating pool. When Then execute steps 2 through 5.

[0109] Step 2. Perform the standard tournament selection method to select two individuals. and .

[0110] Step 3. Randomly select an individual. .

[0111] Step 4. From , and Two individuals are randomly selected from the pool and inserted into the MP.

[0112] After population initialization is completed, the NSGA-II process is executed first to generate a better solution population.

[0113] Step 1. Calculate the fitness value for each solution in the current population.

[0114] Step 2. Perform a non-dominated sorting function on the current population to obtain the Pareto front.

[0115] Step 3. Perform a tournament selection heuristic based on the ensemble population on the current population to obtain the selection index.

[0116] Step 4. Apply a sequence-based crossover operation based on the selected index to obtain the child solution set.

[0117] Step 5. Perform a swap mutation operation on each child solution.

[0118] Step 6. Merge the current population with the offspring population, and execute the non-dominated sorting function on the merged population.

[0119] Step 7. Select the next generation of population based on the Pareto front and the crowding distance function.

[0120] Step Six: Variable Neighborhood Structure Components

[0121] To search for more promising solution spaces, we designed ten neighborhood structures, as described below.

[0122] (1) NS-I: Switching Dispatch – Switching Vehicles

[0123] For the scheduling and vehicle components: First, two cut points are randomly generated. and Then through exchange and The two elements in the solution yield new child solutions.

[0124] (2) NS-II: Insertion Scheduling – Insertion Vehicle

[0125] For the scheduling and vehicle components: First, two cut points are randomly generated. and ,in Then Inserting the element at the position Before that, a new child solution is obtained.

[0126] (3) NS-III: Reverse Dispatch – Vehicle Swap

[0127] For the scheduling part: First, two cut points are randomly generated. and ,in Then reverse and The elements between them yield new child solutions. The vehicle part performs the same swap operation as NS-I.

[0128] (4) NS-IV: Exchange Scheduling – Insert Vehicle

[0129] The scheduling section performs the NS-I exchange operation; the vehicle section performs the NS-II insertion operation.

[0130] (5) NS-V: Insertion Dispatch – Vehicle Swap

[0131] The scheduling section performs NS-II insertion operations; the vehicle section performs NS-I exchange operations.

[0132] (6) NS-VI: Reverse Scheduling – Insert Vehicle

[0133] The scheduling section performs the NS-III reversal operation; the vehicle section performs the NS-II insertion operation.

[0134] (7) NS-VII: Exchange Scheduling – Random Vehicles

[0135] The scheduling section performs the NS-I exchange operation; the vehicle section randomly selects a train and assigns it a random number of vehicles.

[0136] (8) NS-VIII: Critical Switching Scheduling – Switching Vehicles

[0137] For the scheduling part, two jobs are randomly selected from the critical job set and the non-critical job set, respectively. and Then to and Perform the swap operator. For the vehicle part, perform the same swap operator operation as NS-I.

[0138] (9) NS-IX: Critical Switch - Scheduling Insertion - Vehicle

[0139] The neighborhood structure of the scheduling part is similar to that of NS-VIII. For the vehicle part, the same insertion operator is executed as in NS-II.

[0140] (10) NS-X: Critical Switching Scheduling – Random Vehicles

[0141] The neighborhood structure of the scheduling section is similar to that of NS-VIII. For the vehicle section, the same random operations are performed as in NS-VII.

[0142] Step 7: Q-learning component

[0143] The Q-learning method is used to select appropriate search parameters or operators for the VNS method; that is, the neighborhood search operator plays a crucial role in the VNS process. The key components of the Q-learning method are as follows.

[0144] state

[0145] The state represents the current feature of the algorithm. Here we use the structure number of the VNS neighborhood to represent the state.

[0146] (2) Action space

[0147] The action space includes ten neighborhood structures from NS-I to NS-X.

[0148] (3) Reward value To adapt to multi-objective optimization problems, the reward function in this study is calculated as shown in the following formula.

[0149] (37)

[0150] in, and Let represent the initial solution and the neighborhood solution at time t, respectively; and This represents the maximum and minimum values ​​found so far in the k-th target; This represents the k-th target value.

[0151] When the obtained solution is better (i.e. When the reward value is lower, a higher reward value can be obtained. The value of is used to limit the reward value within a certain range, thereby ensuring the stability of the training process.

[0152] (3) Q table update

[0153] The update process for table Q is as follows.

[0154] Step 1. Obtain the current state and actions Corresponding current value .

[0155] Step 2. Calculate the value ,in This represents the maximum value in the i-th row of table Q.

[0156] Step 3. Calculate the updated value And use it to update the corresponding position in the Q table.

[0157] Step 8: Crossover Operator

[0158] For the scheduling part, applications such as Figure 4 , Figure 5 The three cross operators are shown.

[0159] (1) Sequence-based crossover operator

[0160] The order-based crossover operator (denoted as OX-I) used for the scheduling part is described below.

[0161] Step 1. Randomly generate a cutting length. and two parent individuals and Two cutting points and .

[0162] Step 2. Transfer the elements and Two offspring individuals were placed separately. and middle.

[0163] Step 3. Fill the remaining elements into the two child individuals in turn. and middle.

[0164] (2) Job-based crossover operator

[0165] The job-based crossover operator (denoted as JBX-I) used for the scheduling part is described below.

[0166] Step 1. Randomly arrange all jobs and divide them into two sets. and .

[0167] Step 2. [The text appears to be incomplete and contains several grammatical errors. A more accurate translation would require the full and All elements from the parent individual and Offspring individuals copied to the corresponding location and middle.

[0168] Step 3. Fill the remaining elements into the two child individuals in turn. and middle.

[0169] (3) Crossover operator based on two points

[0170] For the vehicle allocation part, a simple two-point crossover operator is used.

[0171] (4) The chronological or causal order of the steps;

[0172]

[0173] Example 2

[0174] This embodiment describes the detailed experimental setup, experimental comparisons, and analysis.

[0175] Experimental setup

[0176] The detailed experimental environment for the proposed algorithm is as follows: the programming language is Python, the algorithm runs on an Ubuntu 20.04 LTS 64-bit system, configured with a 16-core vCPU (@2.60 GHz) and an NVIDIA GeForce RTX4090 (24GB) graphics card.

[0177] The following three types of algorithms were selected as comparison algorithms: (1) CPLEX was used to verify the solution efficiency of the proposed mathematical model on a small set of instances; (2) the multi-objective optimization algorithm based on Q-learning proposed by Li et al. (2023); (3) a set of multi-objective optimization algorithms, including NSGAIII (Deb & Jain, 2014), ARMOEA (Tian et al., 2018), HpaEA (Chen et al., 2020), DAEA (Xu et al., 2021), DSPCMDE (Yu et al., 2022) and CMOEACD (Liu, Han, et al., 2025).

[0178] To verify the efficiency and effectiveness under multi-objective optimization, two commonly used metrics were selected for comparison: hypervolume (HV) (While et al., 2012) and inverse intergenerational distance (IGD) (Sun et al., 2019; Ishibuchi et al., 2015). Furthermore, the relative percentage index (RPI) was used to calculate performance comparisons. Used for performance comparison; among which This represents the HV or IGD value obtained by the comparison algorithm, while This represents the best HV or IGD value obtained from all comparison algorithms.

[0179] The algorithm parameters are set as follows: (1) Mutation rate Set to 0.5; (2) Crossover rate Set to 0.5; (3) The parameters of VNS are learned through the Q-learning component.

[0180] Experimental Examples

[0181] The first set of experimental examples was used to verify the solution efficiency of the proposed mathematical model. Ten sets of examples of different sizes were randomly generated. The number of trains ranged from 3 to 7, and the number of stations was 6. Figure 1 shows the network layout of the small-scale examples.

[0182] The second set of experimental examples (Chai et al., 2024) includes 15 sets of experimental examples of different sizes, comprising 58 stations with train numbers ranging from {40, 52, 60, 72, 80}. Its network layout is shown in the appendix.

[0183] The third set of experimental cases comes from the Shenzhen Metro system in Guangdong Province, China. We selected passenger data from Shenzhen Metro Lines 4 and 5 on January 20, 2020. The main reason for choosing these two lines is that Shenzhen North Station (SZBZ) is one of the busiest stations and can effectively represent the traffic conditions of the Shenzhen Metro system.

[0184] Comparison with CPLEX

[0185] To verify the effectiveness of the mathematical model proposed in Section III, we compare it with the proposed EQVNS algorithm and the mathematical model implemented in CPLEX. Based on the network layout shown in Figure 1, 12 different small-scale instances were randomly generated. Since CPLEX can only solve weighted objective optimization problems, to generate a set of Pareto solutions, we run the mathematical model three times to optimize the three objectives separately. The solution sets obtained from each run were aggregated to generate the Pareto results. The comparison results with CPLEX and EQVNS are shown in Table 1.

[0186] Table I. Comparison of HV values ​​between CPLEX and EQVNS

[0187]

[0188] From Table 1, we can observe that: (1) In the comparison of HV values, the proposed EQVNS algorithm outperforms CPLEX in 10 out of 12 small-scale instances; (2) From the comparison results of IGD, EQVNS also shows strong competitiveness, only slightly inferior in one instance; (3) Combining the comparison results of HV and IGD, we can verify the efficiency and convergence performance of the proposed algorithm.

[0189] In addition, multivariate analysis of variance (ANOVA) was used to assess the significance of the comparison results.

[0190] The table above shows the ANOVA comparison results of EQVNS and CPLEX, demonstrating that the proposed EQVNS algorithm has significant performance advantages and verifying the effectiveness of the established mathematical model.

[0191] Multivariate analysis of variance (ANOVA) was also used to assess the significance of the comparison results. Figure 6 shows the ANOVA comparison results between EQVNS and CPLEX, demonstrating that the proposed EQVNS algorithm has significant performance. The proposed mathematical model was thus validated.

[0192] Table 2. Comparison of IGD values ​​between CPLEX and EQVNS

[0193]

[0194] Efficiency ablation experiments based on Q-learning heuristics

[0195] To verify the effectiveness of the proposed Q-learning driven component, we compared the proposed EQVNS algorithm with EQVNS-NQ, where EQVNS-NQ uses the traditional VNS method, i.e., it does not include the Q-learning driven part.

[0196] Tables 3 and 4 show the comparison results for HV and IGD metrics. The results show that: (1) In terms of HV value, the EQVNS algorithm achieved better results in 10 out of 15 instances, and the average performance in the last row of Table III further verifies the robustness of the algorithm; (2) In terms of IGD value (see Table IV), the EQVNS algorithm with Q-learning component achieved better results in 11 instances, with an average value of 0.2289, which is more than three times lower than EQVNS-NQ, further verifying the efficiency of the Q-learning part.

[0197] Figure 7 shows the ANOVA comparison results of the EQVNS-NQ and EQVNS algorithms, further illustrating the competitive performance of the algorithms after introducing the Q-learning heuristic. (a) ANOVA comparison of HV values ​​in the Q-learning part; (b) ANOVA comparison of IGD values ​​in the Q-learning part.

[0198] Table 3. Comparison of IGD values ​​between EQVNS-NQ and EQVNS

[0199]

[0200] Table 4. Comparison of IGD values ​​between EQVNS-NQ and EQVNS

[0201]

[0202] Efficiency ablation experiments based on VNS components: To verify the effectiveness of the proposed VNS components, we compared the EQVNS algorithm with EQVNS-NV, which does not include the VNS component.

[0203] Tables 5 and 6 show the detailed comparison results of the two methods, and Figure 8 shows their ANOVA comparisons: (a) ANOVA comparison of HV values; (b) ANOVA comparison of IGD values. From the results, we can observe that: (1) In the comparison of HV values, after using the VNS component, the proposed algorithm obtained 11 better results in 15 instances, which is significantly better than EQVNS-NV; (2) In terms of IGD values, the proposed algorithm also obtained 11 better results; (3) The ANOVA comparison results of the two sets further verify the competitive performance of the algorithm after introducing the VNS component.

[0204] Table 5. Comparison of HV values ​​between EQVNS-NV and EQVNS

[0205]

[0206] Table 6. Comparison of IGD values ​​between EQVNS-NV and EQVNS

[0207]

[0208] Ablation study on decoding efficiency

[0209] In Section IV-C, a corresponding decoding heuristic was developed for blocking constraints. To further verify the effectiveness of this decoding heuristic, we compared two methods: EQVNS using the proposed decoding method and EQVNS-ND using a "first-come, first-served" decoding method.

[0210] Tables 7 and 8 present detailed comparison results of the two methods, while Figure 9 shows the ANOVA (Analysis of Variance) comparison results of the two methods: (a) ANOVA comparison of HV values ​​of the VNS heuristic algorithm; (b) ANOVA comparison of IGD values ​​of the VNS heuristic algorithm. It is clear from the two tables and Figure 9 that the performance of the algorithm is significantly improved after using the proposed decoding heuristic method.

[0211] Table 7. Comparison of HV values ​​between EQVNS and EQVNS-ND

[0212]

[0213] Table 8. Comparison of IGD values ​​between EQVNS and EQVNS-ND

[0214]

[0215] Comparison with efficient algorithms

[0216] (1) Comparison of the second set of examples

[0217] To test the performance of the proposed algorithm on the second set of examples, we selected two types of algorithms: the first type is the Q-learning-based multi-objective algorithm QVNS-NSGA-II; the second type consists of six multi-objective optimization algorithms: NSGAIII (Deb & Jain, 2014), ARMOEA (Tian et al., 2018), HpaEA (Chen et al., 2020), DAEA (Xu et al., 2021), DSPCMDE (Yu et al., 2022), and CMOEACD (Liu, Han, et al., 2025).

[0218] Tables 9 and 10 report the comparison results of HV and IGD values ​​between EQVNS and QVNS-NSGA-II, respectively. It is clear from the two tables that the proposed EQVNS algorithm can solve the studied problem more efficiently than the efficient QVNS-NSGA-II algorithm.

[0219] Tables 11 and 12 report the comparison results of seven multi-objective optimization algorithms in terms of HV and IGD values, respectively. From the results, we can observe that: (1) In the comparison of HV values, the proposed EQVNS algorithm achieved 14 better results in 15 cases, while ARMOEA achieved 1 better result; (2) In the comparison of IGD values, the proposed algorithm achieved 13 better results; (3) The average performance of HV and IGD further demonstrates that the proposed algorithm is superior in terms of efficiency and solution diversity.

[0220] Figure 10 shows the ANOVA comparison results of seven multi-objective optimization algorithms, further validating the significant performance of the proposed algorithm.

[0221] Figure 11 shows a Gantt chart of one of the optimal solutions for example “52-58”, and the results verify the effectiveness of the proposed algorithm.

[0222] Table 9. Comparison of HV values ​​between EQVNS and QVNS-NSGA-II in the second set of examples.

[0223]

[0224] Table 10. Comparison of IGD values ​​between EQVNS and QVNS-NSGA-II in the second set of examples.

[0225]

[0226] (2) Comparison based on actual subway system examples

[0227] To verify the performance of the proposed algorithm in a real-world metro train scheduling problem, we selected two Shenzhen Metro lines: Line 4 and Line 5. These two lines were chosen because Shenzhen North Station (SZBZ) is one of the busiest stations in Shenzhen, and these two lines are representative in terms of passenger flow characteristics.

[0228] Figure 12 shows the network layout of two metro lines, each including both northbound and southbound directions. Shenzhen North Station is a transfer station and also the busiest station. There are a total of 40 stations, and passenger numbers are based on real data from the Shenzhen Metro system. Based on this metro network, we randomly selected 10 simulations of different sizes, with the number of trains ranging from 50 to 150, each train randomly choosing one of four operating directions.

[0229] Tables 13 and 14 report the comparison results of HV and IGD values, respectively. From the two tables, it can be observed that: (1) in 10 real-world examples, the proposed algorithm achieved 14 better results in both HV and IGD; (2) the average performance in the last row of the tables further verifies the efficiency of EQVNS.

[0230] Figure 13 shows the ANOVA comparison results of seven multi-objective algorithms, and Figure 14 shows the Pareto front comparison of the examples. Both figures clearly demonstrate the efficiency and effectiveness of the proposed algorithm.

[0231] Table 11. Comparison of HV values ​​of the seven algorithms on the second set of examples (A1–A7 represent CMOEACD, DSPCMDE, DAEA, ARMOEA, HpaEA, NSGAIII, and EQVNS, respectively).

[0232]

[0233] Table 12. Comparison of IGD values ​​of the seven algorithms on the second set of examples (A1–A7 represent CMOEACD, DSPCMDE, DAEA, ARMOEA, HpaEA, NSGAIII, and EQVNS, respectively).

[0234]

[0235] Table 13. Comparison of HV values ​​of seven algorithms on real-world examples (A1–A7 represent CMOEACD, DSPCMDE, DAEA, ARMOEA, HpaEA, NSGAIII, and EQVNS, respectively).

[0236]

[0237] Table 14. Comparison of IGD values ​​of seven algorithms on real-world examples (A1–A7 represent CMOEACD, DSPCMDE, DAEA, ARMOEA, HpaEA, NSGAIII, and EQVNS, respectively).

[0238]

[0239] The above examples illustrate the present invention only to aid in understanding it and are not intended to limit the scope of the invention. Those skilled in the art can make various simple deductions, modifications, or substitutions based on the principles of this invention.

Claims

1. A multi-objective train scheduling method integrating Q-learning, characterized in that... include: Considering the practical constraints of dynamic formation and continuous locomotive operation, the train scheduling problem is transformed into a work-shop scheduling problem with congestion constraints, and the constraints are clarified. Based on the number of times train capacity is insufficient, the total delay value, and the total number of vehicles, optimization indicators are defined. By using key parameters including section passage priority, train running time, and passenger arrival volume, and decision variables including scheduling order, number of vehicles, and arrival time, a mixed integer programming mathematical model for the job shop scheduling problem with congestion constraints is constructed. This model can obtain feasible solutions for small-scale problems through the CPLEX solver. A two-layer vector coding mechanism is established, which includes scheduling order and vehicle allocation. The first dimension represents the train scheduling order, and the second dimension defines the number of vehicles allocated, so as to achieve synchronous optimization of scheduling and vehicle configuration. We propose and prove a blocking-related lemma. Based on this lemma, we design a multi-block left insertion decoding strategy to compress train idle time and improve track utilization and solution quality while satisfying blocking constraints. A multi-objective processing framework is established. Based on the NSGA-II algorithm, an integrated population tournament selection mechanism is introduced, which integrates historical high-quality solutions to guide the search direction. Population evolution is achieved through three crossover operators: sequential crossover, job crossover, and double-point crossover. Based on critical path theory, various neighborhood structures are designed to cover the two-layer coding space of scheduling and vehicle allocation. A variable neighborhood search structure is established and used as the action space. The reward function is designed based on the multi-objective optimization results. The variable neighborhood search is driven by the Q-learning method. The Q-table update mechanism trains the operator selection probability matrix to realize the adaptive selection of neighborhood search operators and improve the algorithm optimization efficiency. Merge the parent and offspring populations, select the next generation population using non-dominated sorting and crowding distance function, repeat the steps of multi-objective evolutionary optimization and variable neighborhood search until the termination condition is met, and output the Pareto optimal train scheduling scheme set.

2. The multi-objective train scheduling method according to claim 1, characterized in that, The construction of the mixed integer programming mathematical model also includes minimizing train delay penalties, passenger dissatisfaction, and the total number of operating vehicles as optimization objectives. The model includes congestion constraints, section passage priority constraints, train arrival time constraints, passenger boarding and alighting constraints, and train delay constraints. The congestion constraints limit that only one train is allowed to pass through the same section at the same time and that train operation cannot be interrupted.

3. The multi-objective train scheduling method according to claim 2, characterized in that, The constraints specifically include: Congestion constraint: The following train must wait for a minimum safe interval after the preceding train enters the next section, and the train must enter the next section immediately after completing the work in the previous section; Section passage priority constraint: A unique passage priority is assigned to each section that a train passes through, and the same priority is assigned to only one train and there is no overlap between consecutive priority sections. Arrival time constraints for each train: Each train operation and arrival time in each section corresponds to a unique time period; Passenger boarding and alighting constraints: The passenger capacity of a train shall not exceed the product of the number of carriages and the capacity of a single carriage. The number of passengers boarding shall be calculated separately for the first train and subsequent trains. Train delay constraint: The delay value is the difference between the actual completion time and the scheduled arrival time of the train, and the delay value is non-negative.

4. The multi-objective train scheduling method according to claim 1, characterized in that, The lemma related to the blocking constraint is as follows: Let the j-th operation of train k be... , and Assignments The start time and finish time, a u The available time for segment u, Define the time interval range of the i-th blocking segment u. ,like ,but ； Then the left insertion length .

5. The multi-objective train scheduling method according to claim 1, characterized in that, The multi-block left insertion decoding strategy includes: Calculate the possible start and finish times for each train on its assigned segments, adjust each operation according to the blocking constraints, fix the last operation of each train, and adjust the start times of its preceding operations from right to left to eliminate idle time between them. Remark all idle blocks on each segment and find possible left insertion positions for each train on each segment. The dual-layer vector encoding mechanism also includes encoding initialization, using the NEH heuristic algorithm to generate initial solutions for the scheduling part, using a random heuristic method to generate initial solutions for the vehicle allocation part, and performing non-dominated sorting on the initial population to select high-quality individuals to form an integrated subpopulation.

6. The multi-objective train scheduling method according to claim 1, characterized in that, The specific implementation of Q-learning includes: State definition: The current state of the algorithm is represented by the neighborhood structure numbering of the variable neighborhood search. Action space: includes ten neighborhood structures, which cover the exchange, insertion, reversal and critical operation operations of scheduling and vehicle allocation; Reward function: The reward value is calculated based on the difference between the objective function values ​​of the current solution and the neighboring solutions, thus introducing a reward function. ； in, and Let represent the initial solution and the neighborhood solution at time t, respectively; and This represents the maximum and minimum values ​​found so far in the k-th target; This represents the k-th target value; when When the value is smaller, a higher reward value can be obtained; The value of is used to limit the reward value within a certain range, thereby ensuring the stability of the training process; The update process for table Q is as follows: Step 1. Obtain the current state and actions Corresponding current value ; Step 2. Calculate the value ,in This represents the maximum value in the i-th row of table Q; Step 3. Calculate the updated value And use it to update the corresponding position in the Q table.

7. The multi-objective train scheduling method according to claim 6, characterized in that, The ten neighborhood structures include: NS-I: Swap elements at two cut points in the scheduling order, and simultaneously swap elements at the corresponding cut points in vehicle allocation; NS-II: Inserts the element of the later cut point in the scheduling order before the previous cut point; the same insertion operation is performed on vehicle allocation. NS-III: Reverses the elements between two cut points in the scheduling order, and performs a swap operation when assigning vehicles; NS-IV: Scheduling and sorting perform swap operations, while vehicle allocation performs insertion operations; NS-V: Scheduling and sorting perform insertion operations, while vehicle allocation performs swap operations; NS-VI: Scheduling and sorting are reversed, and vehicle allocation is inserted. NS-VII: The scheduling and sorting process performs the swap operation, and the vehicle allocation randomly assigns the number of vehicles. NS-VIII: Select two jobs from the critical job set and two jobs from the non-critical job set for swapping, and assign vehicles to perform the swapping operation; NS-IX: Scheduling and sorting perform critical job swapping operations, and vehicle allocation performs insertion operations; NS-X: Schedules and sorts critical job swap operations, and randomly assigns the number of vehicles.

8. The multi-objective train scheduling method according to claim 1, characterized in that, The crossover operator includes: The order-based crossover operator: randomly generates the cutting length and two cutting points, fills the child generation with the elements that do not contain each other in the parent generation, and then fills in the remaining elements; Job-based crossover operator: Randomly divide the job into two sets, copy the corresponding set elements from the parent set to the child set, and then fill in the remaining elements; Two-point crossover operator: For the vehicle allocation part, randomly select two cutting points and swap the elements between the parent cutting points.

9. A multi-objective train dispatching system integrating Q-learning, characterized in that, include: The data input module is used to acquire train operation data, network topology data, passenger demand data and operational constraint parameters. The train operation data includes train passenger capacity, running time of each section and planned arrival time. The operational constraint parameters include minimum safe interval time and section congestion rules. The mathematical modeling module is used to establish a mixed integer programming mathematical model for the job shop scheduling problem with congestion constraints based on the data obtained by the data input module. The model aims to minimize train delay penalties, passenger dissatisfaction, and the total number of operating vehicles, and includes congestion constraints, section passage priority constraints, train arrival time constraints, passenger boarding and alighting constraints, and train delay constraints. The encoding and decoding module includes an encoding unit and a decoding unit. The encoding unit is used to encode the scheduling problem using a two-layer vector encoding mechanism of scheduling order and vehicle allocation. The first vector represents the train scheduling order, and the second vector represents the number of vehicles allocated to each train. The decoding unit is used to decode based on the congestion constraint-related lemma and through a multi-block left insertion heuristic rule to calculate the start time and completion time of the train in each section. The population initialization module is used to generate initial solutions for the scheduling part using the NEH heuristic algorithm, generate initial solutions for the vehicle allocation part using a random heuristic method, and select high-quality individuals to form an integrated subpopulation after performing non-dominated sorting on the initial population. A multi-objective evolutionary optimization module is used to select individuals based on a tournament selection mechanism of integrated population, and to generate offspring population by applying crossover and mutation operators. The crossover operators include order-based crossover operators, task-based crossover operators, and two-point crossover operators. The Q-learning-driven variable neighborhood search module uses ten neighborhood structures as the action space for Q-learning, neighborhood structure numbers as states, a reward function designed based on multi-objective optimization results, and an operator selection probability matrix trained through a Q-table update mechanism to adaptively select neighborhood operators to search the offspring population. The population iterative screening module is used to merge the parent population with the new solution set output by the Q-learning driven variable neighborhood search module, and screen the next generation population through non-dominated sorting and crowding distance function, repeatedly triggering the multi-objective evolutionary optimization module and the Q-learning driven variable neighborhood search module until the termination condition is met. The scheduling scheme output module is used to output a Pareto optimal train scheduling scheme set, which includes the train scheduling sequence, the number of vehicles configured, and the running time of each section.

10. The multi-target train dispatching system according to claim 9, characterized in that, The specific constraints of the mixed integer programming mathematical model in the mathematical modeling module are as follows: Congestion constraint: Only one train is allowed to pass through the same section at the same time. The following train must wait for a minimum safe interval after the preceding train enters the next section, and the train must enter the next section immediately after completing the work in the previous section. Section passage priority constraint: A unique passage priority is assigned to each section through which a train passes. Only one train is assigned the same priority, and the priorities are consecutive and do not overlap. The start time of the next priority is greater than the completion time of the previous consecutive priority. Train arrival time constraints: Each train operation and arrival time in each section corresponds to a unique time period. The mapping relationship between operations and time periods is established through 0-1 variables. Passenger boarding and alighting constraints: The passenger capacity of a train shall not exceed the product of the number of carriages and the capacity of a single carriage. The number of passengers boarding and waiting shall be calculated separately for the first train and subsequent trains. Train delay constraint: The delay value is the difference between the actual completion time and the scheduled arrival time of the train, and the delay value is non-negative; Multi-block left insertion heuristic decoding module: used for efficient decoding of scheduling solutions of two-layer vector codes based on blocking constraint-related lemmas, capable of: The system receives the encoded scheduling and vehicle allocation information, and calculates the initial start time and completion time of each train in each section, taking into account the blockage time interval of each section. The time node for the last section of the train is fixed, and the start time of the preceding section is adjusted in reverse from right to left to eliminate invalid idle time between sections. Identify available free blocks in each section, determine whether the train operation meets the conditions that the end time is later than the planned arrival time and the operation duration is suitable for the free blocks, and perform multi-block left insertion operation; Under the premise of strictly meeting the congestion constraint that only one train is allowed to pass through the same section at the same time, optimize the train operation sequence, improve track utilization and the quality of decoding solutions, and ensure the feasibility and efficiency of the scheduling scheme. Improved Q-learning adaptive decision module: Provides an intelligent operator selection mechanism for variable neighborhood search, capable of: Action space design: The neighborhood structure of ten coverage scheduling ordering and vehicle allocation is used as the action space for Q learning to adapt to the search requirements of dual-layer encoding. Reward value calculation: Based on the multi-objective optimization results, an objective maximum value normalization mechanism is introduced. The reward value is calculated by the difference between the objective function values ​​of the current solution and the neighboring solutions, ensuring that the reward value is stable and can accurately reflect the search effect. Q-table update mechanism: Obtain the Q-table value corresponding to the current state and action, combine it with the immediate reward, discount factor and the maximum value of the row of the Q-table to calculate the comprehensive reward value, and dynamically update the Q-table through the learning rate to train the operator selection probability matrix; Adaptive operator selection: Based on the trained Q-table, the appropriate neighborhood operator is automatically selected according to the current optimization state, which improves the targeting and efficiency of variable neighborhood search and enhances the algorithm's global search and local convergence capabilities.

Images