Rail locomotive background intelligent scheduling control method based on multi-sensor fusion

Abstract

The invention relates to the field of rail traffic intelligent scheduling, and discloses a rail locomotive background intelligent scheduling control method based on multi-sensor fusion, which comprises the following steps: S1, acquiring operation data in real time through multiple types of sensors deployed along a rail, at a station and on a locomotive; s2, carrying out space-time alignment, filtering fusion and redundancy elimination processing on the multi-source heterogeneous sensor data to generate a unified track operation environment state vector; s3, based on the orbit operation environment state vector, constructing an intelligent scheduling model fusing a rhizome growth optimization algorithm and TD3 reinforcement learning, and generating a final scheduling strategy; and S4, converting the final scheduling strategy into a control instruction, issuing the control instruction by a rail transit background control system, coordinating with a full link, and feeding back the actual execution state of the rail locomotive based on a sensor to realize closed-loop control. The intelligent operation level, the resource utilization rate and the operation safety of the rail transit system are remarkably improved.

Description

Technical Field

[0001] The present invention relates to the technical field of rail transit intelligent dispatching, and in particular to a rail locomotive background intelligent dispatching control method based on multi-sensor fusion. Background Art

[0002] With the rapid development of urban rail transit systems and the increasing density of locomotive operations, traditional dispatching methods are no longer able to meet the operational control requirements of multi-objective, highly dynamic environments. Especially in complex rail scenarios, dispatching decisions must simultaneously consider multi-dimensional indicators such as minimizing energy consumption, maximizing operational efficiency, controlling safety risks, and ensuring passenger comfort. However, existing dispatching systems mostly rely on static rules or a single data source, lacking accurate perception and flexible response to complex rail operational situations, which can easily lead to wasted resources, accumulating delays, and increasing safety risks.

[0003] The introduction of multi-sensor fusion technology provides rail transit systems with richer, real-time environmental and status information, laying a solid data foundation for intelligent dispatching. Building high-performance dispatch optimization algorithms based on multi-source, heterogeneous sensor data has become a key issue in enhancing the intelligent operation of rail locomotives. Summary of the Invention

[0004] The present invention aims to provide a background intelligent dispatching and control method for rail locomotives based on multi-sensor fusion, which improves the operating efficiency and safety of rail locomotives in complex environments by integrating perception, optimization and intelligent decision-making mechanisms.

[0005] To achieve the above objectives, the following technical solutions are adopted:

[0006] A method for intelligent scheduling and control of railway locomotives based on multi-sensor fusion, comprising the following steps: Step S1: real-time collection of operating data by multiple types of sensors deployed along the track, at stations and on locomotives, wherein the sensors include at least a positioning module, an obstacle detection device, an environmental monitoring device and an energy monitoring device; Step S2: performing spatiotemporal alignment, filtering fusion and redundancy elimination processing on multi-source heterogeneous sensor data to generate a unified railway operation environment state vector; Step S3: constructing an intelligent scheduling model integrating rhizome growth optimization algorithm and TD3 reinforcement learning based on the railway operation environment state vector to generate a final scheduling strategy; wherein the construction of the fusion The intelligent scheduling model combining the rhizome growth optimization algorithm and TD3 reinforcement learning includes: using the rhizome growth optimization algorithm to perform local optimization of root movement and global exploration of stolon movement in the scheduling parameter space, and outputting a preliminary optimized scheduling plan; inputting the preliminary optimized scheduling plan into the TD3 reinforcement learning model, and performing deep optimization through dual-critic network evaluation, delay strategy update, target action noise addition and target network soft update mechanism; step S4: converting the final scheduling strategy into control instructions, and the rail transit background control system implements the control instruction issuance and full-link coordination, and at the same time realizes closed-loop control based on sensor feedback of the actual execution status of the rail locomotive.

[0007] Furthermore, the execution process of the rhizome growth optimization algorithm includes:

[0008] According to the current environmental state, an initial population of N feasible scheduling schemes is generated and its objective function value is evaluated; each scheduling scheme is composed of multi-dimensional parameters, including: locomotive running time parameters, stop configuration parameters, speed setting parameters, energy allocation parameters, track occupancy sequence parameters; iterative optimization stage: each set of scheduling parameters is updated through two search methods: the first search method is to perform a local search for each scheduling scheme by performing RootsNum root system moves: if , then use the new solution Replace the current solution ; The second search method is: perform RunnersNum global searches for each solution: if , and a new interpretation Better than the worst solution in the population , then use the new solution Replace the solution with the worst objective function value in the population , maintain population diversity; termination conditions, stop if any one of them is met: reaching the maximum number of iterations; the rate of change of the objective function of the population optimal solution for K consecutive rounds is less than the preset threshold; output the optimal scheduling solution , which is input into the TD3 reinforcement learning model as the optimal scheduling solution.

[0009] Furthermore, the method includes:

[0010] In the local perturbation optimization stage, a random perturbation of 5%-20% is applied to the continuous parameters, with priority given to adjusting the departure time, stop duration and running speed to achieve local parameter fine-tuning; in the global perturbation exploration stage, a random perturbation of 30%-60% is applied to the discrete parameters, with priority given to reconstructing the stop decision, track allocation sequence and energy allocation strategy to achieve global reconstruction.

[0011] Furthermore, the TD3 reinforcement learning model construction includes:

[0012] The discrete scheduling solution generated by the root-stalk growth optimization algorithm is converted into a continuous vector through a normalized mapping layer; the action space is defined as an incremental adjustment vector and the final scheduling parameters are generated; a reward function is designed that integrates multi-dimensional indicators such as safety, comfort, energy consumption, and track deviation; a differentiable processing mechanism of forward threshold decision and reverse Gumbel-Softmax gradient estimation is adopted for binary scheduling parameters; when the scheduling scheme violates the dynamic safety constraints, it falls back to the feasible solution of the root-stalk algorithm.

[0013] Furthermore, the preliminary optimized scheduling scheme is input into the TD3 reinforcement learning model and deeply optimized through dual critic network evaluation, delay policy update, target action noise addition and target network soft update mechanism, including:

[0014] Dual Critic Network Evaluation: Take the minimum output of the two Critic networks as the target Q value, and update the Critic network parameters based on the mean square error between the target Q value and the predicted Q values ​​of the two Critic networks; Delayed Policy Update: The Actor network is updated less frequently than the Critic network; Target Action Noise Added: Add truncated noise to the target action to enhance exploratory power; Target Network Soft Update: Progressively update the parameters of the target Actor network and target Critic network through soft update coefficients.

[0015] Furthermore, the truncation noise comes from a normal distribution N(0,σ) and the noise value is limited to the interval [-c,c], where σ is the noise standard deviation.

[0016] Furthermore, the soft update coefficient does not exceed 0.01.

[0017] Furthermore, in step S4, the rail transit background control system implements the issuance of control instructions and full-link coordination, including: implementing locomotive departure time setting, section speed limit, stop priority and power adjustment operations, and the instructions synchronously link signal light control, platform broadcasting and power supply systems; wherein, the control instructions include: locomotive departure time and stop duration instructions; section speed limit and track allocation instructions; power system power adjustment instructions; the closed-loop control realizes dynamic strategy adjustment by comparing the control instructions with the actual position, speed and energy consumption parameters of the locomotive.

[0018] Furthermore, in step S4, the dynamic strategy adjustment of the closed-loop control includes:

[0019] When the actual position or speed exceeds the threshold: the emergency braking protocol is immediately initiated; after emergency braking, steps S2-S3 are re-triggered within t≤10s to generate an emergency scheduling strategy; if no new strategy is generated within the timeout, the baseline scheduling strategy is enabled.

[0020] Compared with the prior art, the present invention achieves the following beneficial effects:

[0021] This paper introduces the Root-Growth Optimization Algorithm (RGO) as a meta-heuristic optimization tool to propose a scheduling optimization method suitable for the complex and dynamic environment of rail transit. By simulating the growth characteristics of strawberry plant roots and runners, it implements a global-local joint search for rail vehicle scheduling solutions, ensuring both diverse and high-quality scheduling results.

[0022] 2. This invention builds a stolon and root search mechanism to achieve dynamic search and update in the scheduling parameter space. Root movement allows for localized fine-tuning, quickly adjusting details such as departure times and stop durations. Stolon movement allows for the exploration of new global solutions over a wide range, effectively transcending local optimal zones and identifying potentially superior scheduling strategies.

[0023] 3. This paper proposes using the TD3 reinforcement learning algorithm to deeply optimize the scheduling strategy. This algorithm, through a dual-critic network architecture, a delayed policy update mechanism, and target action noise design, effectively alleviates the Q-value overestimation problem in reinforcement learning and improves the training stability and generalization capabilities of the rail vehicle scheduling control strategy.

[0024] 4. This paper proposes a target network soft update mechanism, replacing the traditional parameter replication method. By slowly integrating the current network parameters into the target network in smaller steps, it avoids violent fluctuations in the target value, effectively improving the convergence stability and prediction reliability of the critic network, and providing stable support for the optimization and iteration of the entire scheduling strategy.

[0025] Through the integration and coordination of the above methods, the present invention constructs an intelligent dispatching and control system that can adapt to the complex perception environment of multiple sensors, realizes the full-link integration from environmental perception, strategy generation to optimization control, and significantly improves the intelligent operation level, resource utilization and operation safety of the rail transit system.

[0026] It should be understood that the contents described in the summary of the invention are not intended to limit the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become readily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] The above and other features, advantages and aspects of the embodiments of the present invention will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. The accompanying drawings are provided for a better understanding of the present invention and do not constitute a limitation of the present invention. In the accompanying drawings, the same or similar reference numerals represent the same or similar elements, among which:

[0028] Figure 1 It is a flow chart of a method for background intelligent dispatching and control of a rail locomotive based on multi-sensor fusion according to an embodiment of the present invention.

[0029] Figure 2 4 is a flow chart of a rhizome growth optimization algorithm according to an embodiment of the present invention.

[0030] Figure 3 This is a schematic diagram of the main process of the TD3 reinforcement learning model in an embodiment of the present invention. DETAILED DESCRIPTION

[0031] To make the objectives, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0032] In this document, the term "and / or" simply describes a relationship between related objects, indicating that three possible relationships exist. For example, "A and / or B" can represent: A exists alone, A and B exist simultaneously, or B exists alone. Furthermore, the character " / " in this document generally indicates that the related objects are in an "or" relationship.

[0033] Figure 1 This is a flow chart of a method for intelligent dispatching and controlling a railway locomotive based on multi-sensor fusion according to an embodiment of the present invention. Figure 1As shown, a railway locomotive background intelligent dispatching control method based on multi-sensor fusion includes the following steps:

[0034] Step S1: Real-time operation data is collected through multiple types of sensors deployed along the track, at stations and on locomotives. The sensors include at least positioning modules, obstacle detection equipment, environmental monitoring equipment and energy monitoring equipment.

[0035] In step S1, multi-dimensional perception of the entire rail transit operation process is achieved by deploying multiple types of high-precision sensor equipment along the track, in stations, in train control rooms, and at key locations on locomotive bodies. The sensors used include but are not limited to:

[0036] GPS positioning module: used to obtain the latitude and longitude coordinates, running track and location information of the rail locomotive in real time.

[0037] Millimeter-wave radar and lidar: enable accurate perception of obstacles ahead, train spacing, track occupancy status, etc.

[0038] High-definition cameras and infrared thermal imaging equipment: collect video images or thermal information in stations, tunnels, and switch areas to monitor passenger flow, ambient temperature, nighttime visibility, etc.

[0039] Ultrasonic sensors: Assist in monitoring short-range obstacles, tunnel walls or lateral limits.

[0040] Inertial Measurement Unit (IMU): used to compensate for vehicle attitude estimation in areas where GPS signal is lost.

[0041] Track geometry sensors (such as track gauges, inclination sensors) or locomotive axle box lateral acceleration sensors: used to detect track deformation or locomotive lateral shaking.

[0042] Environmental sensors such as temperature and humidity sensors, air pressure sensors, etc.: obtain meteorological parameters of the orbital operating environment.

[0043] Load / passenger flow sensor: Counts the weight of cargo and the number of passengers on the current train to assist in generating scheduling strategies.

[0044] Energy sensors such as electricity, current, and power: used to monitor the energy usage status of rail vehicles and the load of the power supply system.

[0045] Through the above-mentioned multi-source perception system, key operating parameters in the rail transit system are collected in real time to form a structured, multi-dimensional, high-time resolution original data set, providing a data basis for subsequent data fusion and intelligent scheduling.

[0046] Step S2: Perform spatiotemporal alignment, filtering fusion, and redundancy elimination on multi-source heterogeneous sensor data to generate a unified orbital operation environment state vector.

[0047] In step S2, the system introduces a multi-sensor fusion algorithm for the collected multi-source heterogeneous sensor data to improve the accuracy, robustness and integrity of the rail transit system's operational situation awareness. The main processing flow is as follows:

[0048] S2.1: Time and space alignment preprocessing: Achieve unified and standardized processing of various sensor data through timestamp synchronization, spatial coordinate transformation, sampling rate matching, etc.

[0049] S2.2: Kalman filter: Suitable for fusing GPS and IMU data to achieve high-precision positioning of rail locomotives.

[0050] S2.3: Redundancy elimination and consistency verification: Eliminate duplicate, invalid or conflicting data to improve data validity.

[0051] S2.4: Situational awareness information construction: Generate a unified, reliable and structured "track operation environment state vector", including track occupancy status, station passenger flow density, obstacle distribution, weather conditions, train operation status, etc.

[0052] S2.5: The integrated data provides global operational status support for scheduling optimization and is the core basis for decision-making in intelligent control systems.

[0053] Step S3: Based on the track operation environment state vector, an intelligent scheduling model integrating the rhizome growth optimization algorithm and TD3 reinforcement learning is constructed to generate a final scheduling strategy.

[0054] In step S3, based on the situational awareness information fused in step S2, an intelligent scheduling model based on multi-sensor fusion is constructed. This model considers multiple optimization objectives such as transportation efficiency, energy consumption, safety, and passenger comfort, and makes intelligent scheduling decisions on locomotive operation plans, schedules, stops, and travel speeds.

[0055] Root-Growth Optimization Algorithm (RGO) and TD3 collaborative optimization: Root-Growth Optimization Algorithm: By simulating the growth characteristics of strawberry plant roots and runners, it realizes a global-local joint search for rail vehicle scheduling solutions and quickly generates feasible solutions in high-dimensional discrete space. TD3: A feasible solution is obtained through a dual critic network. Perform dynamic tuning to compensate for the lack of adaptability of the Strawberry Algorithm in the continuous action space; Normalization mapping layer: realize the differentiable conversion of discrete scheduling parameters to the continuous action space, so that the gradient can be back-propagated from TD3 to the output layer of the root growth optimization algorithm.

[0056] Furthermore, in step S3, the construction of an intelligent scheduling model integrating the rhizome growth optimization algorithm and TD3 reinforcement learning includes: using the rhizome growth optimization algorithm to perform local optimization of root movement and global exploration of stolon movement in the scheduling parameter space, outputting a preliminary optimized scheduling plan; inputting the preliminary optimized scheduling plan into the TD3 reinforcement learning model, and performing deep optimization through dual critic network evaluation, delayed strategy update, target action noise addition, and target network soft update mechanism. Specifically, it includes:

[0057] Step S3.1 makes a scheduling decision.

[0058] On the basis of completing multi-sensor data fusion and establishing overall rail transit situation awareness, the dispatching system needs to formulate efficient and intelligent dispatching strategies to optimize the running paths, shift arrangements and energy distribution of rail locomotives. Figure 2 As shown, the present invention introduces a rhizome growth optimization algorithm as a meta-heuristic optimization tool for realizing global-local joint search and optimal decision-making for rail locomotive scheduling in a complex dynamic environment.

[0059] Step S3.1.1 Scheduling optimization modeling.

[0060] (1) Generate an initial population containing N feasible scheduling solutions based on the track operation environment state vector generated by multi-sensor fusion in step S2 .

[0061] Among them, each scheduling scheme It is composed of multi-dimensional parameters, including:

[0062] - Locomotive operation time parameters: planned departure time, interval operation time offset; - Stop station configuration parameters: binary identification of stop / pass decision and corresponding stop duration; - Speed ​​setting parameters: interval operation speed, entry deceleration curve, exit acceleration curve; - Energy distribution parameters: traction power distribution ratio coefficient and regenerative braking energy recovery coefficient; - Track occupancy sequence parameters: track resource allocation sequence and conflict buffer time interval.

[0063] (2) Assume the objective function of the rail locomotive dispatching system for:

[0064]

[0065] : is a decision variable vector, including scheduling parameters such as locomotive running time, stop sites, speed setting, energy distribution, etc. : It is the feasible solution space under multiple constraints (such as train interval, safety distance, energy capacity limit, etc.). : Energy consumption function, estimated by battery, current, and slope data collected by multiple sensors; : total running time; : Track resource conflict penalty function (taking into account station congestion and section conflicts), verified in real time through track occupancy sensors; : Safety risk coefficient, V(x) = ∑(obstacle risk value + weather penalty factor), where the risk value is mapped from millimeter-wave radar data, obstacle risk value = 1 / (1 + exp(-0.1D)), D is the obstacle distance detected by the millimeter-wave radar, unit: meter; weather penalty factor = rainfall intensity × 0.5 + inverse visibility × 0.3 (obtained by environmental sensors); : The weight of each objective function, which is set according to the specific scheduling goal.

[0066] Step S3.1.2 constructs the root-runner optimization mechanism.

[0067] In the rhizome growth optimization algorithm, each set of scheduling parameters Updates are made via two search methods:

[0068] 1. Root movement (local disturbance optimization):

[0069]

[0070] : Current scheduling parameter vector, which represents the scheduling strategy configuration of the locomotive at the current stage (such as departure time, stop time, track occupancy order, etc.). : The updated scheduling parameter vector, which represents the new scheduling solution obtained after applying the local perturbation. : Root step size, which indicates the adjustment range for local fine-tuning and is set to the range of possible values ​​of the scheduling parameter ,Right now , used to fine-tune scheduling details, such as fine-tuning locomotive departure times, stop times, operating speeds or interval timings. : Random perturbation factor, in Random numbers are uniformly sampled within a range to simulate local uncertainty and enhance the diversity and exploration of small perturbations.

[0071] 2. Stolon movement (global perturbation exploration):

[0072]

[0073] : Current scheduling parameter vector, same as above, represents the rail locomotive scheduling plan generated under the guidance of the current multi-sensor perception data. : The scheduling parameter vector after global exploration, which represents the new scheduling scheme obtained after a large adjustment, is used to explore the potential optimal scheduling structure. : stolon step size, representing the perturbation scale used for global exploration, set to the parameter range ,Right now , which is suitable for large-scale strategy reconstruction, such as adjusting the dispatch sequence of the entire line, reallocating multiple locomotive tasks or avoidance strategies. : As above, the randomness introduced by the global perturbation ensures that the exploration is uncertain and diverse, and improves the algorithm's ability to escape from the local optimum.

[0074] Step S3.1.3 The execution process of the root growth optimization algorithm includes:

[0075] 1. Initialization:

[0076] Generate the initial scheduling solution population based on the environmental state after multi-sensor fusion , check the feasibility of each solution through the track resource conflict penalty function U(x), eliminate the solution that violates the safety constraint (such as train interval < safety distance), and regenerate until the population is full, and calculate each scheduling plan The objective function value of (Based on the objective function defined in step S3.1.1 ).

[0077] 2. Iterative optimization:

[0078] The first search method is: perform root movement (local perturbation optimization) for each scheduling scheme RootsNum times (default 10 times); if , that is, the new solution Better than the original solution , then use the new solution Replace the current solution .

[0079] The second search method is to perform RunnersNum (default 3) stolon moves (global perturbation exploration) for each scheduling scheme. , and a new interpretation Better than the worst solution in the population , then use the new solution Replace the solution with the worst objective function value in the population (i.e., the individual with the worst objective function value in the population), maintain population diversity and replace the inferior solutions in the current population to enhance diversity.

[0080] 3. Termination conditions:

[0081] Convergence condition: When the maximum number of iterations is reached (e.g. 100 rounds) or the scheduling solution converges to meet the preset threshold, for example, the optimal solution objective function change rate Δf / f < 0.001 (for 5 consecutive rounds); Output: Optimal scheduling solution , which contains the complete decision variable vector and serves as the core input of the TD3 algorithm (step S3.2.1).

[0082] Through this method, optimal scheduling can be achieved in complex environments by comprehensively considering energy consumption, time, safety and track resource utilization, providing high-quality scheduling solutions for the background intelligent control system.

[0083] Step S3.2: Decision optimization.

[0084] This step S3.2 uses the TD3 algorithm to optimize the rail vehicle scheduling strategy. TD3 is a reinforcement learning algorithm suitable for processing problems in high-dimensional, continuous action spaces. Its core advantage lies in the use of dual critic networks, delayed policy updates, and target action noise mechanisms to effectively alleviate the Q-value overestimation problem and improve training stability and policy performance. Figure 3 As shown:

[0085] Step S3.2.1 constructs the state-action space.

[0086] State Space : Contains key information about the target rail vehicle and its surroundings, specifically expressed as:

[0087]

[0088] Indicates the status of the target track vehicle (such as position, speed); Indicates the status of the vehicles in front and behind the target track; Indicates the status of locomotives on adjacent tracks; Indicates the lateral state of the target locomotive (such as lateral offset, axle box acceleration).

[0089] Action Space Defined as the incremental adjustment vector and , d is the scheduling parameter dimension. If the scheduling parameters are departure time and speed, then ,in represents the time offset coefficient, Indicates the speed adjustment factor.

[0090] Generation of final scheduling parameters:

[0091]

[0092] ⊙: Element-by-element multiplication ensures that each scheduling parameter is adjusted independently; is a normal distribution, : noise standard deviation; : Limit the noise value to the interval [-c, c] to avoid large changes in the scheduling action and cause operational risks. η: Scaling coefficient vector, used to normalize the action value Mapping to the actual physical range calculation formula:

[0093]

[0094] For example: departure time parameters The range is [08:00, 18:00] (600 minutes), and λ=0.05, then Minutes. It means the departure time will be delayed by 15 minutes.

[0095] The Actor network directly inputs the state vector s and outputs the optimal scheduling solution The correction amount is used to achieve hierarchical optimization of root-stolon coarse adjustment and TD3 fine adjustment.

[0096] The current optimal scheduling solution output in step S3.1 The discrete parameters (such as departure time and stop time) are normalized through the normalization mapping layer: , , the normalized continuous vector ;in, , It is the minimum and maximum value of the numerical range determined by the physical constraints of the rail system: for example, departure time: [first train time, last train time], speed: [0, maximum speed of the line design]; .

[0097] like Leading to constraint violation (such as train interval < safety distance , where v is the vehicle speed (km / h), then the optimal scheduling solution output by this Actor is abandoned. The correction amount is used to fall back to the feasible solution using the root-stolon algorithm. At the same time, the constraint violation event is recorded as a signal that the environment is changing dynamically or that the model needs to be adjusted. If the constraint violation event occurs more than M times (for example, M = 5) in a row, the baseline scheduling strategy is activated and the sensor data re-collection is triggered (jump to step S1).

[0098] Reward Function : It is a multi-target fusion form, taking the following factors into consideration:

[0099]

[0100] in, : Safety rewards (such as collision avoidance), e.g. ,in The train spacing measured by millimeter-wave radar; : Comfort penalty (control acceleration change and yaw angle), for example , jerk is the acceleration change rate (unit: m / s 3 ); : Energy consumption and emission costs, ,in, is the total energy consumption in kWh; : Penalty for lateral track deviation, e.g. ,in is the lateral offset collected by the axle box sensor (unit: mm), is the penalty coefficient (e.g. ).

[0101] Output the final scheduling parameters of the Actor network Perform inverse normalization to restore the specific scheduling parameter values ​​and form the final executable scheduling strategy. For binary parameters such as docking decisions: Forward propagation: Use threshold comparison (if , otherwise 0); Backpropagation: Use Gumbel-Softmax gradient estimation to maintain differentiability. For example, for a stop decision (binary identification), the actor might output a continuous value in the interval [0, 1]. Then, a threshold (such as 0.5) is set. If the inverse normalized value is ≥ 0.5, the stop is made; otherwise, the stop is passed. The track assignment sequence number is rounded to the nearest integer. For departure time, the continuous value output by the actor is inverse normalized to the range [first train time, last train time] and may need to be quantized to the nearest scheduling time granularity (such as minutes).

[0102] Step S3.2.2 introduces TD3 algorithm optimization.

[0103] The core idea of ​​TD3 is to introduce the following five key mechanisms based on DDPG to improve stability:

[0104] 1. Dual Critic Network:

[0105] To avoid overestimation of the Q value, the minimum value output by the two Critic networks is taken as the target value:

[0106]

[0107] in, : The target Q value of the i-th sample is used to guide the learning of the Critic network and is the future value evaluation benchmark for rail locomotive scheduling actions. : Instant reward value, actual feedback obtained (such as improved energy saving, reduced scheduling delays, improved passenger satisfaction, etc.). : Discount factor , used to weigh the importance of current rewards and future rewards, preferably Set to 0.99. In rail locomotive scheduling, it is used to balance the immediate scheduling effect and future global scheduling efficiency. : No. A target critic network to evaluate the next state Next action The expected cumulative reward. : No. The parameter set of the target critic network. : The track transportation status at the next moment (such as track congestion, locomotive position distribution, station passenger flow changes, etc.). : The next scheduling action (such as adjusting the route, departure order or stop points, etc.).

[0108] 2. Critic Network Update:

[0109] Optimize the Critic network by minimizing the mean square error between the predicted value and the target value:

[0110] : The loss function value of the kth critic network, which measures the deviation between the current Q network prediction value and the target Q value, and is used to guide the iterative optimization of model parameters. M: The number of samples in the training batch, that is, the number of track scheduling state-action pairs participating in the training at one time. : The kth current Critic network, used to evaluate the state Take action The cumulative expected rewards obtained. : The set of trainable parameters of the current Critic network. : The current state of the i-th track system, including fused sensor information, such as track occupancy, real-time positioning, energy consumption status, etc. : In state The dispatching actions performed under the control of the train operator, such as assigning locomotive tasks, adjusting running intervals or rearranging the stop sequence, etc.

[0111] 3. Actor Network Update:

[0112] The policy gradient method is used to optimize the Actor network parameters to maximize the Q value estimation of its output action:

[0113]

[0114] : Actor network parameters For the objective function In the locomotive scheduling task, it indicates how to adjust the parameters of the Actor network so that the scheduling actions it generates can maximize the long-term comprehensive value of the scheduling system (such as efficiency, safety, and minimization of energy consumption). : The number of track scheduling samples included in the training batch. :About action The partial derivative of Under this condition, a small change in the action will affect the Critic evaluation value In the dispatching system, this means whether a slight change in the dispatching action will improve or degrade the overall performance of the rail locomotive under the current state. : The i-th system state, including multi-source sensor fusion data such as track occupancy, train position, energy consumption, and grid load. : Actor network output action to its parameters The partial derivative of , which indicates how a slight adjustment to the internal parameters of the Actor network will affect its output scheduling action. In the track scheduling task, it reflects how a slight adjustment to the policy model will change the output scheduling solution. :Actor network according to the state Output scheduling actions, such as generated train departure times, stop strategies, or track assignment plans. : A collection of parameters for the current Actor network. : The first Critic network evaluates the Actor output action In state The comprehensive dispatch value under the current situation (such as energy saving, safety and operation efficiency). : The parameter set of the current Critic network.

[0115] 4. Policy delay updates and target actions increase noise:

[0116] (1) Strategy delayed update: The Actor network is updated only once every certain number of steps to improve learning stability.

[0117] (2) Adding noise to the target action: Adding truncated noise to the target action enhances the strategy exploration. The noise formula is as follows:

[0118] : The target scheduling action after adding noise is used to enhance the strategy exploration (such as exploring different scheduling combination schemes). :The target Actor network is based on the next state The action decisions given reflect the prediction and control of future orbital resource allocation. : The target actor network parameter set, which is periodically soft-updated from the current actor network. : Additive noise, used to introduce disturbances in action decisions to prevent the strategy from falling into local optimality. The noise source is normally distributed . : Limit the noise value to the interval [-c, c] to avoid large changes in scheduling actions that may lead to operational risks. : Noise standard deviation, used to control the amplitude of scheduling strategy exploration, preferably . : Noise cutoff threshold, preferably c=0.5, limits the degree of disturbance and ensures that the scheduling plan changes within a controllable range.

[0119] 5. Target network soft update:

[0120] To avoid oscillations caused by direct replication, the target network parameters use a soft update mechanism:

[0121]

[0122] in, : Current critic network parameters, learning the value evaluation of the current scheduling scheme. : Target Critic network parameters, which provide a stable training target Value estimation. : Current Actor network parameters, used to generate scheduling strategies. : Target Actor network parameters, providing a reference for strategy stability. : Soft update coefficient (value is 0.001), which controls the update speed of target network parameters and helps avoid scheduling system oscillation or instability caused by drastic parameter fluctuations.

[0123] Through the construction and training of the above-mentioned TD3 reinforcement learning strategy, the system can continuously optimize rail locomotive scheduling decisions in a dynamic and complex rail transit environment, achieving safe, efficient and green scheduling control.

[0124] Step S4: The final scheduling strategy is converted into control instructions, and the rail transit background control system implements the control instruction issuance and full-link coordination, while realizing closed-loop control based on the actual execution status of the rail locomotive fed back by sensors.

[0125] Furthermore, in step S4, the rail transit background control system implements control instruction issuance and full-link coordination, including: implementing locomotive departure time setting, section speed limit, station stop priority and power adjustment operations, and the instructions synchronously link signal light control, platform broadcasting and power supply system.

[0126] Among them, control instructions include: locomotive departure time and stop duration instructions; section speed limit and track allocation instructions; power system power adjustment instructions; closed-loop control realizes dynamic strategy adjustment by comparing control instructions with the actual position, speed and energy consumption parameters of the locomotive.

[0127] After completing the intelligent scheduling model reasoning and obtaining the optimal scheduling strategy, the system converts the scheduling results into specific executable control instructions, and the rail transit background control system implements the instruction issuance and full-link coordination. The execution process is as follows:

[0128] 1. Generation and issuance of control instructions: De-map the dispatch output from step S3 into an instruction set, such as: locomotive departure time setting; section speed limit instructions; stop and pass priority instructions; power system power adjustment instructions.

[0129] 2. Coordinated control of system modules: Autonomously adjust acceleration, deceleration, and driving trajectory based on control instructions; perform path safety verification on train dispatch instructions and adjust signal light status or track switching mechanisms; synchronize dispatch plans with platform announcements, passenger guidance, ticket checking, and other processes; and dynamically adjust power supply network load according to the dispatch plan to optimize energy consumption.

[0130] 3. Real-time feedback and closed-loop control: Sensors provide real-time feedback on the locomotive's execution status. When the actual position / speed deviates from the commanded value by more than a threshold ϵ (dynamically set based on track safety regulations, sensor accuracy, and locomotive performance; for example, the position deviation threshold can be set to 5-10 meters (when positioning accuracy degrades in tunnels, the ε position threshold is automatically relaxed to 15 meters; the default value is 10 meters on clear days), and the speed deviation threshold can be set to 5-10% of the commanded speed), the emergency braking protocol is initiated, re-triggering steps S2-S3 to generate the emergency dispatch strategy. This forms a closed-loop control system, enabling timely strategy adjustments to address unexpected situations (such as sudden congestion and weather events).

[0131] After an emergency brake, the system suspends execution of the current command for t seconds (e.g., t = 10 seconds) and re-triggers steps S2-S3 to generate an emergency dispatch strategy. If no new strategy is obtained within t seconds, the baseline dispatch strategy (e.g., fixed-interval dispatch) is activated until manual intervention. The baseline dispatch strategy uses a fixed-interval mode: with an interval of T = 5 minutes, all trains run at a constant speed of 80% of the line's maximum design speed, and a fixed stop time of 30 seconds at each station.

[0132] This background control mechanism ensures that the dispatching strategy is not only "available" but also "implemented", achieving the goal of fully automatic, high-precision and coordinated dispatching control of rail locomotive operation.

[0133] Test experiment:

[0134] Test parameter description: TD3 training round = 50,000 steps; Root-Runner Algorithm: Population size N = 50, RootsNum = 10, RunnersNum = 3, Conflict Resolution Rate = Percentage of scheduling events that successfully avoided collisions. Test environment: 5 km circular track, 10 locomotives, peak passenger flow density of 2,000 people / hour. Data collected from the average of 100 independent experiments. Tunnel scenario measurement: When the GPS signal is lost, the IMU confidence automatically increases to 0.92 (originally 0.6), and the positioning error is reduced by 41% (from 3.2m to 1.9m). Experiments show that compared with traditional methods, this combined algorithm has significant advantages in scheduling delay reduction and energy saving (see Table 1):

[0135] Table 1

[0136] .

[0137] It should be noted that the various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. Reference can be made to the common and similar parts between the various embodiments. For the devices disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple, and the relevant parts can be referred to the description of the methods.

[0138] It should also be noted that, in the embodiments of the present application, relational terms such as first and second, etc. are merely used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply the existence of any such actual relationship or order between these entities or operations. Moreover, the terms "comprise", "include" or any other variants thereof are intended to cover non-exclusive inclusion, so that the process, method, article or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the statement "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device comprising the elements.

[0139] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined in the embodiments of this application may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not limited to the embodiments shown in this application, but is intended to conform to the broadest scope consistent with the principles and novel features disclosed in the embodiments of this application.

Claims

1. A railway locomotive background intelligent dispatching control method based on multi-sensor fusion, characterized in that: The following steps are involved: Step S1: Real-time operation data is collected through multiple types of sensors deployed along the track, at stations, and on locomotives. The sensors include at least a positioning module, an obstacle detection device, an environmental monitoring device, and an energy monitoring device. Step S2: Perform spatiotemporal alignment, filtering fusion, and redundancy elimination on multi-source heterogeneous sensor data to generate a unified orbital operation environment state vector; Step S3: Based on the track operation environment state vector, an intelligent scheduling model integrating the rhizome growth optimization algorithm and TD3 reinforcement learning is constructed to generate a final scheduling strategy; The construction of an intelligent scheduling model integrating rhizome growth optimization algorithm and TD3 reinforcement learning includes: The root growth optimization algorithm is used to perform local optimization of root movement and global exploration of stolon movement in the scheduling parameter space, and output a preliminary optimized scheduling plan; The preliminary optimized scheduling scheme is input into the TD3 reinforcement learning model and deeply optimized through dual critic network evaluation, delayed policy update, target action noise addition and target network soft update mechanism; Step S4: The final scheduling strategy is converted into control instructions, and the rail transit background control system implements the control instruction issuance and full-link coordination, while realizing closed-loop control based on the actual execution status of the rail locomotive fed back by sensors.

2. The railway locomotive background intelligent dispatching control method based on multi-sensor fusion according to claim 1 is characterized in that: The execution process of the rhizome growth optimization algorithm includes: Based on the current environmental state, an initial population of N feasible scheduling solutions is generated and their objective function values ​​are evaluated. Each scheduling solution consists of multi-dimensional parameters, including locomotive running time parameters, stop configuration parameters, speed setting parameters, energy allocation parameters, and track occupancy sequence parameters. Iterative optimization phase: Each set of scheduling parameters is updated through two search methods: The first search method is to perform a local search for each scheduling scheme by performing RootsNum root system moves: If , then use the new solution Replace the current solution ; The second search method is: perform RunnersNum global searches for each solution: if , and a new interpretation Better than the worst solution in the population , then use the new solution Replace the solution with the worst objective function value in the population , maintaining population diversity; Termination conditions: The system stops when any of the following conditions is met: the maximum number of iterations is reached; the rate of change of the objective function of the population optimal solution for K consecutive rounds is less than the preset threshold; Output optimal scheduling solution , which is input into the TD3 reinforcement learning model as the optimal scheduling solution.

3. The railway locomotive background intelligent dispatching control method based on multi-sensor fusion according to claim 2 is characterized in that: The method comprises: In the local perturbation optimization phase, random perturbations of 5%-20% are applied to continuous parameters, with priority given to adjusting departure time, stop duration, and operating speed to achieve fine-tuning of local parameters. In the global perturbation exploration phase, random perturbations of 30%-60% are applied to discrete parameters, prioritizing the reconstruction of docking decisions, track allocation sequences, and energy allocation strategies to achieve global reconstruction.

4. The railway locomotive background intelligent dispatching control method based on multi-sensor fusion according to claim 3 is characterized in that: The TD3 reinforcement learning model construction includes: The discrete scheduling solution generated by the rhizome growth optimization algorithm is converted into a continuous vector through a normalized mapping layer; Define the action space as the incremental adjustment vector and generate the final scheduling parameters; Design a reward function that integrates multiple indicators including safety, comfort, energy consumption, and track deviation; A differentiable processing mechanism using forward threshold decision and reverse Gumbel-Softmax gradient estimation is used for binary scheduling parameters; When the scheduling scheme violates the dynamic safety constraint, it falls back to the feasible solution of the root-stolon algorithm.

5. The railway locomotive background intelligent dispatching control method based on multi-sensor fusion according to claim 4 is characterized in that: The preliminary optimization scheduling scheme is input into the TD3 reinforcement learning model and deeply optimized through dual critic network evaluation, delayed policy update, target action noise addition and target network soft update mechanism, including: Dual Critic Network Evaluation: Take the minimum value of the two critic network outputs as the target Q value, and update the critic network parameters based on the mean square error between the target Q value and the predicted Q values ​​of the two critic networks; Delayed policy updates: The Actor network updates less frequently than the Critic network. Target action noise addition: Add truncated noise to the target action to enhance exploration; Target network soft update: Progressively update the parameters of the target actor network and target critic network through soft update coefficients.

6. The railway locomotive background intelligent dispatching control method based on multi-sensor fusion according to claim 5 is characterized in that: in, The truncation noise comes from the normal distribution N(0,σ) and the noise value is limited to the interval [-c,c], where σ is the noise standard deviation.

7. The railway locomotive background intelligent dispatching control method based on multi-sensor fusion according to claim 6 is characterized in that: in, The soft update coefficient does not exceed 0.

01.

8. The railway locomotive background intelligent dispatching control method based on multi-sensor fusion according to claim 1 is characterized in that: In step S4, the rail transit backend control system implements control command issuance and full-link coordination, including: implementing locomotive departure time setting, section speed limit, station stop priority and power adjustment operations. The command synchronizes the signal light control, platform broadcasting and power supply system; The control instructions include: locomotive departure time and stop time instructions; section speed limit and track allocation instructions; power system power adjustment instructions; The closed-loop control achieves dynamic strategy adjustment by comparing control instructions with the actual position, speed and energy consumption parameters of the locomotive.

9. The railway locomotive background intelligent dispatching control method based on multi-sensor fusion according to claim 8 is characterized in that: In step S4, the dynamic strategy adjustment of the closed-loop control includes: When the actual position or speed exceeds the threshold: the emergency braking protocol is immediately initiated; after emergency braking, steps S2-S3 are re-triggered within t≤10s to generate an emergency scheduling strategy; if no new strategy is generated within the timeout, the baseline scheduling strategy is enabled.

Images