Virtual marshalling-oriented air rail train distributed active-disturbance-rejection elastic control method
Through multi-agent modeling and adaptive ephemeral optimization methods, the distributed self-immune interference controller is designed, and the stability and comfort problems of virtual marshalling of air rail trains under external disturbances are solved, and the stability and comfort of virtual marshalling of air rail trains is realized. Rapid recovery and adaptive control of trains.
Patent Information
- Application Number
- CN202411152526.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-21
- Publication Date
- 2024-11-29
- Estimated Expiration
- 2044-08-21
AI Technical Summary
When the existing technology faces different degrees of external disturbance, it is impossible to achieve rapid recovery and stable formation operation of virtual marshalling of air rail trains, and the controller parameter adjustment is difficult to adapt, resulting in the stability and comfort of virtual marshalling of trains under interference. reduce.
The multi-agent modeling method is used to construct a distributed dynamic model, based on the two-way-pilot communication topological structure design coordination and collision avoidance control protocol, and optimize the hyperparameters of the depth deterministic strategy gradient algorithm using the adaptive ephemeral optimization method. The latter algorithm trains a distributed self-immune controller to realize adaptive control of virtual marshalling trains.
The rapid recovery of train virtual marshalling and the desired formation operation under interference are achieved, ensuring the stability and comfort of the train, and the controller parameters can be adaptively adjusted online to adapt to different degrees of external disturbances.
Smart Images

Figure CN119037511A_ABST
Abstract
Description
Technical Field
[0001] The invention relates to the field of skyrail train control, and in particular to a distributed self-disturbance rejection elastic control method for skyrail trains oriented to virtual formation. Background Art
[0002] In order to alleviate the problems associated with ground transportation, increase the capacity and speed of urban transportation, and effectively reduce the frequency of automobile use, many countries have focused on the development and application of suspended monorail transportation systems (i.e., skyrail).
[0003] The virtual coupling form of sky-rail train formation is an effective way to improve the flexibility and transportation efficiency of rail transit systems. Due to the complexity and time-varying nature of train operation scenarios, the resistance experienced by the train cannot be accurately expressed, which inevitably causes uncertain interference to the entire control system. Therefore, the tracking controllers of unit trains in the virtual formation need to be able to offset or constrain uncertain disturbances.
[0004] The following literature introduces some studies on disturbance suppression in train tracking control.
[0005] In “X.Wang, L.Zhu, H.Wang, et al, “Robust Distributed Cruise Control of Multiple High-Speed Trains Based on Disturbance Observer”, IEEE Trans. Intell. Transp. Syst., vol. 22, no. 1, pp. 267-279, 2021.”, in order to solve the problem of robust distributed cruise control of multiple high-speed trains under external disturbances, a disturbance observer is designed to approximate the disturbance. In “ZQ Long, Y.Li, W.Xu, “Research on Automatic Driving Algorithm for Maglev Trains Based on Active Disturbance Rejection Control”, Proceedings of the 27th China Control Conference two thousand and eight, 2008.”, based on the constraints of maglev train track operation and passenger comfort indicators, an active disturbance rejection control algorithm is proposed, and its feasibility and superiority are verified through simulation. Considering time-varying external disturbances. In “MG Suo, XR Li. “The application of disturbance observer in train speed prediction and tracking control”, Mechanical Science and Technology, vol. 293, no. 7, pp. 123-130, 2019.”, an optimal predictive control algorithm combined with a disturbance observer was designed to constrain uncertain disturbances, and finally the prediction and tracking of train speed was achieved. In “Y. Liu, Y. Zhou, S. Su, J. Xun, and T. Tang. “Tube-based control strategy for stable formation of high-speed virtually coupled trains with disturbances and delays”, Computer-Aided Civil and Infrastructure Engineering, vol. 38 no. 13 pp. 1093-9687, 2023.”, a Tube model predictive control strategy combining optimization and feedback control methods was designed to solve the complexity of the virtual train marshaling system under uncertain disturbances.In order to realize virtual coupling control of permanent magnet maglev trains, an effective cooperative tracking control method based on distributed active disturbance rejection control is proposed in “ZY Guo, ZQ Li, “Virtual coupling of permanent magnetic maglev trains: Animated cooperative tracking and collision avoidance control protocol”, IET Intell. Transp. Syst., pp. 1–14, 2023.” Compared with traditional control methods, this method has higher tracking accuracy and stronger anti-interference ability. However, these interference suppression methods have certain limitations and are only effective within uncertain interferences with small variations. If the disturbance is large, the controller parameters cannot be readjusted online, and the stability of the train speed control will deteriorate, resulting in reduced comfort. Therefore, designing an effective and resilient distributed cooperative train controller has become a pressing issue in the transportation field.
[0006] There are few studies on online parameter adjustment methods for elastic controllers in the literature. In “A. Savran, “A multivariable predictive fuzzy PID control system”, “Applied Soft Computing”, vol. 13, no. 5, pp. 2658-2667, 2013.”, a fuzzy adaptive proportional integral derivative (PID) controller is designed, which adjusts its parameters by querying the fuzzy matrix table. In “CC Li, CF Zhang, “Adaptive PID control based on minimum resource allocation network”, Application Research of Computers, vol. 32, no. 1, pp. 167-169, 2015.”, a neural network-based adaptive PID control method is proposed, which uses the good approximation ability of neural networks to achieve effective control without the need to identify complex nonlinear controlled objects. An adaptive PID controller based on evolutionary algorithm was designed in “JL Hang, YPLi, XPMa, “PID optimization design based on improved differential evolution algorithm”, Control Engineering, vol. 17, no. 6, pp. 807-810, 2010.” Although this method requires some prior knowledge, it cannot achieve real-time control in practical engineering applications. A PID parameter adjustment method for attitude control of cruise missile based on reinforcement learning was proposed in “JJ Liu, MR Hao, MW Sun, X. Guo, ZQ Chen, “PID parameter adjustment method for attitude control of cruise missile based on reinforcement learning”, Tactical Missile Technology, vol. 5, no. 1, pp. 58-63, 2019.” The evaluation network is used to recursively approximate the control objective function, and the action network is used to obtain the PID control parameters.In “JHZhang, “Research on adaptive PID control strategy based on Actor-Critic learning”, YanShan University, 2018.”, an adaptive PID control strategy based on an actor-critic (AC) structure and a radial basis function neural network is proposed. The controller uses the AC structure to adjust its parameters online and uses a neural network to approximate the value and decision function in the Markov decision process. In “QFSun, H.Ren, YXDuan, “Adaptive PID control design based on asynchronous dominant actuator evaluator learning”, Information and Control, vol.48, no.3, pp.70-74, 2019.”, a new adaptive PID controller based on the asynchronous dominant actuator evaluator (A3C) algorithm is proposed. The controller uses the multi-threaded asynchronous learning capability of the A3C algorithm to train agents based on multiple actuator evaluator structures in parallel. Among the above methods, evolutionary algorithms perform offline parameter adjustment with poor real-time performance, while other methods involve online parameter adjustment but require optimization of multiple hyperparameters.
[0007] Moreover, the distributed cooperative control method proposed in the document "ZY Guo, ZQ Li, "Distributed auto disturbances rejection resilient control of permanent magnetic maglev trains based on the optimized deep deterministic policy gradient algorithm", IET Control Theory and Applications, 2024." only considers the running status information of the adjacent front and rear vehicles, which can accelerate the restoration of a stable formation to a certain extent, but the input of the status information of the front and rear vehicles needs to be weighed. If the controller is not designed properly, it is easy to produce speed and spacing oscillations, and the stability of the queue cannot be guaranteed while taking into account the stability of the single vehicle. In addition, if the formation state deviation of the adjacent rear vehicles is too large, the stability of the queue will deteriorate.
[0008] In summary, the existing technology has the pain points of too many hyperparameters and difficult tuning. When encountering external disturbances of varying degrees, it is impossible to achieve adaptive adjustment control, and the existing train tracking control method cannot be effectively applied. The above shortcomings lead to the inability of the existing technology to ensure the rapid recovery of the virtual formation of trains under interference and maintain the desired formation operation. Summary of the invention
[0009] The purpose of the present invention is to provide a distributed self-disturbance rejection elastic control method for sky-rail trains oriented to virtual formation, which can ensure that the virtual formation of trains can be quickly restored under interference and maintain the expected formation operation.
[0010] To achieve the above object, the present invention provides the following solution.
[0011] A distributed self-disturbance rejection elastic control method for skytrains in virtual formation, comprising: using a multi-agent modeling method to construct a distributed dynamics model of skytrains in virtual formation; determining a formation state error model of the virtual formation according to the expected relative speeds and expected running intervals between adjacent trains and a pilot train in the virtual formation; using a tanh function to smooth the formation state errors of adjacent trains according to the distributed dynamics model and the formation state error model, and establishing a coordination and collision avoidance control protocol based on a two-way-pilot communication topology structure; constructing a coordination and collision avoidance control protocol based on a coordination and collision avoidance control protocol. A distributed auto-disturbance rejection controller of the protocol; an adaptive mayfly optimization method is used to optimize the hyperparameters of the deep deterministic policy gradient algorithm; the adaptive mayfly optimization method adds an mayfly adaptive mutation mechanism based on Steffensen; the distributed auto-disturbance rejection controller is trained using the optimized deep deterministic policy gradient algorithm to obtain a trained distributed auto-disturbance rejection controller; the parameters of the trained distributed auto-disturbance rejection controller are adaptively adjusted online according to time-varying disturbances, and the sky-rail trains in the virtual formation are controlled so that the sky-rail trains in the virtual formation maintain the desired formation under the interference.
[0012] Optionally, a multi-agent modeling method is used to construct a distributed dynamic model of an ASR train in a virtual marshaling, specifically including: considering a multi-agent virtual marshaling system consisting of N trains, each train is a multi-agent rigid point, numbered 0 to N-1 in sequence along the running direction, and the train numbered 0 is used as the leader train, the trains numbered 1 to N-1 are used as follower trains, and the follower trains only receive the status information of the front and rear trains in their respective neighborhoods; wherein, let E represent the set of all trains, E={0,1,…,N-1}; F represents the set of follower trains, F={1,2,3,…,N-1}; G represents the set of neighborhood trains of train number i, G={i-1,i+1}; the train is an ASR train; the distributed dynamic model of the ASR train in the multi-agent virtual marshaling system constructed by the multi-agent modeling method is: In the formula, x i (t) represents the position of train number i at time t, Represents x i The differential of (t), v i (t) represents the speed of train number i at time t, Indicates v i The differential of (t), f(v i ,t) represents the resistance experienced by train number i at time t, b represents the controller coefficient, u i (t) represents the traction / braking force of train number i at time t, represents the external environmental interference experienced by train number i at time t, α v v i (t) represents the known system linear term, α v represents the system linear term coefficient, represents the nonlinear internal disturbance with uncertain system parameter perturbation, σ i (v i (t),t) represents the total disturbance suffered by the system at time t.
[0013] Optionally, according to the distributed dynamics model and the formation state error model, the formation state errors of adjacent trains are smoothed using a tanh function, and a coordination and collision avoidance control protocol based on a two-way-pilot communication topology is established, specifically including: designing a tracking differentiator that enables smooth acceleration or smooth deceleration of the train as: Where, d i,0 (t) represents the actual relative state between train number i and the leader train at time t, d i,0 (t) = [x i,0 (t),v i,0 (t)] T , x i,0(t) represents the actual interval between train number i and the leader train at time t, v i,0 (t) represents the actual relative speed between train number i and the leader train at time t; Indicates d i,0 (t) jump to A step input signal, Represents x i,0 (t) jump to A step input signal, represents the expected interval between train number i and the leader train at time t, Indicates v i,0 (t) jump to A step input signal, represents the expected relative speed between train number i and the leader train at time t; represents the expected relative state between train number i and the leader train at time t, express The transition signal, Indicates the running interval transition signal at time t, Relative speed transition signal at time t; and Both said The differential signal of express The differential signal; r0, r1 both represent the transition speed of the signal, h0, h1 both represent the terms that filter the noise of the signal; fhan() represents the fhan function; in order to observe all unknown terms including internal interference and external interference, a nonlinear extended state observer based on a high-order sliding mode differentiator is designed as follows: Where ψ1(χ1) represents the estimated value of the train speed χ1, χ1 = v i , v i represents the current speed of train number i; ψ2(χ1) represents the estimated value of the total system interference χ2, χ2 = σ i (v i ), σ i (v i ) represents the total disturbance of the system; e1 represents the error of train speed, and e2 represents the error of total disturbance of the system; represents the differential of ψ1(χ1), represents the differential of ψ2(χ1); η1(χ1) represents the internal state of the nonlinear extended observer; f(v i ) represents the resistance of train number i; b represents the controller coefficient, u irepresents the traction / braking force of train number i; a1, b1, q1 and p1 are all adjustable observer parameters, a1>0, b1>0, 0<q1<p1; sgn() represents the sign function; the improved repulsive field function is introduced as: In the formula, represents the improved repulsive field of the neighboring train numbered j relative to the train numbered i, ρ j represents the radius of the repulsive field of the neighborhood train numbered j, r j represents the actual size radius of the neighborhood train numbered j, μ represents the repulsive force scale factor, and ρ i,j represents the actual interval between train number i and its neighboring train number j, Δρ i,j Represents ρ i,j With r j The deviation of ρ j With r j The deviation represents the expected interval between train number i and its neighboring train number j, Represents ρ i,j and The absolute value of the deviation, F represents the set of follower trains, G represents the set of neighboring trains of train number i; ρ j represents the repulsive field radius of the neighboring train numbered j; according to the improved repulsive field function, the repulsive force exerted on each train by a neighboring train is: In the formula, represents the repulsive force exerted by the neighboring train numbered j on the train numbered i, i∈F, j∈G, n j→i The unit repulsive force vector of the neighboring train numbered j pointing to the direction of the train numbered i, n i→B represents the unit vector that moves the train number i in the desired formation direction B, Indicates train number i n j→i The repulsive force in the direction Indicates train number i n i→B The repulsive force on each train in the direction of the train; according to the repulsive force exerted by a neighboring train on each train, the total repulsive force on each train from the neighboring trains in front and behind is determined as: In the formula, F i rep It indicates the total repulsive force of the train numbered i on the neighboring trains in front and behind, i∈F; It means that the train number i is repelled by the neighboring train in front. It indicates that the train numbered i is subject to the repulsive force exerted by the neighboring train behind; the repulsive force exerted by each train by a neighboring train and the total repulsive force exerted by each train by the neighboring trains in front and behind constitute collision avoidance control, and the collision avoidance control, the tracking differentiator and the nonlinear extended state observer together constitute a collaborative and collision avoidance control protocol.
[0014] Optionally, the Steffensen-based adaptive mutation mechanism of mayflies specifically includes: taking the ratio of the number of weight nodes adjacent to the current mayfly optimal solution to the total number of weight nodes as the mayfly population diversity evaluation index; if the diversity evaluation index is less than a preset threshold, the Steffensen method is used, according to the formula Mutate the mayfly population; where X 1,k represents the kth dimension of the ephemera X1 to be mutated, X 1,k ′ is X 1,k The value after mutation; θ(X 1,k ) indicates the distance between X1 and Z1 and X 1,k The deviation of the expected distance between ′ and Z1, θ(X 1,k )=L(X1,Z1)-l * , L(X1,Z1) represents the distance between the mayfly individual X1 to be mutated and the weight node Z1, l * Represents X 1,k ′ and the expected distance between Z1, l * =α·L(X1,Z1),α∈[0,1], where α represents a random number in the interval [0,1].
[0015] Optionally, according to the expected relative running speed and expected running interval between adjacent trains and the leader train in the virtual formation, the formation state error model of the virtual formation is determined, and then the following assumptions are made: Assumption 1: Assume that each train obtains the actual speed and position information of the front and rear trains and the leader train in the neighborhood in real time through vehicle-to-vehicle wireless communication; Assumption 2: Assume that the initial time is t0, and the initial state deviation between the trains in the neighborhood satisfies ||ξ i (t0)-ξ j (t0)||≤δ1,j∈G, the external disturbance to the system satisfies||σ i (t)||≤δ2; where δ1,δ2 are both bounded positive constants, ξ i (t0) represents the state vector of train number i at the initial time t0, ξ j (t0) represents the state vector of the neighborhood train numbered j at the initial time t0, σ i (t) represents the external disturbance to the system at time t; Assumption 3: Assume that the actual speed of the leader train is consistent with the target speed trajectory; The final control target of multi-train virtual marshaling collaborative tracking is expressed as: in, and They are all extremely small positive values, and v0 represents the speed of train number i at the initial moment.
[0016] According to the specific embodiments provided by the present invention, the present invention discloses the following technical effects.
[0017] The embodiment of the present invention uses an adaptive ephemera optimization method to optimize the hyperparameters of a deep deterministic policy gradient algorithm, and uses the optimized deep deterministic policy gradient algorithm to continuously train a distributed controller based on a cooperative and collision avoidance control protocol, thereby obtaining a trained distributed self-disturbance rejection elastic controller to control each unit train of the virtual formation. The trained distributed self-disturbance rejection elastic controller has good self-disturbance rejection performance. When encountering external disturbances of varying degrees, the parameters of the trained distributed self-disturbance rejection elastic controller can be adaptively adjusted online to ensure that the virtual formation of trains can recover quickly and maintain the desired formation under the influence of interference. BRIEF DESCRIPTION OF THE DRAWINGS
[0018] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0019] Figure 1 A schematic flow chart of a distributed adaptive disturbance rejection elastic control method for a sky-rail train with virtual formation provided by an embodiment of the present invention.
[0020] Figure 2 A schematic diagram of virtual grouping provided by an embodiment of the present invention.
[0021] Figure 3 A schematic diagram of a weight node provided by an embodiment of the present invention.
[0022] Figure 4 A schematic diagram of mayfly mutation based on Steffensen provided in an embodiment of the present invention.
[0023] Figure 5 A schematic diagram of the distance L(X1, Z1) provided in an embodiment of the present invention.
[0024] Figure 6 A schematic diagram of a virtual formation in a simulation experiment provided in an embodiment of the present invention.
[0025] Figure 7 A schematic diagram of iterative results provided by an embodiment of the present invention.
[0026] Figure 8 A schematic diagram of the noise standard deviation provided in an embodiment of the present invention.
[0027] Fig. 9 A schematic diagram of the noise standard deviation attenuation rate provided in an embodiment of the present invention.
[0028] Fig.10A schematic diagram of a network learning rate provided by an embodiment of the present invention.
[0029] Fig.11 A schematic diagram of a reward function curve provided by an embodiment of the present invention.
[0030] Fig.12 A schematic diagram of the actor network structure provided by an embodiment of the present invention.
[0031] Fig.13 A schematic diagram of a critic network structure provided in an embodiment of the present invention.
[0032] Fig.14 A schematic diagram of round report results provided by an embodiment of the present invention.
[0033] Fig.15 A schematic diagram of average return results provided by an embodiment of the present invention.
[0034] Fig.16 A schematic diagram of the control capability verification result provided by an embodiment of the present invention.
[0035] Fig.17 A schematic diagram of the generalization capability verification result provided by an embodiment of the present invention.
[0036] Fig.18 A schematic diagram of train speed change provided by an embodiment of the present invention.
[0037] Fig.19 A schematic diagram of train speed change provided by an embodiment of the present invention.
[0038] Fig. 20 A schematic diagram of train control parameter changes provided by an embodiment of the present invention.
[0039] Fig.21 A schematic diagram of changes in train running intervals provided in an embodiment of the present invention. DETAILED DESCRIPTION
[0040] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0041] Since the external interference to the train during operation is uncertain and time-varying, conventional distributed controllers are difficult to meet the requirements of stable control of the virtual formation of sky-rail trains. The present invention relates to a distributed self-disturbance rejection elastic control method and system for sky-rail trains for virtual formation. A multi-agent modeling method is first used to construct a distributed dynamic model of the sky-rail train, and the formation state error is calculated based on the expected running speed and running interval of the unit train. Based on the multi-agent train model and the formation state error, a collaborative and collision avoidance control protocol based on a two-way-pilot communication topology is designed for the virtual formation unit train. The Adaptive Mayfly algorithm (AMA) is used to optimize the hyperparameters of the Deep Deterministic Policy Gradient Algorithm (DDPG). The optimized DDPG algorithm is used to continuously train the distributed controller based on the cooperative and collision avoidance control protocol. The trained distributed self-disturbance rejection elastic controller is obtained to control each unit train of the virtual formation. This method has good self-disturbance rejection performance. When encountering external disturbances of different degrees, the system controller parameters can be adaptively adjusted online to ensure that the virtual train formation can quickly recover and maintain the expected formation under the influence of interference.
[0042] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.
[0043] like Figure 1 As shown, the distributed self-disturbance rejection elastic control method for virtual formation of sky-rail trains in this embodiment includes the following steps 1 to 7.
[0044] Step 1: Use multi-agent modeling method to build a distributed dynamic model of virtual skyrail train.
[0045] Considering the internal interference of system parameter perturbations and the uncertain external environmental interference, a multi-agent modeling method is adopted to construct the sky-rail train dynamics model based on virtual marshaling.
[0046] Consider a multi-agent virtual marshaling system consisting of N skyrail trains. Each train is a multi-agent rigid point, numbered 0 to N-1. Let the train numbered 0 be the leader train; the trains numbered 1 to N-1 be the follower trains, and the follower trains only receive the status information of the trains in their respective neighborhoods. The interaction between trains can be described by a directed graph. Let E = {0, 1, ..., N-1} represent the set of all trains, F = {1, 2, 3, ..., N-1} represent the set of follower trains, and G = {i-1, i+1} represent the set of neighboring trains of train number i. The dynamic model of the skyrail train for multi-agents is expressed as formula (1).
[0047]
[0048] Where i∈E, x i ,v i ,u i ,f i and M i They represent the current position, speed, traction / braking force, resistance and train mass of train number i. The resistance f i By the basic resistance f bi , additional resistance of curve f wi 、 Additional resistance of ramp f si and the friction force f between the running wheel and the track ci Among them, the additional resistance of the curve f wi 、 Additional resistance of ramp f si are related to the curve radius and the slope angle respectively; and the basic resistance f bi It has a nonlinear relationship with the train speed and can be expressed as formula (2) according to the empirical formula.
[0049]
[0050] Where α0, α1 and α2 are resistance coefficients, all of which change with the changes in the train running environment, and g is the acceleration of gravity.
[0051] Since the skyrail train is inevitably affected by the external environment during operation, the system model has the characteristics of time-varying, uncertainty and nonlinearity. Therefore, the multi-agent train model can also be expressed as follows.
[0052]
[0053] In the formula, x i (t) represents the position of train number i at time t, Represents x i The differential of (t), v i(t) represents the speed of train number i at time t, Indicates v i The differential of (t), f(v i ,t) represents the resistance experienced by train number i at time t, b represents the controller coefficient, u i (t) represents the traction / braking force of train number i at time t, represents the external environmental interference experienced by train number i at time t, α v v i (t) represents the known system linear term, α v represents the system linear term coefficient, represents the nonlinear internal disturbance with uncertain system parameter perturbation, σ i (v i (t),t) represents the total disturbance suffered by the system at time t.
[0054] Step 2: Determine the formation state error model of the virtual formation according to the expected relative speed and expected running interval between the adjacent follower trains and the leader train in the virtual formation.
[0055] This paper studies the problem of flexible control of virtual marshaling of sky-rail trains based on leader-follower structure under uncertain disturbance. The train controller is continuously trained by deep reinforcement learning method, so that the controller parameters can be continuously adjusted online with the change of disturbance to ensure the stable control of the virtual marshaling formation of the train. Figure 2 shown. Figure 2 In it, LT is leader train, FT is follower train, and h is the train length.
[0056] In the virtual formation, the leader train runs according to the target speed trajectory given by the ground signal control center, while the follower train regards the actual speed of the preceding train as the expected running speed, thus forming a stable virtual formation.
[0057] The state vector of train number i is expressed as formula (4).
[0058] ξ i =[x i ,v i ] T ,i∈E (4).
[0059] The expected relative state between train number i and neighboring train number j Relative to the actual state d i,j It is formula (5).
[0060]
[0061] Wherein, when i∈F, the train numbered i is the follower train. represents the expected relative state between train number i and its neighboring train number j, d i,j represents the actual relative state between the train numbered i and the neighboring train numbered j, || represents the absolute value symbol, F represents the set of follower trains, F = {1,2,3,…,N-1}; G represents the set of neighboring trains of the train numbered i, G = {i-1,i+1}, j = i-1 or j = i+1. represents the expected interval between train number i and its neighboring train number j, x i,j represents the actual interval between train number i and its neighboring train number j, represents the expected relative speed between train number i and neighboring train number j, generally v i,j represents the actual relative speed between train number i and its neighboring train number j, x i represents the current position of train number i, x j represents the current position of the train in the neighborhood of number j, v i represents the current speed of train number i, v j represents the current speed of the train in the neighborhood of number j, and the superscript T represents transposition. It can be expressed as formula (6).
[0062]
[0063] Among them, S t represents the safety margin at time t, S brake (v i ) indicates that the speed of train number i is v i Braking distance at , S brake (v j ) indicates that the speed of the train in the neighborhood of number j is v j When the speeds of the leader train and the follower train are the same, the expected running interval is Only affected by the train length h and the safety margin S at time t t The expected relative state between train number i and the leader train Relative to the actual state d i,0 It is formula (7).
[0064]
[0065] In the formula, represents the expected interval between train number i and the leader train, x i,0 represents the actual interval between train number i and the leader train, represents the expected relative speed between train number i and the leader train, v i,0 represents the actual relative speed between train number i and the leader train, x0 represents the current position of the leader train, and v0 represents the current speed of the leader train.
[0066] The formation state error model can be expressed as Equation (8) and Equation (9).
[0067]
[0068] In the formula, e i,j represents the formation state error between train number i and its neighboring train number j, e i,0 Represents the formation state error between train number i and the leader train.
[0069] For the convenience of research, the following assumptions are made without loss of generality.
[0070] Assumption 1: Assume that train number i (i∈F) obtains the position and speed of the two trains in front and behind and the leader train in the neighborhood in real time through vehicle-to-vehicle wireless communication with low latency and low packet loss rate.
[0071] Assumption 2: Assume that the initial time is t0, the initial state deviation between the follower trains in the neighborhood ||ξ i (t0)-ξ j (t0)||≤δ1,j∈G, the external disturbance to the system||σ i (t)||≤δ2, where δ1,δ2 are bounded positive constants. ξ i (t0) represents the state vector of train number i at the initial time t0, ξ j (t0) represents the state vector of the neighborhood train numbered j at the initial time t0, σ i (t) represents the external disturbance to the system at time t.
[0072] Represents the norm.
[0073] Assumption 3: Assume that the leader train speed is guaranteed to be consistent with the target speed trajectory.
[0074] The final control objective of multi-train virtual marshaling cooperative tracking can be expressed as formula (10). In formula (10), and are all minimum positive values. v0 represents the speed of train number i at the initial moment.
[0075]
[0076] Step 3: According to the distributed dynamics model and the formation state error model, the formation state errors of adjacent trains are smoothed using the tanh function, and a coordination and collision avoidance control protocol based on a two-way-pilot communication topology is established.
[0077] Due to the uncertain external disturbance and internal disturbance of system parameter perturbation, conventional control methods are difficult to meet the stability and comfort requirements of virtual train coordinated operation. Therefore, a distributed active disturbance rejection control strategy is proposed.
[0078] 3.1 Tracking differentiator design.
[0079] Since the initial formation of multiple trains has a large deviation from the expected formation, the acceleration of the follower train is too large during the dynamic formation stage, and the train ride comfort deteriorates. The tracking differentiator (TD) is designed to provide a transition process for the train state deviation. The state deviation increases smoothly from zero, and the train accelerates or decelerates smoothly, which improves the train ride comfort. The formation state deviation consists of two parts: the running interval deviation and the speed deviation.
[0080] The train number i (i∈F) tracks the differentiator as shown in equation (11).
[0081]
[0082] Where, d i,0 (t) = ξ i (t)-ξ0(t) represents the actual relative state between the follower train No. i and the leader train at time t; d i,0 (t) = [x i,0 (t),v i,0 (t)] T , x i,0 (t) represents the actual interval between train number i and the leader train at time t, v i,0 (t) represents the actual relative speed between train number i and the leader train at time t; express The differential signal. Indicates d i,0 (t) jump to A step input signal, Represents x i,0 (t) jump to A step input signal, Indicates v i,0 (t) jump to A step input signal, represents the expected relative speed between train number i and the leader train at time t.
[0083] represents the expected relative state between train number i and the leader train at time t, express The transition signal, Indicates the running interval transition signal at time t, Relative speed transition signal at time t; and Both said The differential signal of express The differential signal; r0, r1 both represent the transition speed of the signal, h0, h1 both represent the terms that filter the noise of the signal; fhan() represents the fhan function, The specific form of is as shown in formula (12).
[0084]
[0085] In the formula, s, y, w0, w1, w2, s y ,s a It is the intermediate variable of the fhan function.
[0086] 3.2 Design of extended state observer.
[0087] The purpose of designing an extended state observer (ESO) is to observe all unknown items including internal and external disturbances. However, the accuracy of ESO observation disturbances will determine the effect of distributed cooperative operation of multiple trains. Therefore, it is particularly important to design an ESO with better observation performance. Conventional ESO parameters are severely coupled, resulting in repeated oscillations, and there is a disadvantage that the convergence time gradually increases with the increase of initial error. Therefore, a nonlinear ESO based on a high-order sliding mode differentiator is designed as shown in Equation (13).
[0088]
[0089] Where ψ1 and ψ2 represent the train speed χ1=v i And the total system interference χ2=σ i (v i ). For any initial error e1(t0), e2(t0), there is always a small error interval that e1(t), e2(t) converge to in a finite time. ψ1(χ1) represents the estimated value of the train speed χ1, χ1 = v i , v i represents the current speed of train number i; ψ2(χ1) represents the estimated value of the total system interference χ2, χ2 = σ i (v i ), σ i (vi ) represents the total disturbance of the system; e1 represents the error of train speed, and e2 represents the error of total disturbance of the system; represents the differential of ψ1(χ1), represents the differential of ψ2(χ1); η1(χ1) represents the internal state of the nonlinear extended observer; f(v i ) represents the resistance of train number i; b represents the controller coefficient, u i represents the traction / braking force of train number i; a1, b1, q1 and p1 are all adjustable observer parameters, a1>0, b1>0, 0<q1<p1; sgn() represents the sign function.
[0090] 3.3 Collision avoidance control.
[0091] When multiple SkyRail trains operate in a virtual marshaling mode, the biggest safety hazard is the collision between trains. Therefore, it is very necessary to design an effective collision avoidance control strategy. Here, we draw on the idea of artificial potential field method to achieve effective collision avoidance in the coordinated operation of multiple trains.
[0092] Considering the actual size of the train, we use ρ j represents the radius of the repulsive field of the neighborhood train numbered j, r j represents the actual radius of the neighboring train numbered j. When the distance between the train numbered i and the neighboring train numbered j is less than ρ j When the train number i is repelled, and the distance is equal to or less than r j When , it is considered that the two trains collided. The traditional repulsive field function of the artificial potential field method has certain limitations. When there are obstacles near the target point, there will be a problem that the target is unreachable. Therefore, an improved repulsive field function is introduced as shown in formula (14).
[0093]
[0094] In the formula, represents the improved repulsive field of the neighboring train numbered j relative to the train numbered i, r j represents the actual size radius of the neighborhood train numbered j, μ represents the repulsive force scale factor, and ρ i,j represents the actual interval between train number i and its neighboring train number j, Δρ i,j Represents ρ i,j With r j The deviation of ρ j With r j The deviation represents the expected interval between train number i and its neighboring train number j, Represents ρ i,j and The absolute value of the deviation, F represents the set of follower trains, and G represents the set of neighboring trains of train number i.
[0095] According to the improved repulsive field function, the repulsive force exerted on each train by a neighboring train is obtained as shown in Equation (15) and Equation (16).
[0096]
[0097] In the formula, represents the repulsive force exerted by the neighboring train numbered j on the train numbered i, i∈F, j∈G, n j→i The unit repulsive force vector of the neighboring train numbered j pointing to the direction of the train numbered i, n i→B represents the unit vector that moves the train number i in the desired formation direction B, Indicates train number i n j→i The repulsive force in the direction Indicates train number i n i→B The repulsive force in the direction.
[0098] Based on the repulsive force exerted on each train by a neighboring train, the total repulsive force exerted on each train by the front and rear neighboring trains is determined as follows.
[0099]
[0100] In the formula, F i rep It indicates the total repulsive force of the train numbered i on the neighboring trains in front and behind, i∈F; It means that the train number i is repelled by the neighboring train in front. It means that the train numbered i is subject to repulsion from the neighboring train behind it.
[0101] The repulsive force exerted by a neighboring train on each train and the total repulsive force exerted by the front and rear neighboring trains on each train constitute collision avoidance control. The collision avoidance control, the tracking differentiator and the nonlinear extended state observer together constitute a coordination and collision avoidance control protocol.
[0102] Step 4: Construct a distributed active disturbance rejection controller based on the cooperative and collision avoidance control protocol.
[0103] The multi-train cooperative control problem in the virtual marshaling mode is transformed into a multi-agent consistency problem. In order to better eliminate the adverse effects of interference on the system, the distributed anti-disturbance cooperative controller of train number i is as follows.
[0104]
[0105] In the formula, e i,0 represents the formation state error between train number i and the leader train, ei,j represents the formation state error between train number i and its neighboring trains, express The transition signal, represents the expected relative state between train number i and the neighboring train ahead, i∈F; ξ i represents the state vector of train number i, ξ i =[x i ,v i ] T , x i Indicates the current position of train number i, v i represents the current speed of train number i; ξ0 represents the state vector of the leader train, ξ j represents the neighborhood train state vector; represents the coordinated part of train number i, k, and Both represent the adjustable coefficient matrix of the coordination and collision avoidance control protocol, ψ2 represents the estimated value of the total interference of the system; k = [k1, k2], k1 represents the position component of k, k2 represents the velocity component of k, express The component in the position direction, express The component in the direction of the velocity, express The component in the position direction, express The component in the direction of the speed; b represents the controller coefficient; u i Indicates the traction / braking force of train number i, F i rep represents the total repulsive force on train number i from the neighboring trains in front and behind, t represents time t, tanh() represents the hyperbolic tangent function, and t represents time t. ψ2 is used as the total disturbance compensation term. F i rep From equation (17), we get: Equation (18) is related to equation (11) as the transition signal Equation (18) is associated with equation (13) by the total disturbance compensation term ψ2.
[0106] Step 5: Optimize the hyperparameters of the deep deterministic policy gradient algorithm using an adaptive mayfly optimization method; the adaptive mayfly optimization method adds an mayfly adaptive mutation mechanism based on Steffensen.
[0107] Since the disturbances to the system are time-varying and uncertain, it is difficult for fixed distributed controller parameters to adapt to meet actual control requirements. Therefore, the present invention proposes a distributed active disturbance rejection control algorithm based on AMA-DDPG to improve the adaptive anti-disturbance capability of the system.
[0108] 5.2 Hyperparameter optimization based on adaptive mayfly optimization method.
[0109] Since the DDPG algorithm has too many hyperparameters and is difficult to tune, an adaptive ephemera optimization method is designed to optimize multiple hyperparameters of the DDPG algorithm in the search space. Since the noise standard deviation, noise standard deviation decay rate and network learning rate have a great influence on the training effect of the DDPG algorithm, they are taken as hyperparameter optimization objects.
[0110] If in a D-dimensional target search space, a mayfly population Q is constructed by Y mayflies, where the number of male and female mayflies is Y / 2. i′ =(xm i′,1 ,xm i′,2 ,…,xm i′,D ) represents the position of the ith male mayfly; Vm i′ =(vm i′,1 ,vm i′,2 ,…vm i′,D ) represents the search speed of the ith male mayfly; Xf i′ =(xf i′,1 ,xf i′,2 ,…,xf i′,D ) represents the position of the i′th female mayfly; Vf i′ =(vf i′,1 ,vf i′,2 ,…,vf i′,D ) represents the search speed of the i′th female mayfly, i′=1,2,…,Y.
[0111] Since the conventional Mayfly algorithm (MA) has the defect of being easily trapped in a local optimal point, the present invention proposes an adaptive MA algorithm. The adaptive MA algorithm is mainly embodied in the adaptive mutation mechanism of Steffensen's Mayfly algorithm.
[0112] 5.2.1 Status updates and mating and reproduction behaviors of male and female mayflies.
[0113] 5.2.1.1 Male mayfly status update.
[0114] The position of the i′th male mayfly in the nth generation population is updated as follows:
[0115]
[0116] In the formula, They represent the j′th dimension position and velocity of the male mayfly, i′=1,2,…,Y,j′=1,2,…,D.
[0117] If the male mayfly is not the best individual in the mayfly population, its speed is updated as follows:
[0118]
[0119] Where pbest i′,j′ represents the individual optimal point searched by the ith male mayfly in the nth generation population, gbest represents the global optimal point searched by the ith male mayfly in the nth generation population, λ1, λ2, β are constants, r p is the current position and pbest i′,j′ The Euclidean distance, r g is the Euclidean distance between the current location and gbest.
[0120] If the male mayfly is the best individual in the mayfly population, its speed is updated as follows:
[0121]
[0122] Where λ is a constant and r1 is a random number in [-1,1].
[0123] 5.2.1.2 Female mayfly status update.
[0124] The position of the i′th female mayfly in the nth generation population is updated as follows:
[0125]
[0126] In the formula, They represent the j′-th dimension position and velocity of the female mayfly respectively.
[0127] If the female mayfly is worse than the paired male, its speed is updated as follows:
[0128]
[0129] In the formula, xm i ″ ,j′ The position of the male mayfly that the female mayfly is paired with, is the position of the female mayfly, r mf The position between the male and female mayflies.
[0130] If the female mayfly is superior to the paired male, its speed is updated as follows:
[0131]
[0132] In the formula, fl is a constant and r2 is a random number in [-1,1].
[0133] 5.2.1.3 Mating and reproductive behavior.
[0134] In the mating behavior of mayflies, a pair of female and male mayflies will produce two offspring, and the two offspring produced can represent:
[0135]
[0136] Where L′ is a random number in the range of (0,1). and represents the two offspring of a pair of male and female mayflies in the nth generation of mayfly population. The mating behavior of mayflies provides a global search capability for the mayfly algorithm. Selecting better individuals from the offspring can speed up the convergence of the algorithm.
[0137] 5.2.2 Adaptive mutation mechanism of mayfly based on Steffensen.
[0138] In the process of searching for the global optimum point by the conventional mayfly optimization algorithm, the population diversity of mayflies will become worse and worse, and premature maturity may occur. Therefore, the present invention proposes an adaptive mutation mechanism of mayflies based on Steffensen.
[0139] 5.2.2.1 Diversity evaluation indicators based on weight nodes.
[0140] The present invention proposes a diversity evaluation index based on weight nodes to evaluate the distribution diversity of mayfly populations. The core idea of the evaluation index is to select a group of weight nodes evenly distributed in a one-dimensional target space. The current distribution of mayfly populations is evaluated by calculating the ratio κ of the number of weight nodes adjacent to the current mayfly optimal solution and the total number of weight nodes.
[0141] The weight node diagram is as follows Figure 3 As shown in the figure, Z1~Z3 represent a group of weight nodes evenly distributed in the one-dimensional target space R (adaptive space), and X1~X3 are the current mayfly individuals. It can be seen that X1~X3 are closest to the Z2 node, and the only adjacent weight node in the entire mayfly population is Z2. Therefore, the current κ value is 1 / 3. κ∈[0,1]. The smaller the κ value, the more concentrated the distribution of the mayflies in the current population.
[0142] 5.2.2.2 Adaptive mayfly mutation strategy based on Steffensen method.
[0143] The Steffensen method is an iterative root-finding method for solving the nonlinear equation f(x) = 0. The Steffensen formula is as follows.
[0144]
[0145] k=0,1,2…
[0146] In the formula, x0 is the initial root value, x k and x k+1 are the root values before and after the k+1th iteration respectively.
[0147] The present invention adopts a mutation strategy based on the Steffensen method to enhance the diversity of the mayfly population. If the κ index of the mayfly population is less than a given threshold ξ, each mayfly in the population is mutated.
[0148] like Figure 4 As shown in the figure, consider the mayfly individual X1. First, a weight node (assuming it is Z1) is randomly selected from the nearest weight nodes. Then, with the goal of shortening the distance L(X1, Z1) between the mayfly individual X1 and Z1, the mayfly individual X1 is mutated to obtain the mutant mayfly individual X1′ that is closer to Z1. By performing the above operations on the mayfly individuals in the population one by one, a group of mutant mayfly individuals that are evenly distributed in the target space are obtained.
[0149] like Figure 5 As shown, the operation process of the Steffensen method is explained by taking the mayfly individual X1 to be mutated and the corresponding weight node Z1 as an example.
[0150] First, according to Figure 4 The distance between the mayfly individuals X1 and Z1 is calculated to be L(X1, Z1) = l. The smaller L(X1, Z1), the closer the mayfly individual X1 is to Z1. Next, we calculate l * =α·L(X1,Z1),α∈[0,1], obviously, l * <L(X1,Z1). Randomly mutate one dimension of the mayfly individual X1, so that the mutated mayfly individual X1′ satisfies L(X1,Z1)-l * = 0. If the kth dimension of the mayfly individual X1 is selected for mutation, it is equivalent to the nonlinear equation L(X 1,k ,Z1)-l * = 0 to solve the problem. Use the Steffensen method to solve L(X 1,k ,Z1)-l * = 0, the first iteration approximate solution is taken as the mayfly individual X 1,k According to the iterative formula of Steffensen method, the value of mayfly individual X is obtained.1,k The mutated values are as follows.
[0151]
[0152] Where, X 1,k represents the kth dimension of the mayfly individual X1 to be mutated, X 1,k ′ is X 1,k The value after mutation; θ(X 1,k ) represents the distance between the mayfly individuals X1 and Z1 to be mutated and X 1,k The deviation of the expected distance between ′ and Z1, θ(X 1,k )=L(X1,Z1)-l * , L(X1,Z1) represents the distance between the mayfly individual X1 to be mutated and the weight node Z1, l * Represents X 1,k ′ and the expected distance between Z1, l * =α·L(X1,Z1),α∈[0,1], where α represents a random number in the interval [0,1].
[0153] 5.3AMA optimization process.
[0154] The AMA optimization method is used to optimize the four hyperparameters of the DDPG algorithm in the specified search space, namely the noise standard deviation σ, the noise standard deviation decay rate ψ, and the Actor network learning rate α. u and Critic network learning rate α Q .
[0155] The fitness function based on ITAE (time times integral of absolute error) indicator is selected as follows.
[0156]
[0157] In the formula, e(k) represents the current value of the cumulative reward R(k) and the target value of the cumulative reward R of the kth iteration (round) of the training agent process. o The deviation of o The value is related to the reward function and the maximum number of training steps in a single round. The AMA optimization method is used to optimize the DDPG hyperparameters so that the fitness function J corresponding to the DDPG algorithm training results is finally stabilized at the minimum value.
[0158] The specific process of optimizing the hyperparameters of the deep deterministic policy gradient algorithm using the adaptive ephemera optimization method is as follows:
[0159] 5.3.1: Initialize the mayfly population and generate a set of weight nodes evenly distributed in the target space.
[0160] 5.3.2: Use the initialized mayfly population to assign values to the hyperparameters to be optimized in the deep deterministic policy gradient algorithm; the hyperparameters to be optimized include the noise standard deviation, the noise standard deviation decay rate, the Actor network learning rate, and the Critic network learning rate.
[0161] 5.3.3: Calculate the diversity evaluation index of the current mayfly population.
[0162] 5.3.4: If the diversity evaluation index of the current mayfly population is less than the preset threshold, then based on the weight node, the Steffensen method is used to mutate each mayfly in the current population, and the mutated mayfly population and the original mayfly population are selected to form a new mutant mayfly population with the same number as the original population.
[0163] 5.3.5: Divide the new variant mayfly population into half female and half male, calculate the individual fitness values of the mayflies contained in the new variant population according to the fitness function based on the ITAE indicator, and select the individual optimal mayfly and the global optimal mayfly from all male mayflies.
[0164] 5.3.6: Based on the individual optimal mayfly and the global optimal mayfly selected so far, the position and speed of the male mayfly are updated using the male mayfly state update formula, and the individual optimal mayfly and the global optimal mayfly are updated; the position and speed of the female mayfly are updated according to the female mayfly state update formula, and the fitness values of all mayflies in the mayfly population are calculated.
[0165] 5.3.7: Each pair of female and male in the mayfly population mates and produces two offspring according to the mating and reproduction behavior formula. From the mated mayflies, the same number of excellent individuals as the original population are selected and retained for the next generation.
[0166] 5.3.8: Check whether the end condition is met; if so, output the optimal hyperparameters; if not, replace the initialized mayfly population with the latest mutant population, and return to step "Use the initialized mayfly population to assign values to the hyperparameters to be optimized in the deep deterministic policy gradient algorithm".
[0167] The brief steps of AMA to optimize DDPG hyperparameters are as follows.
[0168] Step 1: Initialize the position and velocity states of the mayfly population Y, and generate a set of weight nodes Z={Z1,Z2,…,Z k}.
[0169] Step 2: The mayfly population assigns values to the hyperparameters to be optimized by the DDPG algorithm. The DDPG algorithm trains the sky-rail train agent to realize the multi-train adaptive collaborative tracking control process.
[0170] Step 3: Calculate the diversity κ index of the current mayfly population. If κ is less than the given threshold ζ, mutate each mayfly in the mayfly population, and select the best from the mutated mayfly population and the original mayfly population to form a new mutant mayfly population of size Y.
[0171] Step 4: Divide the new variant mayfly population into half female and half male, and calculate the fitness value J of each individual mayfly in the population. Select the best individual mayfly PBEST and the global best mayfly GBEST from all male mayflies.
[0172] Step 5: Update the position and speed of male mayflies according to formula (19)-formula (21), and update the individual optimal mayfly and the global optimal mayfly; update the position and speed of female mayflies according to formula (22)-formula (24); calculate the fitness values of all mayflies in the mayfly population.
[0173] Step 6: A pair of female and male in the mayfly population mates and produces two offspring according to formula (25); Y superior individuals are selected from the 2Y mayflies after mating and retained for the next generation.
[0174] Step 7: Finally, check whether the end condition is met. If so, give the result of parameter optimization. If not, return to Step 2 to continue the next iteration.
[0175] Aiming at the pain point of too many hyperparameters and difficult tuning of the DDPG algorithm, the present invention proposes an adaptive MA algorithm (AMA) to optimize the hyperparameters such as noise standard deviation, noise standard deviation decay rate and network learning rate in the search space to improve the learning effect of the DDPG algorithm. Subsequent simulation results show that compared with the MA-AC algorithm, MA-DQN and MA-DDPG algorithms, the AMA-DDPG algorithm performs better in both training and verification stages. This method effectively realizes the adaptive online adjustment of controller parameters for different degrees of disturbances, ensuring the stability control of the virtual formation of trains under interference.
[0176] Step 6: Use the optimized deep deterministic policy gradient algorithm to train the distributed active disturbance rejection controller to obtain a trained distributed active disturbance rejection controller.
[0177] The DDPG algorithm is used to continuously train each skytrain agent, and the trained train agent can adaptively adjust the distributed controller parameters according to time-varying disturbances. First, the learning environment of the train agent is defined, including the state space, action space and reward function.
[0178] 6.1.1 Learning environment.
[0179] (1) State space.
[0180] In the virtual marshaling mode, the state vector of the leader train is represented by ξ0 = [x0, v0] T , when i∈F, the follower train state vector is expressed as ξ i =[x i ,v i ] T The comfort of train riding is an important indicator of high-quality train operation. Here, the jerk of the follower train J needs to be considered. i Therefore, the state space S of the follower train can be expressed as: In the formula, It represents the double derivative of the follower train speed with respect to time, i.e. the train jerk.
[0181] (2) Action space.
[0182] The core idea of the distributed active disturbance rejection controller is to observe the disturbances to the system through NESO (nonlinear extended state observer) and perform control compensation. However, when the amplitude and frequency of the disturbance to the system change, it is difficult to select the appropriate NESO adjustable parameter a. 1,i Interference can be effectively suppressed. Therefore, the action space A can be expressed as follows.
[0183] A={[a 1,i ]},i∈F (30).
[0184] (3) State transfer equation.
[0185] The state of the intelligent train at time t is s t Take action a t Transfer to state s at time t+1 t+1 The state transition probability P s a = 1. According to equations (1), (2) and (3), the skyrail train motion state transfer equation can be expressed as follows.
[0186]
[0187] In the formula, g i represents the average acceleration of the follower train during the ΔT time, represents the rate of change of the average jerk of the follower train within ΔT time, Due to g i The term includes the traction / braking force u of train number i i ,u i As the output of the controller, i.e., the control input of train number i, u iRelated to the NESO parameter. ΔT represents a smaller time interval. From formula (13), it can be seen that by changing the NESO parameter, the transfer direction of the system state can be directly affected.
[0188] (4)Reward function.
[0189] The reward function is designed based on the principle of heavy rewards and light punishments. Here, the normal distribution reward function r is selected.
[0190]
[0191] In the formula, σ1, μ1 represent the standard deviation and mathematical expectation of the normal distribution reward function, respectively, and η′1, η2 represent the adjustable parameters of the reward function, which are used to adjust the degree of reward and punishment of the reward function. x = J1 represents the train jerk as a random variable of the normal distribution reward function. The closer J1 is to μ1, the greater the positive reward value, and vice versa, the smaller the negative reward value.
[0192] 6.1.2DDPG algorithm.
[0193] The DDPG algorithm is a deep deterministic policy gradient algorithm. It adds an experience replay mechanism and a dual network structure based on the Actor-Critic (AC) algorithm framework. The DDPG framework mainly includes a learning environment, an experience pool for storing samples, an actor network, and a critic network. The agent generates sample data by continuously interacting with the environment and stores these samples in an experience pool. A certain number of samples (minibatch samples) are extracted from the experience pool using a random strategy to train the network parameters. In order to improve the learning efficiency of the algorithm, the DDPG algorithm adopts a dual network structure. Both the actor network and the critic network contain an eval network and a target network. The target network and the eval network are a pair of neural networks with exactly the same structure. The eval network updates the network parameters (θ u ,θ Q ), and the target network parameters (θ u′ ,θ Q′ ) will periodically copy the eval network parameters through soft updates.
[0194]
[0195] In the formula, τ is the soft update coefficient. In order to enhance the agent’s ability to explore the environment, it is necessary to output the action μ(s t ) is added with OU noise N′ that obeys normal distribution as follows.
[0196]
[0197] In the formula, μ2 represents the expected mean, μ2=0, σ0 represents the initial noise standard deviation, and n represents the current iteration number of a round. The noise standard deviation σ2 gradually decreases with the increase of the iteration number n, reducing the exploration of the environment. ψ represents the standard deviation decay rate. The final action a t OU noise is superimposed.
[0198] a t =μ(s t )+N′ (35).
[0199] The actor network and the critic network are trained using different loss functions. When using batch data, the loss function of the critic network is as follows.
[0200]
[0201] Where K is the number of samples in the batch data; Q(s t ,a t |θ Q ) is the value of the eval network evaluating the current state and the action generated by the eval network in the actor.
[0202] y t = r(s t ,a t )+γQ′(s t+1 ,μ′(s t+1 |θ u′ )|θ Q′ ) (37).
[0203] In the formula, r(s t ,a t ) is the current state to perform action a t The immediate reward after that; γ is the reward attenuation coefficient; Q′(s t+1 ,μ′(s t+1 |θ u′ )|θ Q′ ) is the target network evaluating the next state s t+1 and the target network in the actor for the action μ(s) selected for the next state t+1 )’s value.
[0204] The loss function of the actor network is as follows.
[0205]
[0206] The parameter update formula of the critic network is as follows.
[0207]
[0208] In the formula, α Q is the learning rate of the critic network. Represents the gradient of the critic's current network loss function.
[0209] The parameter update formula of the actor network is as shown in formula (40).
[0210]
[0211] In the formula, α u is the learning rate of the actor network. Represents the gradient of the Actor's current network loss function.
[0212] 6.1.3DDPG training process.
[0213] Each SkyRail train is used as an intelligent entity, and the train controller is continuously trained using the DDPG algorithm. The trained train controller can adaptively adjust the controller parameters online according to different disturbances, effectively suppressing the adverse effects of external disturbances.
[0214] The steps for adaptive parameter training of distributed collaborative controller based on DDPG algorithm are as follows.
[0215] 1) Initialize the experience pool D with a storage capacity of M. 2) Initialize the policy network eval of the actor and critic: μ(s; θ u ) and Q(s t ,a t |θ Q ); copy the parameters of the eval network to the corresponding target network: θ u′ ←θ u ,θ Q′ ←θ Q ; The target network target of actor and critic is initialized: μ(s;θ u′ ) and Q(s t ,a t |θ Q′ ). 3) Forepisode = 1 to MaxEpisode do. 4) Initialize OU noise process N t 5) Randomly initialize the initial state of the skytrain and external disturbance to obtain the initial state of the simulation environment. 6) Fort = 1 to MaxStep do. 7) Through the train state s t Select a as the controller parameter of the action t =π θ (s t )+N t Among them, π θ(s t ) indicates state s t 8) According to the controlled object model (3), a simulation environment is established, and the control quantity is obtained by formula (16). The control quantity is input into the simulation environment to obtain the state of the skytrain at the next moment, that is, the state s of the environment at the next moment t+1 9) Get the immediate reward value r of the environment based on the current state t . 10) The experience sample [s t ,a t ,r t ,s t+1 ] is stored in the experience replay pool D. 11) Randomly sample from the experience replay pool D to obtain a sample set of size BatchSize {[s t ,a t ,r t ,s t+1 ]}. 12) Update the policy network eval parameter θ of the critic action network Q . 13) Update the policy network eval parameter θ of the actor action network u . 14) Update the target network parameters θ in the two networks u′ ,θ Q′ 15) EndFor. If the round end mechanism is satisfied, Break.
[0216] The nonlinear extended state observer in the distributed active disturbance rejection controller is used to observe and suppress external disturbances. Here, different degrees of external disturbances that trains often encounter are selected as training environments, and the AMA-DDPG algorithm is used to continuously train the train controller. After training, the train controller can suppress different degrees of disturbances by adjusting the NESO parameters online.
[0217] Step 7: According to the time-varying disturbance, the trained distributed anti-disturbance control controller parameters are adaptively adjusted online, and the sky-rail trains in the virtual formation are controlled so that the sky-rail trains in the virtual formation maintain the desired formation under the influence of the disturbance.
[0218] The specific process of step 7 is as follows.
[0219] 7.1: Obtain the actual speed and position information of the skyrail train at the current moment.
[0220] 7.2: Obtaining the actual speed and position information of the front neighboring train, the rear neighboring train and the pilot train of the SkyRail train at the current moment, and calculating the actual interval between the SkyRail train and the front neighboring train, the actual interval between the SkyRail train and the rear neighboring train, and the actual interval and actual relative speed between the SkyRail train and the pilot train at the current moment through deviation;
[0221] 7.3: The actual interval and the actual relative speed between the skyrail train and the leader train at the current moment are used as the input signal of the tracking differentiator, and the tracking differentiator provides a transition signal for the train formation state error and outputs the running interval transition signal and the relative speed transition signal at the current moment;
[0222] 7.4: The control quantity and actual speed of the skytrain at the current moment are used as the input of the nonlinear extended state observer, and the estimated value of the total disturbance of the system is output through state observation.
[0223] 7.5: The actual interval between one of the skyrail trains and the neighboring train in front, and the actual interval between one of the skyrail trains and the neighboring train in the back are used as inputs of the collision avoidance control. If the inputs of the collision avoidance control are both smaller than the set train repulsion field radius, then one of the skyrail trains will be subjected to the repulsion force exerted by the neighboring train in front and the neighboring train in the back respectively, and the total repulsion force exerted by the neighboring train in front and the neighboring train in the back prevents the skyrail train from colliding with each other.
[0224] 7.6: Based on the current state information of one of the skyrail trains, the front neighboring train, the rear neighboring train and the leader train, the running interval transition signal and the relative speed transition signal output by the tracking differentiator, the estimated value of the total system disturbance output by the nonlinear extended state observer and the total repulsion signal output by the collision avoidance control are used as the current input of the distributed active disturbance rejection controller to calculate the output control quantity of the distributed active disturbance rejection controller for one of the skyrail trains at the next moment; the state information includes the actual speed and position information.
[0225] 7.7: Use the output control amount at the next moment to drive one of the skyrail trains to track the neighboring train in front for coordinated operation, and obtain the actual speed and position information of the skyrail train at the next moment.
[0226] 7.8: Update the current moment to the next moment and return to step 7.2.
[0227] The distributed anti-disturbance controller ensures the stability control of the virtual train formation under the influence of interference by controlling the traction and braking force of the train.
[0228] Taking train number i as an example, the specific control process of distributed active disturbance rejection control is as follows.
[0229] Step 1: t0 is taken as the initial time, and the output control quantity u of the train controller number i is initialized i (t0).
[0230] Step 2: Train number i obtains its actual speed v at the current time t through the speed measurement and positioning device. i (t) and position information x i (t), the actual speed v of the front train numbered i-1, the rear train numbered i+1 and the leader train at the current time t is obtained by using the vehicle-to-vehicle wireless communication method. i-1 (t),v i+1 (t), v0(t) and position information x i-1 (t),x i+1 (t),x0(t). The actual interval x between the train numbered i and the preceding train numbered i-1 at the current time t is obtained by calculating the deviation. i,i-1 (t) and relative speed v i,i-1 (t), the actual interval x between train number i and the following train number i+1 i,i+1 (t) and relative speed v i,i+1 (t) and the actual interval x between train number i and the leader train i,0 (t) and relative speed v i,0 (t).
[0231] Step 3: The actual distance x between the train number i and the leader train is i,0 (t) and relative speed v i,0 (t) is the input signal of the tracking differentiator, which is the train formation state error Provide a transition process Output the operation interval transition signal of the current time t and relative velocity transition signal
[0232] Step 4: Set the control quantity u of train number i at the current time t i (t) and the actual speed v i (t) is used as the input of the nonlinear extended state observer, and the external disturbance σ is output through state observation i (v i )’s estimated signal ψ2.
[0233] Step 5: The actual interval x between the train numbered i and the preceding train numbered i-1 and between the train numbered i and the following train numbered i+1 at the current time t i,i-1 (t),x i,i+1 (t) is used as the input of collision avoidance control. If x i,i-1 (t),x i,i+1 (t) are all smaller than the set train repulsion field radius ρ j When the train number i is equal to the train number i-1, the train number i+1 will be subject to repulsive forces respectively. The total repulsive force F obtained by the train number i is i rep To prevent collisions between trains.
[0234] Step 6: Based on the status information of the current time t of the train numbered i, the front train numbered i-1, the rear train numbered i+1 and the leader train, the running interval transition signal output by the tracking differentiator is and relative velocity transition signal The disturbance estimation signal ψ2 output by the nonlinear extended state observer and the total repulsion signal F output by the collision avoidance control i rep As the current input of the distributed anti-disturbance control controller, the output control quantity u of the controller of train number i at the next time t+1 is obtained through calculation i (t+1).
[0235] Step 7: The controller output control quantity u obtained above i (t+1) Drive train number i to track the preceding train and obtain the actual speed v of train number i at the next moment t+1 i (t+1) and position signal x i (t+1).
[0236] Step 8: The output control quantity and state information of the next time t+1 obtained above are used as the output control quantity and state information of the train number i at the current time t, that is, u i (t) = u i (t+1), v i (t) = v i (t+1), x i (t) = x i (t+1).
[0237] Step 9: Return to Step 2 and repeat the above steps to finally realize the virtual marshaling of each unit train to maintain stable formation operation.
[0238] The method proposed in the existing document “ZY Guo, ZQ Li, “Distributed auto disturbances rejection resilient control of permanent magnetic maglev trains based on the optimized deep deterministic policy gradient algorithm”, IET Control Theory and Applications, 2024.” only considers the running status information of adjacent front and rear vehicles, which can accelerate the restoration of stable formation to a certain extent, but the input of the status information of the front and rear vehicles needs to be weighed. If the controller is not designed properly, it is easy to produce speed and spacing oscillations, and the stability of the queue cannot be guaranteed while taking into account the stability of the single vehicle. Moreover, if the deviation of the formation state of the adjacent rear vehicles is too large, the stability of the queue will deteriorate. The method proposed in the present invention introduces the running status information of the pilot vehicle on the basis of the existing literature to improve the oscillation situation. Using the running status information of the pilot train and the front and rear vehicles as the control input can take into account both the stability of the queue and the stability of the single vehicle.
[0239] The main beneficial effects of the present invention are as follows.
[0240] (1) Compared with the existing literature, the distributed control method proposed in this invention is designed based on the two-way-pilot communication topology structure, introduces the pilot vehicle operation status information, can take into account both the stability of the queue and the stability of the single vehicle, and achieves a satisfactory control effect. In addition, the tanh function is used to smooth the formation state error with adjacent trains to prevent improper control caused by too large an error.
[0241] (2) The present invention proposes an adaptive mayfly optimization method (AMA), which effectively solves the problem that conventional mayfly methods are prone to fall into local optimal points.
[0242] (3) The present invention adopts the AMA method to optimize the hyperparameters of the DDPG algorithm, thereby improving the learning efficiency of the DDPG algorithm.
[0243] (4) The present invention proposes a distributed anti-disturbance resilience control method based on AMA-DDPG for skytrains to achieve adaptive collaborative control, which is a major innovation in theoretical development.
[0244] In order to solve the adaptive control problem of external disturbances of different degrees and realize the effective virtual formation of sky-rail trains, the present invention proposes a distributed anti-disturbance elastic control method based on deep deterministic policy gradient algorithm and adaptive ephemera optimization (AMA-DDPG). A nonlinear extended state observer (NESO) based on a high-order sliding mode differentiator is designed to observe disturbances, and a deep deterministic policy gradient algorithm (DDPG) is designed to adaptively adjust the parameters of NESO to adapt to changes in external disturbances. However, the DDPG algorithm has a large number of hyperparameters, which makes the calculation of the optimal value difficult. Therefore, in order to optimize the hyperparameters of the network such as the noise standard deviation (SD), the noise SD decay rate and the learning rate, an adaptive ephemera optimization method (AMA) is proposed. Each train in the virtual formation is regarded as an intelligent agent. The uniqueness of this method is that the AMA-DDPG algorithm is designed to continuously train all intelligent agent trains. After successful training, the controller parameters of the sky-rail train can be adaptively adjusted online according to the different degrees of external disturbances. Compared with other methods, the AMA-DDPG algorithm exhibits stronger anti-disturbance performance.
[0245] The method of the present invention is verified by simulation as follows.
[0246] (1)Simulation environment.
[0247] Here, three SkyRail trains form a virtual formation for simulation test. No. 0 is the leader train, No. 1 and No. 2 are the follower trains, and No. 1 follower train runs in the middle of the virtual formation. The present invention mainly focuses on No. 1 follower train. Figure 6 shown.
[0248] Assume that the initial states of the leader train and each follower train are as shown in Table 1.
[0249] Table 1 Initial state of the agent train
[0250] Intelligent Train Location (m) Speed (m / s) Pilot Train 135 0 1#Follower Train 95 0 2# Follower Train 55 0
[0251] Internal disturbance of the follower skyrail train It can be expressed as follows.
[0252]
[0253] The sampling period of the system simulation settings is h = 0.01, the controller coefficient b = 0.00059, α v = -0.0078. During the simulation, the expected running interval between adjacent trains is changed from 8m to 30m at 950s.
[0254] The design parameters of the differential tracker (TD) of the follower train are: r0 = 0.1, h0 = 0.01. The remaining NESO parameters of the follower train are: b1 = 1.0, q1 = 1.37, p1 = 2.35.
[0255] The parameters of the distributed active disturbance rejection controller (18) are selected as follows.
[0256]
[0257] Parameters selection of artificial potential field method: train repulsion influence radius ρ = 10m, train actual radius r = 3m, repulsion scale factor μ = 5.9.
[0258] (2) AMA method optimization.
[0259] The AMA method is used to optimize multiple hyperparameters of the DDPG algorithm. The mayfly population size Y = 100, the maximum number of iterations N d =50, adjustable parameters λ1=λ2=β=2, λ=fl=0.1, ephemeral dimension D=3, here we set α u =α Q . [v min ,v max ]=[-0.1,0.1], the search space settings of 4 hyperparameters: standard deviation σ∈[1,20], standard deviation decay rate ψ∈[0.01,1], network learning rate α u ,α Q ∈[0.0001,0.01].
[0260] Table 2 Iteration results
[0261] Optimization Algorithm Convergence iteration number Convergence fitness MA 32 280 AMA 12 240
[0262] according to Figure 7 As shown in Table 2, compared with the conventional MA algorithm, the AMA algorithm converges faster and has a smaller fitness after convergence. It can effectively find the global optimal point, making up for the defect that the conventional MA algorithm is prone to fall into the local optimal point.
[0263] Depend on Figures 8 to 10 It can be seen that the optimal hyperparameters of the DDPG algorithm are: noise standard deviation σ = 15.66, noise standard deviation decay rate ψ = 0.49, Actor and Critic network learning rate α u =α Q =0.00205.
[0264] (3) DDPG algorithm training and verification.
[0265] The DDPG algorithm is used to train the skytrain intelligent agent, and the parameters of the distributed active disturbance rejection controller are adaptively adjusted online under different external interference conditions.
[0266] Other hyperparameter settings of DDPG algorithm: simulation step size is 0.01, soft update coefficient τ = 0.001, experience replay pool size M = 100000, discount factor γ = 0.99, number of samples BatchSize = 300, maximum number of steps in a single round MaxStep = 50, maximum number of training rounds MaxEpisode = 5000. The initial value of parameter action a = 5.1. AdamOptimizer is used as the optimizer of network parameters in training. The reward function curve is as follows Fig.11 shown.
[0267] The target network and eval network of the actor use a multi-hidden layer feedforward neural network, the hidden layer uses relu as the activation function, and the output layer uses tanh as the activation function, so that the output parameter action of the intelligent agent is limited to a certain range: [a min ,a max ]=[-5,5]. The actor network structure is as follows Fig.12 As shown, Fig.12 The numbers in the middle represent the number of neurons in a layer.
[0268] The target network and eval network of the critic also use a multi-hidden layer feedforward neural network. The network input layer consists of two parts: the state of the train agent and the parameter action generated by the actor network. The network output layer outputs a state action value. Its network structure is as follows Fig.13 As shown, Fig.13 The numbers in the middle represent the number of neurons in a layer.
[0269] 3.1 Training process
[0270] In order to better train the adaptive ability of the skytrain to external disturbances and increase the credibility of the training, the initial external disturbance training environment setting of the present invention is expressed by formula (43). In each training round, the system randomly gives an initial external disturbance to train the intelligent train.
[0271]
[0272] in, is an external disturbance.
[0273] Four algorithms, MA-PG, MA-AC, MA-DDPG and AMA-DDPG, are used to train the train agent to realize the adaptive adjustment function of control parameters. The training process results are shown in the figure. Figure 14 to Figure 15 shown.
[0274] Depend on Fig.14 and Fig.15It can be seen that compared with the MA-PG and MA-AC algorithms, the AMA-DDPG algorithm shows faster and smoother convergence during the training process. The MA-DDPG algorithm fell into a local optimal point during the training process. The AMA-DDPG algorithm avoids this problem well, finds the global optimal point, and can obtain a larger round reward.
[0275] 3.2 Verification process.
[0276] In order to verify the performance of the trained distributed elastic controller, the present invention randomly gives initial external disturbances, forming 500 verification rounds, and verifies the results of the four algorithms, MA-PG, MA-AC, MA-DDPG and AMA-DDPG. Initial external disturbances in the verification process:
[0277]
[0278] Fig.16 and Fig.17 The control and generalization capabilities of the four algorithms are shown in Table 3. Assuming that the rounds with more than 12 round rewards are called successful rounds, the control success rate and generalization success rate of the four learning algorithms are shown in Table 3.
[0279] Table 3 Verification results
[0280] Learning Algorithms Control success rate Generalization success rate MA-PG 73.6% 60.6% MA-AC 78.0% 63.4% MA-DDPG 91.6% 84.0% AMA-DDPG 100% 95.8%
[0281] Depend on Fig.16 , Fig.17 As shown in Table 3, in terms of control ability, MA-PG has the worst verification effect and the lowest control success rate, followed by MA-AC, MA-DDPG performs well, and AMA-DDPG has the best verification effect, with a control success rate of up to 100%. In terms of generalization ability, MA-PG has the worst generalization ability, followed by MA-AC, MA-DDPG has good generalization ability, and AMA-DDPG has the best generalization ability, with a generalization success rate of up to 95.8%.
[0282] In order to further demonstrate the effectiveness of the AMA-DDPG algorithm, three different degrees of external disturbance environments are set in the collaborative tracking process of multiple skytrains to verify the effect of the train agent adaptively adjusting the control parameters as the external disturbance changes. The time-varying external disturbance environment can be expressed as follows.
[0283]
[0284] The AMA-DDPG algorithm is used to train the agent train. Fig.18 and Fig.19 Respectively represent the changing trends of train speed before and after training. Fig. 20It indicates the changing trend of the running interval between trains after training. Fig.21 Represents the comparison of control parameter changes of the follower train before and after training.
[0285] Combination Figures 18 to 20 It can be seen that compared with the untrained intelligent agent train, the trained intelligent agent train has better adaptive ability. Despite encountering three different degrees of external disturbances, the system control parameters can still be adaptively adjusted according to the changing disturbances, and the speed control curve fluctuates slightly. However, the control parameters of the untrained intelligent agent train are unchanged, and the speed control curve fluctuates greatly, which is difficult to meet the high-precision speed control requirements of the sky rail train. Fig.21 It can be seen that the distributed adaptive cooperative control algorithm based on AMA-DDPG can achieve high-precision cooperative operation of three skyrail trains maintaining a certain safety interval in the virtual marshaling mode.
[0286] This paper studies the problem of coordinated control of virtual marshaling of sky-rail trains under uncertain disturbances, and proposes a distributed self-disturbance rejection elastic control algorithm based on AMA-DDPG. During the training process, the train agent's adaptive ability to different degrees of disturbances is gradually improved. The main conclusions of this study are as follows.
[0287] 1) Compared with the conventional MA (Mayfly algorithm), AMA has the outstanding advantages of faster convergence speed and better optimization effect in optimizing multiple hyperparameters of DDPG, and converges after only 12 iterations.
[0288] 2) Compared with MA-PG, MA-AC and MA-DDPG algorithms, the AMA-DDPG algorithm shows smoother and faster convergence during the training process; during the verification process, it shows a higher control success rate and stronger generalization ability.
[0289] 3) Compared with conventional methods, the distributed disturbance rejection resilient controller based on AMA-DDPG can adaptively adjust controller parameters online according to disturbances of different degrees, thus showing stronger self-disturbance rejection performance.
[0290] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.
[0291] The present invention uses specific examples to illustrate the principles and implementation methods of the present invention. The above examples are only used to help understand the method and core ideas of the present invention. At the same time, for those skilled in the art, according to the ideas of the present invention, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting the present invention.
Claims
1. A distributed self-disturbance rejection elastic control method for virtual marshaling sky-rail trains, characterized in that: include: A distributed dynamics model of sky-high rail trains in virtual marshaling is constructed using a multi-agent modeling approach. Determine a formation state error model of the virtual formation according to the expected relative running speeds and expected running intervals between adjacent follower trains and the leader train in the virtual formation; According to the distributed dynamics model and the formation state error model, the formation state errors of adjacent trains are smoothed using a tanh function, and a coordination and collision avoidance control protocol based on a two-way-pilot communication topology is established; Construct a distributed active disturbance rejection controller based on cooperative and collision avoidance control protocol; The hyperparameters of the deep deterministic policy gradient algorithm are optimized by using an adaptive mayfly optimization method; the adaptive mayfly optimization method adds an adaptive mutation mechanism based on Steffensen's mayfly; Using an optimized deep deterministic policy gradient algorithm to train the distributed active disturbance rejection controller to obtain a trained distributed active disturbance rejection controller; The parameters of the trained distributed anti-disturbance control controller are adaptively adjusted online according to the time-varying disturbance, and the sky-rail trains in the virtual formation are controlled so that the sky-rail trains in the virtual formation can maintain the desired formation under the influence of disturbance.
2. The distributed auto-disturbance rejection elastic control method for virtual marshaling sky-rail train according to claim 1 is characterized in that: The distributed dynamics model of the sky-high rail train in virtual marshaling is constructed by using a multi-agent modeling method, which includes: Consider a multi-agent virtual marshaling system consisting of N trains, each train is a multi-agent rigid point, numbered 0 to N-1 along the running direction, and the train numbered 0 is used as the leader train, the trains numbered 1 to N-1 are used as follower trains, and the follower trains only receive the status information of the trains before and after them in their respective neighborhoods; wherein, let E represent the set of all trains, E = {0, 1, ..., N-1}; F represents the set of follower trains, F = {1, 2, 3, ..., N-1}; G represents the set of neighboring trains of train number i, G = {i-1, i+1}; the train is an aerial rail train; The distributed dynamics model of the skyrail train in the multi-agent virtual marshaling system is constructed using a multi-agent modeling method as follows: In the formula, x i (t) represents the position of train number i at time t, Represents x i The differential of (t), v i (t) represents the speed of train number i at time t, Indicates v i The differential of (t), f(v i ,t) represents the resistance experienced by train number i at time t, b represents the controller coefficient, u i (t) represents the traction / braking force of train number i at time t, represents the external environmental interference experienced by train number i at time t, α v v i (t) represents the known system linear term, α v represents the system linear term coefficient, represents the nonlinear internal disturbance with uncertain system parameter perturbation, σ i (v i (t),t) represents the total disturbance suffered by the system at time t.
3. The distributed auto-disturbance rejection elastic control method for virtual marshaling sky-rail train according to claim 1 is characterized in that: The formation state error model is: In the formula, e i,j represents the formation state error between train number i and its neighboring train number j, represents the expected relative state between train number i and its neighboring train number j, d i,j represents the actual relative state between train number i and its neighboring train number j, e i,0 represents the formation state error between train number i and the leader train, represents the expected relative state between train number i and the leader train, d i,0 represents the actual relative state between the train numbered i and the leader train, || represents the absolute value symbol, F represents the set of follower trains, F = {1, 2, 3, ..., N-1}; G represents the set of neighboring trains of the train numbered i, G = {i-1, i+1}, j = i-1 or j = i+1; In the formula, represents the expected interval between train number i and its neighboring train number j, h is the length of the train, S t represents the safety margin at time t, S brake (v i ) indicates that the speed of train number i is v i Braking distance at , S brake (v j ) indicates that the speed of the train in the neighborhood of number j is v j Braking distance at time; x i,j represents the actual interval between train number i and its neighboring train number j, represents the expected relative speed between train number i and its neighboring train number j, v i,j represents the actual relative speed between train number i and its neighboring train number j, x i represents the current position of train number i, x j represents the current position of the train in the neighborhood of number j, v i represents the current speed of train number i, v j It represents the current speed of the neighboring train numbered j, and the superscript T indicates transposition. In the formula, represents the expected interval between train number i and the leader train, x i,0 represents the actual interval between train number i and the leader train, represents the expected relative speed between train number i and the leader train, v i,0 represents the actual relative speed between train number i and the leader train, x0 represents the current position of the leader train, and v0 represents the current speed of the leader train.
4. The distributed auto-disturbance rejection elastic control method for virtual marshaling sky-rail train according to claim 1 is characterized in that: According to the distributed dynamics model and the formation state error model, the formation state errors of adjacent trains are smoothed using the tanh function, and a coordination and collision avoidance control protocol based on a two-way-pilot communication topology structure is established, specifically including: The tracking differentiator designed to make the train accelerate or decelerate smoothly is: Where, d i,0 (t) represents the actual relative state between train number i and the leader train at time t, d i,0 (t) = [x i,0 (t),v i,0 (t)] T , x i,0 (t) represents the actual interval between train number i and the leader train at time t, v i,0 (t) represents the actual relative speed between train number i and the leader train at time t; Indicates d i,0 (t) jump to A step input signal, Represents x i,0 (t) jump to A step input signal, represents the expected interval between train number i and the leader train at time t, Indicates v i,0 (t) jump to A step input signal, represents the expected relative speed between train number i and the leader train at time t; represents the expected relative state between train number i and the leader train at time t, express The transition signal, Indicates the running interval transition signal at time t, Relative speed transition signal at time t; and Both said The differential signal of express The differential signal of the signal; r0, r1 both represent the transition speed of the signal, h0, h1 both represent the terms that filter the noise of the signal; fhan() represents the fhan function; In order to observe all unknown items including internal disturbance and external disturbance, a nonlinear extended state observer based on a high-order sliding mode differentiator is designed as follows: Where ψ1(χ1) represents the estimated value of the train speed χ1, χ1 = v i , v i represents the current speed of train number i; ψ2(χ1) represents the estimated value of the total system interference χ2, χ2 = σ i (v i ), σ i (v i ) represents the total disturbance of the system; e1 represents the error of train speed, and e2 represents the error of total disturbance of the system; represents the differential of ψ1(χ1), represents the differential of ψ2(χ1); η1(χ1) represents the internal state of the nonlinear extended observer; f(v i ) represents the resistance of train number i; b represents the controller coefficient, u i represents the traction / braking force of train number i; a1, b1, q1 and p1 are all adjustable observer parameters, a1>0, b1>0, 0<q1<p1; sgn() represents the sign function; represents the absolute value sign; The improved repulsive field function is introduced as follows: In the formula, represents the improved repulsive field of the neighboring train numbered j relative to the train numbered i, ρ j represents the radius of the repulsive field of the neighborhood train numbered j, r j represents the actual size radius of the neighborhood train numbered j, μ represents the repulsive force scale factor, and ρ i,j represents the actual interval between train number i and its neighboring train number j, Δρ i,j Represents ρ i,j With r j The deviation of ρ j With r j The deviation represents the expected interval between train number i and its neighboring train number j, Represents ρ i,j and The absolute value of the deviation, F represents the set of follower trains, and G represents the set of neighboring trains of train number i; According to the improved repulsive field function, the repulsive force exerted on each train by a neighboring train is: In the formula, represents the repulsive force exerted by the neighboring train numbered j on the train numbered i, i∈F, j∈G, n j→i The unit repulsive force vector of the neighboring train numbered j pointing to the direction of the train numbered i, n i→B represents the unit vector that moves the train number i in the desired formation direction B, Indicates train number i n j→i The repulsive force in the direction Indicates train number i n i→B The repulsive force in the direction; According to the repulsive force exerted by a neighboring train on each train, the total repulsive force exerted by the neighboring trains in front and behind each train is determined as: In the formula, F i rep It indicates the total repulsive force of the train numbered i on the neighboring trains in front and behind, i∈F; It means that the train number i is repelled by the neighboring train in front. It means that the train number i is subject to the repulsive force from the neighboring train behind; The repulsive force exerted by a neighboring train on each train and the total repulsive force exerted by the front and rear neighboring trains on each train constitute collision avoidance control. The collision avoidance control, the tracking differentiator and the nonlinear extended state observer together constitute a coordination and collision avoidance control protocol based on a two-way-pilot communication topology structure.
5. The distributed auto-disturbance rejection elastic control method for virtual marshaling sky-rail train according to claim 4 is characterized in that: The distributed active disturbance rejection controller is: In the formula, e i,0 represents the formation state error between train number i and the leader train, e i,j represents the formation state error between train number i and its neighboring trains, express The transition signal, represents the expected relative state between train number i and the neighboring train ahead, i∈F; ξ i represents the state vector of train number i, ξ i =[x i ,v i ] T , x i Indicates the current position of train number i, v i represents the current speed of train number i; ξ0 represents the state vector of the leader train, ξ j represents the neighborhood train state vector; represents the coordinated part of train number i, k, and Both represent the adjustable coefficient matrix of the coordination and collision avoidance control protocol, ψ2 represents the estimated value of the total interference of the system; k = [k1, k2], k1 represents the position component of k, k2 represents the velocity component of k, express The component in the position direction, express The component in the direction of the velocity, express The component in the position direction, express The component in the direction of the speed; b represents the controller coefficient; u i Indicates the traction / braking force of train number i, F i rep represents the total repulsive force exerted on train number i by its neighboring trains in front and behind, t represents time t, and tanh() represents the hyperbolic tangent function.
6. The distributed auto-disturbance rejection elastic control method for virtual marshaling sky-rail train according to claim 1 is characterized in that: The Steffensen-based mayfly adaptive mutation mechanism specifically includes: The ratio of the number of weighted nodes adjacent to the current optimal solution of mayfly to the total number of weighted nodes is used as the evaluation index of the diversity of mayfly population; If the diversity evaluation index is less than the preset threshold, the Steffensen method is used according to the formula To mutate the population of mayflies; where X 1,k represents the kth dimension of the individual X1 to be mutated, X 1,k ′ is X 1,k The value after mutation; θ(X 1,k ) indicates the distance between X1 and Z1 and X 1,k The deviation of the expected distance between ′ and Z1, θ(X 1,k )=L(X1,Z1)-l * , L(X1,Z1) represents the distance between the mayfly individual X1 to be mutated and the weight node Z1, l * Represents X 1,k ′ and the expected distance between Z1, l * =α·L(X1,Z1),α∈[0,1], α represents a random number in the interval [0,1].
7. The distributed auto-disturbance rejection elastic control method for virtual marshaling sky-rail train according to claim 6 is characterized in that: The hyperparameters of the deep deterministic policy gradient algorithm are optimized using the adaptive ephemera optimization method, including: Initialize the mayfly population and generate a set of weight nodes evenly distributed in the target space; Using the initialized mayfly population to assign values to the hyperparameters to be optimized in the deep deterministic policy gradient algorithm; the hyperparameters to be optimized include the noise standard deviation, the noise standard deviation decay rate, the Actor network learning rate, and the Critic network learning rate; Calculate the diversity evaluation index of the current mayfly population; If the diversity evaluation index of the current mayfly population is less than a preset threshold, then based on the weight node, each mayfly of the current population is mutated using the Steffensen method, and the mutated mayfly population and the original mayfly population are selected to form a new mutated mayfly population with the same number as the original population; The new variant mayfly population is divided into half female and half male. According to the fitness function based on the ITAE indicator, the fitness values of the individual mayflies contained in the new variant population are calculated, and the best individual mayflies and the global best mayflies are selected from all male mayflies. According to the individual optimal mayfly and the global optimal mayfly selected so far, the position and speed of the male mayfly are updated using the male mayfly state update formula, and the individual optimal mayfly and the global optimal mayfly are updated; the position and speed of the female mayfly are updated according to the female mayfly state update formula, and the fitness values of all mayflies in the mayfly population are calculated; Each pair of female and male in the mayfly population mates and produces two offspring according to the mating and reproduction behavior formula. From the mated mayflies, the same number of excellent individuals as the original population are selected and retained for the next generation. Check whether the end condition is met; if so, output the optimal hyperparameters; if not, replace the initialized mayfly population with the latest mutant population, and return to step "Use the initialized mayfly population to assign values to the hyperparameters to be optimized in the deep deterministic policy gradient algorithm".
8. The distributed auto-disturbance rejection elastic control method for virtual marshaling sky-rail train according to claim 3 is characterized in that: According to the expected relative running speed and expected running interval between the adjacent follower trains and the leader train in the virtual formation, a formation state error model of the virtual formation is determined, and then the following is also included: Assume the following: Assumption 1: Assume that each train obtains the actual speed and position information of the preceding and following trains and the leader train in the neighborhood in real time through inter-train wireless communication; Assumption 2: Assume that the initial time is t0, and the initial state deviations between trains in the neighborhood satisfy ||ξ i (t0)-ξ j (t0)||≤δ1,j∈G, the external disturbance to the system satisfies||σ i (t)||≤δ2; where δ1,δ2 are both bounded positive constants, ξ i (t0) represents the state vector of train number i at the initial time t0, ξ j (t0) represents the state vector of the neighborhood train numbered j at the initial time t0, σ i (t) represents the external disturbance to the system at time t; || || represents the norm; Assumption 3: Assume that the actual speed of the leader train is guaranteed to be consistent with the target speed trajectory; The ultimate control objective of multi-train virtual marshaling collaborative tracking is expressed as: in, and They are all extremely small positive values, and v0 represents the speed of train number i at the initial moment.
9. The distributed auto-disturbance rejection elastic control method for virtual marshaling sky-rail train according to claim 1, characterized in that: According to the time-varying disturbance, the parameters of the trained distributed anti-disturbance controller are adaptively adjusted online, and the sky-rail trains in the virtual formation are controlled so that multiple sky-rail trains in the virtual formation maintain the desired formation under the interference. Specifically, the following are included: Obtaining actual speed and position information of the skyrail train at the current moment; Obtaining the actual speed and position information of the front neighboring train, the rear neighboring train and the pilot train of the SkyRail train at the current moment, respectively, and calculating the actual interval between the SkyRail train and the front neighboring train, the actual interval between the SkyRail train and the rear neighboring train, and the actual interval and actual relative speed between the SkyRail train and the pilot train at the current moment through deviation; The actual interval and the actual relative speed between the skyrail train and the leader train at the current moment are used as input signals of the tracking differentiator, and the tracking differentiator provides a transition signal for the train formation state deviation and outputs the running interval transition signal and the relative speed transition signal at the current moment; Taking the control quantity and actual speed of one of the skyrail trains at the current moment as inputs of a nonlinear extended state observer, and outputting an estimated value of the total disturbance of the system through state observation; The actual interval between one of the skyrail trains and the neighboring train in front, and the actual interval between one of the skyrail trains and the neighboring train in the back are used as inputs of the collision avoidance control. If the inputs of the collision avoidance control are both smaller than the set train repulsion field radius, one of the skyrail trains will be subjected to the repulsion force exerted by the neighboring train in front and the neighboring train in the back, respectively. The total repulsion force exerted by the neighboring train in front and the neighboring train in the back prevents the skyrail train from colliding with each other. Based on the current state information of the skyrail train, the front neighboring train, the rear neighboring train and the leader train, the running interval transition signal and the relative speed transition signal output by the tracking differentiator, the estimated value of the total disturbance of the system output by the nonlinear extended state observer and the total repulsion signal output by the collision avoidance control are used as the current input of the distributed active disturbance rejection controller to calculate the output control amount of the distributed active disturbance rejection controller for the skyrail train at the next moment; the state information includes the actual speed and position information; Using the output control amount at the next moment to drive one of the skyrail trains to track the neighboring train in front for coordinated operation, and obtaining the actual speed and position information of one of the skyrail trains at the next moment; Update the current moment to the next moment, and return to step "obtain the actual speed and position information of the front neighboring train, the rear neighboring train and the leader train of the skyrail train at the current moment, and calculate the actual interval and actual relative speed between the skyrail train and the front neighboring train, the actual interval and actual relative speed between the skyrail train and the rear neighboring train, and the actual interval and actual relative speed between the skyrail train and the leader train at the current moment through deviation".
Citation Information
Patent Citations
Virtual marshalling train control method based on elastic tracking model
CN114834503A
Virtual marshalling train reference curve calculation method based on improved reinforcement learning algorithm
CN116090336A
Virtual coupling high-speed train group cooperative operation control method
CN116279690A
Virtual marshalling train stable tracking control method and system considering packet loss
CN116853325A
Train tight tracking adaptive control method and device and computing equipment
CN117002565A
Cited By
Deep peak regulation dry-wet conversion control system based on extended state observer
CN119645141A
High-speed train cooperative control method based on distributed sampling observer
CN119668103A
Vehicle safety control method and apparatus
CN122684436A
