Dynamic reinforcement learning enabled overload train group cooperative control method, system, terminal and medium

Abstract

The application discloses a dynamic reinforcement learning enabled heavy-load train group cooperative control method and system, a terminal and a medium, wherein the method comprises the following steps: acquiring real-time running information of trains in a heavy-load train group; designing a train group cooperative control law according to the deviation of the real-time speed of the trains from a reference speed, the speed deviation of front and rear trains and the distance between the front and rear trains; establishing a reinforcement learning model for each train in the train group for dynamically adjusting the weight of the speed deviation term and the distance deviation term in the control law of the corresponding train, so that the system can learn and adapt to different running environments in the training, the reward function considers the control objectives of speed cooperation and safety distance, and a penalty function is introduced to deal with the situation of deviating from the expected state and the safety boundary; iteratively training each deep deterministic policy gradient model to obtain the control law of the corresponding train and apply the control law to the traction and braking system of the train, so that the train group can be intelligently and cooperatively controlled in high-speed running, and the running efficiency and safety are taken into account.

Description

Technical Field

[0001] This invention relates to the field of rail transit control technology, and in particular to a method, system, terminal, and medium for collaborative control of heavy-load train groups enabled by dynamic reinforcement learning. Background Technology

[0002] With socio-economic development, heavy-haul trains play a vital role in logistics transportation. However, as train group sizes increase and transportation tasks become more complex, traditional train cooperative control methods face a series of challenges. First, due to the large scale of train groups, traditional control methods may struggle to adapt to complex dynamic environments and real-time changing transportation demands. Second, the mutual influence and cooperative behavior between trains require more intelligent and adaptive control strategies. Furthermore, considering the safety and operational efficiency of the train system, more advanced decision-making mechanisms need to be introduced into cooperative control.

[0003] While existing research considers the safety distance constraint, the weighting coefficients for the speed coordination and safety distance terms are typically determined empirically or through trial and error when designing the controller. This approach is subjective and fixed, failing to fully account for the dynamic changes in the actual operating environment. Since the setting of these weighting coefficients directly affects the performance of speed tracking and operational safety, the lack of precise adjustments based on dynamic environmental data can lead to inconsistent system performance under different conditions.

[0004] Chinese patent CN117389157A discloses a sliding mode control method for virtual train formation operation. This invention constructs a dynamic model of a virtual high-speed train formation, builds an operation mode based on displacement and speed information, and determines the sliding mode control law for high-speed train operation. Chinese patent CN117104305A discloses a distributed cooperative tracking control method for virtual train formation. This method includes steps such as constructing a global optimal control problem, solving local optimal control subproblems, and updating local variables. Through iterative solving, the optimal control command sequence for the current train unit is obtained, realizing distributed cooperative control of the train. Chinese patent CN116279690A discloses a virtual coupled high-speed train group cooperative operation control method, further compressing train tracking intervals and realizing cooperative operation control of high-speed train groups with different initial states. The above-mentioned prior art mainly considers objectives such as speed tracking cooperation and safe distance control between trains, lacking intelligent control strategies for dynamic environments. Summary of the Invention

[0005] This invention provides a method, system, terminal, and medium for coordinated control of heavy-haul train groups enabled by dynamic reinforcement learning. It introduces reinforcement learning algorithms such as deep deterministic policy gradient to achieve adaptive trade-offs between speed coordination terms and safety distance terms, thereby improving the performance and adaptability of coordinated operation of train groups.

[0006] To achieve the above technical objectives, the present invention adopts the following technical solution:

[0007] In a first aspect, the present invention provides a method for cooperative control of heavy-load train groups enabled by dynamic reinforcement learning, comprising:

[0008] S1: Obtain real-time operation information of trains in the heavy-haul train group; wherein, the operation information includes the speed information and location information of each train;

[0009] S2: Design a train group cooperative control law based on the deviation between the train's real-time speed and the reference speed, the speed deviation between the trains in front and behind, and the distance between the trains in front and behind.

[0010] S3: Establish a separate reinforcement learning model for each train in the train group, specifically using a deep deterministic policy gradient model to dynamically adjust the weights of the speed deviation and distance deviation terms in the corresponding train's control law; the reward function of the reinforcement learning model considers the control objectives of speed coordination and safe distance, and introduces a penalty function to handle deviations from the desired state and safety boundary.

[0011] S4: Iteratively train the gradient model of each depth to determine the policy, obtain the control variables of the corresponding train, and apply them to the traction braking system of the train.

[0012] Furthermore, the control law for train i in the design of the train group cooperative control law is as follows:

[0013]

[0014] In the formula, u i (t) represents the control law of train i at time t; Let be the deviation between the real-time speed of train i and the reference speed at time t. Let be the speed deviation between train i and the vehicles in front and behind it at time t. Let α be the deviation between the distance between the train i and the car in front at time t and the safe distance, and let α be the deviation of the train's speed in front and behind. The weights, where β is the distance deviation term. The weights; where:

[0015]

[0016]

[0017]

[0018] In the formula, Let v be the reference speed of train i at time t. i (t) represents the real-time speed of train i at time t; v j (t) represents the real-time speed of train j at time t, n represents the total number of trains in the train group, and a ij As an adjacency parameter, when train j is the train before or after train i, a ij =1, otherwise 0; p i (t) represents the real-time position of train i at time t; p k (t) represents the real-time position of train k at time t, where train k is the following train of train i, and d safe The safe distance that should be maintained between vehicles.

[0019] Furthermore, a reinforcement learning model is established for each train, including:

[0020] (1) Define the state space, that is, determine the representation of the system state s, including the speed deviation e between the real-time speed and the reference speed of the train. ref Speed ​​deviation between front and rear vehicles e vel Deviation between front and rear vehicle distances and safe distance e pos ;

[0021] (2) Define the action space, that is, determine the system's execution action a and its value range, where the execution action a includes the weight α of the velocity deviation term and the weight β of the distance deviation term;

[0022] (3) Design the reward function r as follows:

[0023] r = r ref +r vel +r pos -p

[0024] in, For the reference speed following reward, k1 is a constant that adjusts the reference speed following reward. Here, k2 is the speed coordination bonus for the front and rear vehicles, and k2 is a constant that adjusts this bonus. For the safety distance bonus between vehicles, k3 is a constant that adjusts the safety distance bonus between vehicles.

[0025]

[0026] Where p is the penalty function, and k4 and k5 are the coefficients of the speed penalty term and the distance penalty term, respectively.

[0027] Furthermore, the training process of the depth-determined policy gradient model is as follows:

[0028] S4.1: Establish a depth-deterministic policy gradient model using an Actor-Critic structure; the Actor network is as follows:

[0029] a=Actor(s|θ actor )

[0030] Where a is the action to be performed in reinforcement learning, s is the state representation in reinforcement learning, and θ actor These are the parameters of the Actor network;

[0031] The Critic network is as follows:

[0032] Q(s, a|θ) critic )

[0033] Where Q is the value function, θ critic These are the parameters of the Critic network;

[0034] S4.2: Set optimization goals: Define cumulative rewards Where T is the time step and γ is the discount factor;

[0035] S4.3: Maximize cumulative rewards:

[0036]

[0037] S4.4: Iteratively train the depth-deterministic policy gradient model, adjust the weights α and β in the execution action a in real time, and obtain the control law of train i at time t. And it acts on the traction and braking system of train i to generate traction or braking force; among which, Let be the deviation between the real-time speed of train i and the reference speed at time t. Let be the speed deviation between train i and the vehicles in front and behind it at time t. Let t be the deviation between the distance between the train i and the train i in front and behind and the safe distance.

[0038] Furthermore, in step S4.3, when maximizing the cumulative reward, the Actor network parameters are updated using gradient ascent, and the Critic network parameters are updated using gradient descent.

[0039]

[0040]

[0041] Where λ and μ are the learning rates, The cumulative reward J is the sum of the Actor network parameters θ. actor gradient, The Q-value is related to the Critic network parameters. The gradient;

[0042] Secondly, the present invention provides a dynamically reinforcement learning-enabled cooperative control system for heavy-haul train groups, comprising:

[0043] The measurement module is used to: acquire real-time operating information of high-speed trains from onboard equipment and trackside equipment, the operating information including speed information and position information of each train;

[0044] The communication module is used for transmitting real-time operating information between adjacent high-speed trains in a high-speed train group.

[0045] The control module is used to: obtain the control law for each high-speed train based on the deviation between the train's real-time speed and the reference speed, the speed deviation between the trains in front and behind, and the distance between the trains in front and behind. The control law includes the weight α of the speed deviation term and the weight β of the distance deviation term.

[0046] The reinforcement learning module is used to dynamically adjust the weights α of the speed deviation term and β of the distance deviation term in the control law based on real-time train information.

[0047] The power module is used to output corresponding traction or braking force according to the control law to control the speed changes of high-speed trains.

[0048] Thirdly, the present invention provides an electronic terminal, including a processor and a memory, wherein the memory stores a computer program, and the processor invokes the computer program to perform the steps of the method described above.

[0049] Fourthly, the present invention provides a readable storage medium storing a computer program that, when invoked by a processor, performs the steps of the method described above.

[0050] Beneficial effects

[0051] This invention proposes a method, system, terminal, and medium for cooperative control of heavy-haul train groups enabled by dynamic reinforcement learning. By acquiring real-time train operation information and employing a deep deterministic strategy gradient algorithm, the cooperative control strategy is dynamically adjusted. This invention can intelligently adjust the trade-off between speed coordination and safety distance terms based on parameters such as real-time train speed, speed deviation between trains ahead and behind, and distance between trains ahead and behind, thereby improving the intelligence level of the train group. Through iterative training, this invention also optimizes the weights of the speed and distance deviation terms in the cooperative control law, achieving a comprehensive performance improvement and effectively increasing the operating efficiency of the train group. This invention emphasizes the system's adaptability; through online learning and dynamic adjustment, the system can flexibly adapt to different operating environments, greatly improving its adaptability and robustness. Regarding safety, this invention designs a reward function that comprehensively considers the control objectives of speed coordination and safety distance, and introduces a penalty function to handle abnormal situations, effectively preventing trains from deviating from the desired state and safety boundaries. This invention features real-time performance and iterative optimization, enabling train groups to respond quickly and flexibly to different transportation demands and unexpected situations. Overall, the technical solution of this invention takes into account intelligence, performance optimization, adaptability, security, and real-time performance. Attached Figure Description

[0052] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0053] Figure 1 This is a flowchart of a method for coordinated control of heavy-load train groups enabled by dynamic reinforcement learning, provided in an embodiment of the present invention.

[0054] Figure 2 This is a schematic diagram of the gradient algorithm for determining the depth strategy provided in this invention.

[0055] Figure 3 This is a schematic diagram of a dynamic reinforcement learning-enabled cooperative control system for heavy-load train groups provided by an example of the present invention. Detailed Implementation

[0056] The embodiments of the present invention will be described in detail below. These embodiments are based on the technical solutions of the present invention and provide detailed implementation methods and specific operation processes to further explain the technical solutions of the present invention.

[0057] This invention provides a method, system, terminal, and medium for coordinated control of heavy-haul train groups enabled by dynamic reinforcement learning. The method learns and adjusts the weight coefficients of speed coordination and safety distance terms in real time, enabling the train group to adapt more flexibly to different operating environments and transportation needs, improving the system's adaptability and intelligence level, and enabling trains to operate more efficiently and safely in coordination.

[0058] Example 1

[0059] like Figure 1 As shown, this embodiment provides a method for cooperative control of heavy-load train groups enabled by dynamic reinforcement learning, including:

[0060] S1: Obtain real-time operation information of trains in the heavy-haul train group; wherein, the operation information includes the speed information and location information of each train.

[0061] S2: Design a train group cooperative control law based on the deviation between the train's real-time speed and the reference speed, the speed deviation between the trains in front and behind, and the distance between the trains in front and behind.

[0062] S2.1: Calculate the speed deviation between the real-time speed and the reference speed of the train, where for train i, the formula for calculating the speed deviation between the real-time speed and the reference speed is:

[0063]

[0064] in, Let v be the reference speed of train i at time t. i (t) represents the real-time speed of train i at time t. Let t be the deviation between the real-time speed of train i and the reference speed at time t.

[0065] S2.2: Calculate the speed deviation between the front and rear vehicles, where for train i, the formula for calculating its speed deviation from that of the front and rear vehicles is:

[0066]

[0067] Among them, v i (t) represents the real-time speed of train i at time t, v j (t) represents the real-time speed of train j at time t, n represents the total number of trains in the train group, and a ij As an adjacency parameter, when train j is the train before or after train i, a ij =1, otherwise 0. Let t be the speed deviation between train i and the vehicles in front and behind it.

[0068] S2.3: Calculate the deviation between the distance between the train and the preceding and following trains and the safe distance. For train i, the formula for calculating the deviation between its distance from the preceding and following trains and the safe distance is:

[0069]

[0070] Where, p i (t) represents the real-time position of train i at time t; p k (t) represents the real-time position of train k at time t, where train k is the following train of train i, and d safe The safe distance that should be maintained between vehicles. Let t be the deviation between the distance between the train i and the train i in front and behind and the safe distance.

[0071] S2.4: Based on the deviation between the train's real-time speed and the reference speed, the speed deviation between the trains ahead and behind, and the distance between the trains ahead and behind, design the train group cooperative control law. The calculation formula for the control law of train i is:

[0072]

[0073] Where α is the train speed deviation term. The weights, where β is the distance deviation term. The weight.

[0074] S3: Establish a separate reinforcement learning model for each train in the train group, specifically using a deep deterministic policy gradient model to dynamically adjust the weights of the speed deviation and distance deviation terms in the corresponding train control law; the reward function of the reinforcement learning model considers the control objectives of speed coordination and safe distance, and introduces a penalty function to handle deviations from the desired state and safety boundary.

[0075] Specifically, a reinforcement learning model is established for each train, including:

[0076] (1) Define the state space, that is, determine the representation of the system state s, including the speed deviation e between the real-time speed and the reference speed of the train. ref Speed ​​deviation between front and rear vehicles e vel Deviation between front and rear vehicle distances and safe distance e pos ;

[0077] (2) Define the action space, that is, determine the system's execution action a and its value range, where the execution action a includes the weight α of the velocity deviation term and the weight β of the distance deviation term;

[0078] (3) Design a reward function to provide a positive reward when speed coordination and safe distance control reach the desired state, and introduce a penalty function to provide a negative reward when the speed deviation is large or the distance exceeds the safe range; the reward function is as follows:

[0079] r = r ref +r vel +r pos -p

[0080] in, For the reference speed following reward, k1 is a constant that adjusts the reference speed following reward. Here, k2 is the speed coordination bonus for the front and rear vehicles, and k2 is a constant that adjusts this bonus. For the safety distance bonus between vehicles, k3 is a constant that adjusts the safety distance bonus between vehicles.

[0081]

[0082] Where p is the penalty function, and k4 and k5 are the coefficients of the speed penalty term and the distance penalty term, respectively.

[0083] S4: Iteratively train the gradient model of each depth to determine the policy, obtain the control variables of the corresponding train, and apply them to the traction braking system of the train.

[0084] S4.1: Establish a depth-deterministic policy gradient model, such as... Figure 2 As shown, an Actor-Critic structure is adopted; the Actor network is as follows:

[0085] a=Actor(s|θ actor )

[0086] Where a is the action to be performed in reinforcement learning, s is the state representation in reinforcement learning, and θ actor These are the parameters of the Actor network;

[0087] The Critic network is as follows:

[0088] Q(s, a|θ) critic )

[0089] Where Q is the value function, θ critic These are the parameters of the Critic network;

[0090] S4.2: Set optimization goals: Define cumulative rewards Where T is the time step and γ is the discount factor;

[0091] S4.3: Maximize cumulative rewards:

[0092]

[0093] In this embodiment, gradient ascent is used to update the Actor network parameters, and gradient descent is used to update the Critic network parameters.

[0094]

[0095]

[0096] Where λ and μ are the learning rates, The cumulative reward J is the sum of the Actor network parameters θ. actor gradient, The Q-value is related to the Critic network parameters. The gradient;

[0097] S4.4: Iteratively train the depth-deterministic policy gradient model, adjust the weights α and β in the execution action a in real time, and obtain the control law u of train i at time t. i (t), and acts on the traction and braking system of train i to generate traction or braking force.

[0098] Example 2

[0099] like Figure 3 As shown, this embodiment provides a dynamically reinforcement learning-enabled cooperative control system for heavy-haul train groups, including:

[0100] The measurement module is used to: acquire real-time operating information of high-speed trains from onboard equipment and trackside equipment, the operating information including speed information and position information of each train;

[0101] The communication module is used for transmitting real-time operating information between adjacent high-speed trains in a high-speed train group.

[0102] The control module is used to: obtain the control law for each high-speed train based on the deviation between the train's real-time speed and the reference speed, the speed deviation between the trains in front and behind, and the distance between the trains in front and behind. The control law includes the weight α of the speed deviation term and the weight β of the distance deviation term.

[0103] The reinforcement learning (DDPG) module is used to dynamically adjust the weights α of the speed deviation term and β of the distance deviation term in the control law based on real-time train information.

[0104] The power module is used to output corresponding traction or braking force according to the control law to control the speed changes of high-speed trains.

[0105] Example 3

[0106] This embodiment provides an electronic terminal including a processor and a memory, wherein the memory stores a computer program, and the processor calls the computer program to execute the steps of the method described in Embodiment 1.

[0107] Example 4

[0108] This embodiment provides a readable storage medium storing a computer program that, when invoked by a processor, performs the steps of the method described in Embodiment 1.

[0109] It should be understood that, in the embodiments of the present invention, the processor may be a Central Processing Unit (CPU), or it may be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor. The memory may include read-only memory and random access memory, and provides instructions and data to the processor. A portion of the memory may also include non-volatile random access memory. For example, the memory may also store device type information.

[0110] The readable storage medium is a computer-readable storage medium, which can be an internal storage unit of the controller described in any of the foregoing embodiments, such as the controller's hard drive or memory. The readable storage medium can also be an external storage device of the controller, such as a plug-in hard drive, Smart Media Card (SMC), Secure Digital (SD) card, or Flash Card equipped on the controller. Further, the readable storage medium can include both the controller's internal storage unit and external storage devices. The readable storage medium is used to store the computer program and other programs and data required by the controller. The readable storage medium can also be used to temporarily store data that has been output or will be output.

[0111] Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned readable storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0112] It is understood that the same or similar parts in the above embodiments can be referred to each other, and the contents not described in detail in some embodiments can be referred to the same or similar contents in other embodiments.

[0113] The above embodiments are preferred embodiments of this application. Those skilled in the art can make various changes or improvements based on them. Without departing from the overall concept of this application, these changes or improvements should fall within the scope of protection claimed in this application.

Claims

1. A method for cooperative control of heavy-haul train groups enabled by dynamic reinforcement learning, characterized in that, include: S1: Obtain real-time operation information of trains in the heavy-haul train group; wherein, the operation information includes the speed information and location information of each train; S2: Design a train group cooperative control law based on the deviation between the train's real-time speed and the reference speed, the speed deviation between the trains in front and behind, and the distance between the trains in front and behind. S3: Establish a separate reinforcement learning model for each train in the train group, specifically using a deep deterministic policy gradient model to dynamically adjust the weights of the speed deviation and distance deviation terms in the corresponding train's control law; the reward function of the reinforcement learning model considers the control objectives of speed coordination and safe distance, and introduces a penalty function to handle deviations from the desired state and safety boundary. This includes establishing a reinforcement learning model for each train, including: (1) Define the state space, that is, determine the system state. The representation includes the speed deviation between the train's real-time speed and the reference speed. Speed ​​deviation between front and rear vehicles Deviation between distance to front and rear vehicles and safe distance ; (2) Define the action space, that is, determine the system's execution actions. and its value range, including the action to be performed. Including the weight of the speed deviation term Weights of distance deviation term ; (3) Design reward function for: ； in, For reference, speed follows the reward item. It is a constant that adjusts the reference speed to follow the reward. For the speed coordination bonus between the front and rear vehicles, It is a constant that adjusts the speed-coordinated reward between the front and rear vehicles. The reward is for maintaining a safe following distance between vehicles. It is a constant used to adjust the reward for safe following distance between vehicles; ； in, It is a penalty function. , These are the coefficients for the speed penalty and the distance penalty, respectively. S4: Iteratively train the policy gradient model at each depth to obtain the control variables of the corresponding train and apply them to the train's traction braking system; The training process of the depth-determined policy gradient model is as follows: S4.1: Establish a depth-deterministic policy gradient model using an Actor-Critic structure; the Actor network is as follows: ； in, It is the execution action of reinforcement learning. It is a state representation in reinforcement learning. These are the parameters of the Actor network; The Critic network is as follows: ； in It is a value function. These are the parameters of the Critic network; S4.2: Set optimization goals: Define cumulative rewards ,in It is a time step. It is a discount factor; S4.3: Maximize cumulative rewards: ； S4.4: Iteratively train the depth-deterministic policy gradient model and adjust the execution actions in real time. Weights in and ,get Time Train control law And act on the train The traction and braking system generates traction or braking force; among which, Train at time t The deviation between the real-time speed and the reference speed, Train at time t Speed ​​deviation from the vehicles in front and behind, Train at time t Deviation between the distance between the front and rear vehicles and the safe distance.

2. The method for cooperative control of heavy-haul train groups enabled by dynamic reinforcement learning according to claim 1, characterized in that, Train at time t The deviation between the real-time speed and the reference speed , Train at time t Speed ​​deviation from vehicles in front and behind , Train at time t Deviation between distance to front and rear vehicles and safe distance The calculation formulas are as follows: ； ； ； In the formula, for Time Train Reference speed, for Time Train Real-time speed; for Time Train Real-time speed, This represents the total number of trains in the train group. For adjacency parameters, when the train It is a train When the car in front or behind, Otherwise, it is 0; for Time Train Real-time location; for Time Train Real-time location of the train For train The car behind, The safe distance that should be maintained between vehicles.

3. The method for cooperative control of heavy-haul train groups enabled by dynamic reinforcement learning according to claim 1, characterized in that, In step S4.3, when maximizing the cumulative reward, the Actor network parameters are updated using gradient ascent, and the Critic network parameters are updated using gradient descent. ； ； in, , It's the learning rate. The cumulative reward J is the sum of the parameters of the Actor network. gradient, The Q-value is related to the Critic network parameters. The gradient.

4. A dynamic reinforcement learning-enabled cooperative control system for heavy-haul train groups, characterized in that, include: The measurement module is used to acquire real-time operating information of high-speed trains from onboard equipment and trackside equipment, the operating information including speed information and position information of each train; The communication module is used for transmitting real-time operating information between adjacent high-speed trains in a high-speed train group. The control module is used to: derive the control law for each high-speed train based on the deviation between the train's real-time speed and the reference speed, the speed deviation between the trains ahead and behind, and the distance between the trains ahead and behind. The control law includes the weight of the speed deviation term. Weights of distance deviation term ; The reinforcement learning module is used to dynamically adjust the weight of the speed deviation term in the control law based on real-time train information. Weights of distance deviation term ; The power module is used to output corresponding traction or braking force according to the control law to control the speed changes of the high-speed train. This includes establishing a reinforcement learning model for each train, including: (1) Define the state space, that is, determine the system state. The representation includes the speed deviation between the train's real-time speed and the reference speed. Speed ​​deviation between front and rear vehicles Deviation between distance to front and rear vehicles and safe distance ; (2) Define the action space, that is, determine the system's execution actions. and its value range, including the action to be performed. Including the weight of the speed deviation term Weights of distance deviation term ; (3) Design reward function for: ； in, For reference, speed follows the reward item. It is a constant that adjusts the reference speed to follow the reward. For the speed coordination bonus between the front and rear vehicles, It is a constant that adjusts the speed-coordinated reward between the front and rear vehicles. The reward is for maintaining a safe following distance between vehicles. It is a constant used to adjust the reward for safe following distance between vehicles; ； in, It is a penalty function. , These are the coefficients for the speed penalty and the distance penalty, respectively. The reinforcement learning model specifically uses a deep deterministic policy gradient model, and the training process is as follows: S4.1: Establish a depth-deterministic policy gradient model using an Actor-Critic structure; the Actor network is as follows: ； in, It is the execution action of reinforcement learning. It is a state representation in reinforcement learning. These are the parameters of the Actor network; The Critic network is as follows: ； in It is a value function. These are the parameters of the Critic network; S4.2: Set optimization goals: Define cumulative rewards ,in It is a time step. It is a discount factor; S4.3: Maximize cumulative rewards: ； S4.4: Iteratively train the depth-deterministic policy gradient model and adjust the execution actions in real time. Weights in and ,get Time Train control law And act on the train The traction and braking system generates traction or braking force; among which, Train at time t The deviation between the real-time speed and the reference speed, Train at time t Speed ​​deviation from the vehicles in front and behind, Train at time t Deviation between the distance between the front and rear vehicles and the safe distance.

5. An electronic terminal, characterized in that: It includes a processor and a memory, the memory storing a computer program, the processor calling the computer program to perform the steps of the method according to any one of claims 1-3.

6. A readable storage medium, characterized in that: A computer program is stored, which, when invoked by a processor, performs the steps of the method according to any one of claims 1-3.

Images