A method and system for controlling special vehicles at a ramp intersection
By using multi-agent deep reinforcement learning and Transformer architecture to optimize vehicle paths at apron intersections, the problems of multi-vehicle collisions and waiting times at apron intersections were solved, achieving coordinated and optimized vehicle traffic and improving airport traffic efficiency.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- CIVIL AVIATION UNIV OF CHINA
- Filing Date
- 2026-04-30
- Publication Date
- 2026-07-28
AI Technical Summary
Existing technologies have failed to effectively optimize the risk of multi-vehicle collisions and waiting times at apron intersections. Traditional methods cannot optimize the travel time of the entire intersection area, resulting in low airport traffic efficiency.
The method employs Multi-Agent Deep Reinforcement Learning (MADRL) to collect information on the environmental state and relative motion of vehicles. It then uses a pre-trained policy network to generate action commands and future path trajectories, optimizing vehicle passage at intersections. By combining this with a Transformer structure to handle variable-length inputs, it achieves cooperative and optimized passage for vehicles.
It significantly reduces average waiting time and traffic delays in intersection areas, improves airport traffic efficiency, adapts to complex environments, and achieves coordinated and optimized vehicle traffic.
Smart Images
Figure CN122473929A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the fields of multi-agent reinforcement learning and traffic control technology, and in particular to a method and system for controlling special vehicles at apron intersections. Background Technology
[0002] As airport operations move towards reduced manpower and automation, autonomous special-purpose vehicles are receiving significant attention as a key component of airport unmanned equipment. In 2021, the Civil Aviation Administration of China (CAAC) released the "Airport Unmanned Equipment Application Roadmap," emphasizing the plan to achieve large-scale demonstration applications by 2025 and establish comprehensive technical standards, testing specifications, and operation and maintenance systems. Particularly in the integrated application of unmanned equipment in areas such as airport ground transportation, material handling, and baggage transport, the roadmap stresses the safety, controllability, and seamless integration with existing airport systems.
[0003] Autonomous special-purpose vehicles, combining vehicle-to-infrastructure (V2I) and intelligent driving technologies, offer greater support for intelligent airport management compared to traditional special-purpose vehicle operation methods. They exhibit significant advantages in improving operational efficiency, reducing operating costs, enhancing safety, and adapting to complex environments. With the introduction of intelligent traffic control methods, airport special-purpose vehicles can achieve continuous operation without the limitations of human fatigue. Advanced sensors' rapid identification of the external environment, combined with the precise decision-making and planning of intelligent agents, can significantly optimize transit times, improve transportation levels and efficiency, and reduce the time costs caused by human judgment and operational errors. It has been proven that autonomous special-purpose vehicles demonstrate excellent adaptability in the complex and high-density environment of airports, improving traffic efficiency and reducing traffic accidents to a certain extent. Furthermore, they also perform exceptionally well in assisting airport internal traffic management towards intelligent and data-driven management, improving resource utilization and service quality.
[0004] In recent years, significant progress has been made in combining autonomous vehicle resource scheduling and traffic control with reinforcement learning technology. However, most applications in airport scenarios have only dispatched autonomous special vehicles to intersections on the apron, without considering the scenario where vehicles autonomously enter the apron. Intersections between the apron and driving lanes are often the areas with the most complex environmental information, the greatest reliance on driver judgment, and the highest risk of traffic conflicts during the operation of autonomous special vehicles. Because traffic at unsignalized intersections lacks explicit rules, the risk of multi-vehicle collisions is extremely high, accompanied by increased waiting time to mitigate these risks, impacting operational efficiency. Current mainstream conflict avoidance or path planning methods, which focus on individual vehicles, cannot optimize the travel time for all vehicles in the entire intersection area, contradicting the concept of building efficient and smart airports. Summary of the Invention
[0005] In view of the deficiencies of the existing technology, the present invention provides a method and system for controlling special vehicles at apron intersections, which solves the existing problems.
[0006] The present invention adopts the following technical solution: In a first aspect, the present invention provides a method for controlling special vehicles at apron intersections, comprising the following steps: The system collects observation information for each special vehicle within the apron intersection area at the current moment. The observation information includes the environmental status information of each special vehicle and its relative motion status information with multiple adjacent special vehicles. The environmental status information includes the current time, coordinates within the apron intersection area, current speed, current heading angle, and entrance / exit type. The relative motion status information includes the relative positions and relative speeds of the nearest multiple adjacent vehicles at Euclidean distance. The observation information of each special vehicle at the current moment is input into the pre-trained policy network to generate corresponding action instructions, which include acceleration and deceleration actions, front wheel steering angle actions, and desired speed actions. The path trajectory of each special vehicle within a set time period is generated based on the current environmental state information and action commands. Each special vehicle travels according to the path trajectory within the set time period. After the travel is completed, the motion state of each special vehicle is updated to generate the environmental state information for the next moment.
[0007] Preferably, the apron intersection area includes multiple lanes and open areas, and each lane and open area has a corresponding entrance and exit point.
[0008] Preferably, the pre-training of the policy network includes the following steps: For the current training round, obtain the global status of all special vehicles in the apron intersection area at the current moment and the observation information of each special vehicle; The observation information of each special vehicle is input into the initial policy network, and the output is a distributed sample. The corresponding action is generated based on the distributed sample. The global state of all special vehicles and the observation information of each special vehicle are generated at the next moment based on the action of each special vehicle, and the corresponding instant reward is obtained based on the reward function. Based on immediate rewards, the value network generates corresponding value estimates for the current global state and the next global state; and generates a generalized advantage estimate based on the value estimate, and generates a corresponding discounted reward target through the generalized advantage estimate and the current value estimate. Repeat the above steps to obtain the experience pool for the current training round based on the global state, value estimation, generalized advantage estimation, discounted reward objective, observation information, actions, and reward functions at multiple time points; The value parameters of the value network are updated based on the mean squared error loss. The policy parameters of the policy network are updated based on policy loss and entropy regularization loss to obtain a pre-trained policy network.
[0009] Preferably, the reward function is as follows: ; In the formula, For the reward function, As the core reward, As a penalty for collision, To lock in the punishment, For smoothness penalty, , , and These are the weighting coefficients.
[0010] Preferably, the core reward is the sum of the differences between the distance from the starting point to the target point and the current distance to the target point for all feature vehicles.
[0011] Preferably, the value network and policy network adopt a Transformer structure.
[0012] Preferably, the path trajectory of each special vehicle within a set time period is generated based on the current environmental state information and action commands, specifically including the following steps: Construct a vehicle kinematic model; Based on the vehicle kinematics model, the path trajectory within a set time period is obtained by numerical integration using the Runge-Kutta method.
[0013] Preferably, when a special vehicle leaves the intersection area, the control of that vehicle ends; if a new special vehicle is added, a new global state is formed.
[0014] Preferably, the import / export type is used to characterize the intent to travel.
[0015] Secondly, the present invention provides a special vehicle control system for apron intersections, comprising: The data acquisition module is used to collect observation information of each special vehicle in the apron intersection area at the current moment. The observation information includes environmental state information of each special vehicle and relative motion state information with multiple adjacent special vehicles. The environmental state information includes the current time, coordinates in the apron intersection area, current speed, current heading angle, and entrance / exit type. The relative motion state information includes the relative position and relative speed of the nearest multiple adjacent vehicles in Euclidean distance. The generation module is used to input the observation information of each special vehicle at the current moment into the pre-trained policy network to generate corresponding action instructions, which include acceleration and deceleration actions, front wheel steering angle actions, and desired speed actions. The control module is used to generate the path trajectory of each special vehicle within a set time period in the future based on the current environmental state information and action commands; each special vehicle travels according to the path trajectory within the set time period, and after the travel is completed, the motion state of each special vehicle is updated to generate the environmental state information for the next moment.
[0016] Compared with the prior art, the above-mentioned at least one technical solution adopted by the present invention can achieve the following beneficial effects: This invention collects the vehicle's own state (position, speed, heading angle, and entrance / exit type) and relative motion information (relative position and speed) with multiple neighboring vehicles, enabling the policy network to simultaneously perceive the dynamic relationship between the vehicle and surrounding vehicles. The output action commands (acceleration / deceleration, front wheel steering angle, and desired speed) can drive multiple vehicles to collaboratively optimize traffic flow, rather than simply achieving obstacle avoidance for a single vehicle. Compared to traditional methods, this significantly reduces average waiting time and traffic delays within intersection areas. Attached Figure Description
[0017] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0018] Figure 1 This is a scene diagram of the apron intersection topology modeling proposed in this invention; Figure 2 This is a flowchart of the training framework for the intelligent decision-making algorithm for special vehicles at apron intersections proposed in this invention. Figure 3 This is a flowchart of the special vehicle operation control system for apron intersection proposed in this invention; Figure 4 This is a diagram of the traffic control system architecture for special vehicles at apron intersections proposed in this invention. Figure 5 This is a flowchart of a special vehicle control method for an apron intersection according to the present invention. Detailed Implementation
[0019] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0020] Reference Figures 1-5 This invention provides a method for controlling special vehicles at apron intersections, comprising the following steps: As attached Figure 1 As shown, the topology modeling of the apron intersection scenario is performed. To adapt to the characteristics of apron intersections—"no traffic light control, multi-directional openness, and mixed traffic"—this invention first performs topology modeling of the intersection area: on the two-way apron driving lanes, a [missing information - likely a specific type of road or structure] is defined. L×L area A This is an intersection area, which is essentially a multi-directional open intersection without traffic lights. There are three types of entrances and three types of exits for vehicle flow, and the interior is a mixed traffic area.
[0021] The intersection area is L×L Given a square, establish a Cartesian coordinate system: The bottom left corner is ( 0,0 ), the upper right corner is ( L,L The two lanes are arranged horizontally, with the lower lane... d for l 1 ≤ y ≤ l 2 upper lane u for l 2 ≤ y ≤ l 3 Below the driving lane is an unstructured, laneless area. Special vehicles can use the driving lane. u and d or lower boundary b There are three types of entrances and exits on the road. Define a discrete set of entrance and exit points:
[0022] Entry point: In b ( l, 0), l∈ [0 ,L ]、 In u ( L , l ), l∈ [ l 2 ,l3 ] In d (0 ,l ), l∈ [ l 1 ,l 2 ]; Exit point: Out b ( l, 0), l∈ [0 ,L ]、 Out u (0, l ), l∈ [ l 2 ,l 3 ] Out d ( L,l ), l∈ [ l 1 ,l 2 ]; Each vehicle has a clearly defined starting point and destination. OD Yes, it is used to indicate traffic intentions such as going straight, turning left, and turning right. For example: left → right straight, left → down right turn, down → left left turn.
[0023] A Markov decision process modeling method based on multi-agent deep reinforcement learning (MADRL) is proposed for handling traffic optimization problems under dynamic vehicle count conditions at apron intersections.
[0024] Time and agent set: Discretizing time into t= 1,2, … , K ( k For the agent to perform the first k The moment of the next decision), the set of vehicle intelligent agents is defined as follows: V i = {1,2,…, N}.in, This represents the number of vehicles detected in the current area, which can change dynamically as vehicles enter or leave the area.
[0025] State space: S ( k ) = {[ S 1 ,S 2 ,…,S N ] , N∈R}( RThe set of all vehicles detected within the region represents the current global environmental state information, where... S i = [ t i ( k ), ( x i ( k ), y i ( k )), v i ( k ), θ i ( k ), od i ] T For the first i The agent in the th... k The local state space before the next decision, where... t i ( k ) is the first k The time before the next decision, ( x i ( k ), y i ( k )) represents the current coordinates within the intersection area. v i ( k () represents the current speed. θ i ( k ( ) represents the current heading angle. od i This is an import / export type.
[0026] Local observation space: O i = [ S i ( k ) , Δ S i ( k ) ,V i ],in, S i ( k ) represents the vehicle's state space, Δ S i ( k () represents the relative positions and relative speeds of the nearest neighboring vehicles in Euclidean distance. V iThe vehicle number is represented and concatenated to form each intelligent agent. i Local observation space O i In this embodiment, there are specifically 7 vehicles.
[0027] Action space A= [ A 1, A 2,…, A N ],in A i = [ a i ( k ), δ i ( k ), v i ( k )] is the first i The first intelligent agent k The actions of this decision-making process, among which a i ( k )∈[-1,1] represents emergency acceleration / deceleration actions, and the adjustment amount for speed control is mapped to [ a min , a max ]; δ i ( k )∈[-1,1] represents the front wheel steering angle, which is mapped to [ as steering control]. δ min ,δ max ]; v i ( k )∈[0,1] represents the desired speed, which is used as a speed control mapping to [0,1]. ,v max ].
[0028] State transition probability P Defined as: , ; The instructions from the decision-making module are used to generate path points that are then tracked by the underlying controller. This is because vehicle kinematics are deterministic, and the environment automatically adjusts in the next frame whenever a vehicle enters or leaves. R Therefore, the state transition result is deterministic, meaning the state transition probability is 1. When a vehicle enters, a new state for that vehicle is added, forming a new set of agents; when a vehicle leaves, its state is deleted, and the agent's process ends.
[0029] reward function R Based on the global environmental state information of the decision point and the actions performed by the agent A i ( k Changes in traffic conditions at any given time are defined as follows: The core reward is the sum of the differences between the distances from the starting point to the target point and the current distance to the target point for all vehicles within the area. ; in: ; ; In the formula, The x-coordinate is the exit point. The x-coordinate is the entrance. The vertical coordinate is at the exit point. The vertical coordinate is the entrance. Collision penalty: ; In the formula, A large negative value indicates a very large penalty, less than -1000.
[0030] Locked penalty: ; To prevent vehicles from coming to a standstill, Indicates instantaneous velocity is less than 1 The vehicles.
[0031] Smoothness penalty: ; in, The longitudinal stationarity weighting coefficient is used. Let be the weighting coefficient for horizontal stationarity. R The negative number of the weighted sum of the absolute values of acceleration of all vehicles and the absolute values of steering angle of the front wheels is used to ensure the smoothness of vehicle driving.
[0032] The core reward is the sum of the differences between the starting point and the target point of all vehicles in the area and the current distance to the target point. It is combined with collision penalties, lock-up penalties to prevent stalling, and a smoothness penalty of the negative sum of the weighted sum of the absolute values of vehicle acceleration and front wheel steering angle in the area.
[0033] In summary, the first k The total global reward obtained after this decision is executed is as follows: ; in, , , and These are the weighting coefficients.
[0034] The Dy-MAPPO algorithm used in this invention employs an Actor-Critic architecture that features policy parameter sharing and global value estimation. Specifically, all agents share a single set of policy parameters. Actor Network:
[0035] ; in, It is a strategy parameter The strategy is represented.
[0036] Input: Single agent observation .
[0037] Output: Gaussian distribution parameters for continuous motion: ; ; in, It is the mean. It is variance. It follows a Gaussian distribution.
[0038] All agents adopt the value parameter as Centralized global Critic network: ; in, It is based on single-step public rewards. The defined first The expected discount return of the step-by-step approach, It is a discount factor. This is the expected symbol.
[0039] Input: The global state at the current moment .
[0040] Output: Scalar state value .
[0041] In this invention, the Actor-Critic network in the Dy-MAPPO algorithm needs to handle variable-length dynamic inputs. The Transformer performs excellently when handling variable-length inputs, dynamically adjusting its processing method according to the size of the state space at each time step, without being limited by fixed-length inputs. The Actor-Critic network, improved with Encoder-Decoder, ensures that the model exhibits strong collaborative capabilities in multi-agent systems.
[0042] When the state information at the apron intersection changes dynamically as the number of vehicles increases or decreases, the Transformer can automatically adjust its attention to adapt to this dynamic change based on the current state input, better capture the information of the network input, and help the Critic network estimate the state value and the probability distribution of Actor generation.
[0043] In several environmental interaction episodes, the vehicle uses the current policy. Sampling action execution Step 1: Collect experience trajectories. Utilize batch episode data for multiple epoch updates; for each epoch, the Actor-Critic network completes one update.
[0044] In each episode: Environment initialization, obtaining the initial global state. and local observations of each vehicle ; Observe its local area Enter vehicle Actor network; The Actor network outputs the mean and variance; A specific action is obtained by sampling based on this distribution. ; The actions of all vehicles constitute a combined action. Input environment; The environment performs joint actions and transitions to the next state. ; Returning the local observations for the next moment to each vehicle ; Each vehicle received an instant reward. .
[0045] After an episode ends, for each step Use Critic to analyze the global state and Value estimation and Calculate the generalized dominance estimate (residuals) : ; ; in, It is the decision-making step size. It is the GAE smoothing coefficient.
[0046] Calculate the discount return target for each step. : ; The processed data from the entire episode is used as the vehicle's trajectory to form an experience pool. ; in, It is a moment Number of vehicles in the area.
[0047] Centralized Critic updates bring the Critic's predicted values closer to the previously calculated target return. Calculate the mean squared error loss of the value function: ; in, It is a value estimate. It's a batch of data.
[0048] Minimize by gradient descent ,right Update.
[0049] For each vehicle Remember the old strategy before the last update. Define the probability ratio: ; The loss of the clip strategy in PPO is: ; in, It is the cutoff range parameter.
[0050] Adding an entropy regularization term encourages exploration: ; in It is the entropy coefficient.
[0051] The average total loss is calculated as follows: ; ; Parameters of all Actor networks Perform gradient descent updates.
[0052] This involves completing a Dy-MAPPO policy update, repeating the process multiple times on the experience pool data. In each epoch, the data can be shuffled and a new batch sampled. After the update is complete, the new policy and the new value estimator begin data collection for the next episode.
[0053] After training is completed and deployment is complete, for a single decision step The overall workflow is as follows: 1. Vehicle environmental sensors at all times After the update, the information will be converted into the current global state information. ; 2. Construct each vehicle Local observation space: ; 3. Use the same Actor network for all vehicles. Forward propagation: ; 4. Execution of actions and environmental evolution Use actions As a high-level command, the vehicle's kinematic state is updated through mid-level local path planning and low-level controller.
[0054] Vehicle kinematics model state transition matrix: ; The local path planning method uses the Runge-Kutta method for numerical integration to generate the vehicle's future path. s The path trajectory within.
[0055] The fourth-order Runge-Kutta method approximates the state at the next time step by calculating four intermediate values: ; ; ; ; in, It's the step length. It is the kinematic model of the vehicle.
[0056] The updated formula is as follows: ; Substituting this formula into the position of the state variable... ,speed Heading angle θ i Generate the future state of the vehicle and calculate the future state within each decision step. s The target position and direction. It can serve as a local target point for the vehicle, and this point must meet the desired speed and vehicle motion constraints. As the vehicle moves, the local target point will be dynamically adjusted according to the new state.
[0057] Each time a new round of decisions is made, new high-level instructions are input into the planning system. Using these new instructions and updated status, the future path is recalculated. In this way, the path is dynamically adjusted in real time based on the latest environmental information and vehicle status, and the path trajectory is continuously optimized to ensure that the vehicle avoids collisions in complex environments and travels along the optimal path.
[0058] Thus, a new environmental state is obtained. Vehicles will enter and exit the area during the environmental update process.
[0059] A vehicle drove in: New corresponding This forms a new vehicle group; A vehicle drove out: Delete its state and end the process for that vehicle.
[0060] As attached Figure 3 , attached Figure 4 As shown, the operational phase of this invention includes the following steps: After the agent is trained and deployed, the overall workflow for a single decision step preferably includes the following steps: Step T1: State initialization and sensor information encoding.
[0061] After the environmental sensors in the intersection area are updated, they encode the perceived data into a state structure that can be read by the decision-making module. This structure includes vehicle position, speed, heading, and OD, and outputs the current global state to the central control module.
[0062] Step T2: Generate the agent's observation space and select actions according to the policy probability.
[0063] Based on the encoded state structure, a local observation space is constructed for each agent. The forward propagation parameters of the Actor network are shared. The same network parameters are used as the decision module to output the action distribution of all agents and sample to obtain the high-level action instructions.
[0064] Step T3: Perform local path planning and generate local target points based on high-level instructions.
[0065] Using actions as high-level commands, the vehicle's kinematic state is updated by the local path planning module and the low-level control module. The local path planning method uses the Runge-Kutta numerical integration method to generate the vehicle's path trajectory within the next 2 seconds. The obtained target position and direction within the next 2 seconds are used as the vehicle's local target point, and the target point satisfies the desired speed and vehicle motion constraints.
[0066] Step T4: The underlying control tracks the local target point and performs rolling replanning.
[0067] The underlying control module tracks and controls local target points. As the vehicle moves, the local target points are dynamically adjusted according to the new state. Whenever a new round of decision-making occurs, new high-level instructions are input into the local path planning module. The new instructions and the updated state are used to recalculate the future path, thereby realizing real-time dynamic adjustment and continuous optimization of the path, ensuring that there are no collisions in complex environments and that the vehicle travels along the optimal path.
[0068] Step T5: Vehicle entry and exit processing and exit conditions.
[0069] If a vehicle enters during the environment update process, a new set of intelligent agents is formed by adding a corresponding state. If a vehicle leaves, its state is deleted and the process of that intelligent agent ends. The engineering rule for determining whether the current coordinates are within the area is used as the exit condition. If the vehicle leaves the area, the current round of control ends.
[0070] More preferably, a closed-loop safety mechanism of "path anomaly judgment - strategy readjustment - return to action selection" can be set up: when an anomaly is detected, the avoidance distance is increased, the expected speed is reduced, and the environment is reassessed and replanned until the abnormal area is safely passed.
[0071] The method and system for vehicle decision-making and control at apron roadway intersections provided by this invention can improve the efficiency of flight support operations and adapt to complex mixed traffic scenarios to achieve efficient airport construction.
[0072] Based on the same concept, the present invention also provides a special vehicle control system for apron intersections, including a data acquisition module, a generation module and a control module.
[0073] The data acquisition module is used to collect observation information of each special vehicle in the apron intersection area at the current moment. The observation information includes the environmental status information of each special vehicle and the relative motion status information with multiple adjacent special vehicles. The environmental status information includes the current time, coordinates in the apron intersection area, current speed, current heading angle and entrance / exit type. The relative motion status information includes the relative position and relative speed of the nearest multiple adjacent vehicles in Euclidean distance.
[0074] The generation module is used to input the observation information of each special vehicle at the current moment into the pre-trained policy network to generate corresponding action instructions, including acceleration and deceleration actions, front wheel steering angle actions, and desired speed actions.
[0075] The control module is used to generate the path trajectory of each special vehicle within a set time period in the future based on the current environmental state information and action commands; each special vehicle travels according to the path trajectory within the set time period in the future, and after the travel is completed, the motion state of each special vehicle is updated to generate the environmental state information for the next moment.
[0076] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including both the preferred embodiments and all changes and modifications falling within the scope of the invention.
[0077] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.
Claims
1. A method for controlling special vehicles at an apron intersection, characterized in that, Includes the following steps: The system collects observation information for each special vehicle within the apron intersection area at the current moment. The observation information includes the environmental status information of each special vehicle and its relative motion status information with multiple adjacent special vehicles. The environmental status information includes the current time, coordinates within the apron intersection area, current speed, current heading angle, and entrance / exit type. The relative motion status information includes the relative positions and relative speeds of the nearest multiple adjacent vehicles at Euclidean distance. The observation information of each special vehicle at the current moment is input into the pre-trained policy network to generate corresponding action instructions, which include acceleration and deceleration actions, front wheel steering angle actions, and desired speed actions. The path trajectory of each special vehicle within a set time period is generated based on the current environmental state information and action commands. Each special vehicle travels according to the path trajectory within the set time period. After the travel is completed, the motion state of each special vehicle is updated to generate the environmental state information for the next moment.
2. The method for controlling special vehicles at an apron intersection as described in claim 1, characterized in that, The apron intersection area includes multiple lanes and open areas, each lane and open area having a corresponding entrance and exit point.
3. The method for controlling special vehicles at an apron intersection as described in claim 1, characterized in that, The pre-training of the policy network includes the following steps: For the current training round, obtain the global status of all special vehicles in the apron intersection area at the current moment and the observation information of each special vehicle; The observation information of each special vehicle is input into the initial policy network, and the output is a distributed sample. The corresponding action is generated based on the distributed sample. The global state of all special vehicles and the observation information of each special vehicle are generated at the next moment based on the action of each special vehicle, and the corresponding instant reward is obtained based on the reward function. Based on immediate rewards, the value network generates corresponding value estimates for the current global state and the next global state; and generates a generalized advantage estimate based on the value estimate, and generates a corresponding discounted reward target through the generalized advantage estimate and the current value estimate. Repeat the above steps to obtain the experience pool for the current training round based on the global state, value estimation, generalized advantage estimation, discounted reward objective, observation information, actions, and reward functions at multiple time points; The value parameters of the value network are updated based on the mean squared error loss. The policy parameters of the policy network are updated based on policy loss and entropy regularization loss to obtain a pre-trained policy network.
4. The method for controlling special vehicles at an apron intersection as described in claim 3, characterized in that, The reward function is as follows: ； In the formula, For the reward function, As the core reward, As a penalty for collision, To lock in the punishment, For smoothness penalty, , , and These are the weighting coefficients.
5. The method for controlling special vehicles at an apron intersection as described in claim 4, characterized in that, The core reward is the sum of the differences between the distance from the starting point to the target point and the current distance to the target point for all characteristic vehicles.
6. The method for controlling special vehicles at an apron intersection as described in claim 3, characterized in that, The value network and policy network adopt a Transformer structure.
7. The method for controlling special vehicles at an apron intersection as described in claim 1, characterized in that, The path trajectory of each special vehicle within a set time period is generated based on the current environmental status information and action commands. The specific steps include: Construct a vehicle kinematic model; Based on the vehicle kinematics model, the path trajectory within a set time period is obtained by numerical integration using the Runge-Kutta method.
8. The method for controlling special vehicles at an apron intersection as described in claim 3, characterized in that, When a special vehicle leaves the intersection area, control of that vehicle ends. If a new special vehicle is added, a new global state is formed.
9. A method for controlling special vehicles at an apron intersection as described in claim 1, characterized in that, The import / export type is used to indicate the intent to travel.
10. A special vehicle control system for an apron intersection, characterized in that, include: The data acquisition module is used to collect observation information of each special vehicle in the apron intersection area at the current moment. The observation information includes environmental state information of each special vehicle and relative motion state information with multiple adjacent special vehicles. The environmental state information includes the current time, coordinates in the apron intersection area, current speed, current heading angle, and entrance / exit type. The relative motion state information includes the relative position and relative speed of the nearest multiple adjacent vehicles in Euclidean distance. The generation module is used to input the observation information of each special vehicle at the current moment into the pre-trained policy network to generate corresponding action instructions, which include acceleration and deceleration actions, front wheel steering angle actions, and desired speed actions. The control module is used to generate the path trajectory of each special vehicle within a set time period in the future based on the current environmental state information and action commands; each special vehicle travels according to the path trajectory within the set time period, and after the travel is completed, the motion state of each special vehicle is updated to generate the environmental state information for the next moment.