This application relates to the field of intelligent driving technology, specifically providing a proximal policy optimization training method,
system, storage medium, and vehicle. It aims to solve the technical problem of how to construct a PPO training
system based on
asynchronous computation without changing the PPO optimization objective and convergence characteristics. To this end, this application includes: a stage load-aware scheduling module that uniformly manages
data transmission and
model parameter version synchronization between the training data generation module, the policy and value joint sampling module, the reward and
advantage calculation module, and the policy and
value network optimization module, based on a
delay consistency alignment mechanism. Through the
delay consistency alignment mechanism, this application achieves effective control over asynchronously sampled trajectory data, reward values,
advantage values, and
model parameter updates, ensuring that the parameter update process conforms to the on-policy constraints of the PPO
algorithm. Simultaneously, it can improve the
parallel processing capability of the training
system and guarantee the stability and convergence of the PPO training process.