An autonomous driving online self-evolution learning method and system based on an asynchronous architecture

Abstract

The application discloses an automatic driving online self-evolution learning method and system based on an asynchronous architecture, which comprises the following steps: S1, establishing an automatic driving asynchronous closed-loop online interaction environment and completing initialization configuration; S2, running an asynchronous front-end closed-loop control process of a vehicle intelligent agent; S3, collecting closed-loop process interaction learning data and asynchronously delivering the data to a background learning system; S4, the background learning system generates a supervision signal based on interaction data according to an automatic supervision system, and forms online training samples in association with corresponding interaction data; S5, the background learning system executes model updating based on online training samples and historical experience samples and manages the experience samples; and S6, executing model updating verification and asynchronous deployment. The application overcomes the problem of missing true value labels in online learning reality, generates reliable supervision signals, combines a double-buffer real-time continuous learning mechanism and a champion-challenger safety gate, and realizes safe online self-evolution of an automatic driving model in a closed-loop running process.

Description

Technical Field

[0001] This invention relates to the fields of autonomous driving, artificial intelligence, and online learning, and in particular to an online self-evolutionary learning method and system for autonomous driving based on an asynchronous architecture. Background Technology

[0002] In recent years, significant progress has been made in end-to-end autonomous driving models based on deep learning. However, existing autonomous driving systems typically employ a static paradigm of "offline data collection - centralized cloud training - vehicle-based deployment." This paradigm is highly susceptible to distribution shift problems when facing complex, long-tailed, and dynamically changing traffic scenarios in the open world, leading to a decline in model performance. Therefore, endowing autonomous driving models with the ability to continuously learn and self-evolve online during closed-loop interactions has become a critical technical challenge that urgently needs to be addressed.

[0003] However, achieving online self-evolution for autonomous driving still faces technical bottlenecks. First, traditional closed-loop evaluation and training frameworks for autonomous driving often employ a synchronous stepping mechanism, where the time progression in the simulated world is forcibly blocked to wait for the agent's computation to complete. This mechanism masks the unavoidable system latency and computational power fluctuations in the real physical world, making models trained or evolved in such environments highly susceptible to decision-making errors due to latency issues during actual deployment. Second, in real closed-loop interactions, the system cannot obtain perfect human annotations or simulator privileged information as in offline datasets. Providing stable and reliable supervision and guidance for online model updates, relying solely on observable information from onboard sensors in the absence of globally real labels, remains a significant challenge. Furthermore, online updates in continuous data streams can easily lead to overfitting to the new scenario and forgetting previously acquired driving skills, resulting in catastrophic forgetting. Therefore, unverified online updates may introduce incorrect driving strategies, directly threatening the safety of closed-loop interactions.

[0004] Therefore, this invention proposes a novel online self-evolving learning method and system for autonomous driving, which enables the vehicle's intelligent agent to continuously collect online data, automatically generate reliable supervision signals, asynchronously execute background learning, reasonably manage experience samples, verify the model update effect, and complete the safe deployment of the model without interrupting the front-end driving execution process. Summary of the Invention

[0005] The purpose of this invention is to propose an online self-evolving learning method and system for autonomous driving based on an asynchronous architecture. First, an asynchronous closed-loop online interactive environment for autonomous driving is established, initializing the simulation world, vehicle agent, background learning system, automatic supervision system, and system communication interfaces, and configuring each system unit to operate collaboratively in an asynchronous manner. Second, the asynchronous front-end execution flow of the vehicle agent is run, enabling the vehicle agent to generate real-time driving decisions based on sensor data, vehicle status, and navigation information, and execute closed-loop vehicle control. Then, online learning data is collected during the closed-loop interaction process, continuously recording interaction data during the interaction between the vehicle agent and the simulation world, and asynchronously transmitting this interaction data to the background learning system. Next, automatic supervision signals required for online learning are generated. The automatic supervision system generates supervision signals based on the interaction data and associates these supervision signals with corresponding interaction data to form online training samples. Subsequently, background online learning and experience management are executed. The background learning system updates the model based on the online training samples and historical experience samples, and manages the experience samples. Finally, model update verification and asynchronous deployment are performed. The online updated model is verified, and models that meet the verification results are asynchronously updated to the front-end execution flow of the vehicle agent.

[0006] To achieve the above objectives, the technical solution provided by this invention is as follows:

[0007] An online self-evolutionary learning method and system for autonomous driving based on an asynchronous architecture, the method comprising:

[0008] S1. Establish an asynchronous closed-loop online interactive environment for autonomous driving, initialize the simulation world, vehicle intelligent agent, background learning system, automatic supervision system and system communication interface, and configure each system unit to operate collaboratively in an asynchronous manner;

[0009] S2. Run the asynchronous front-end execution process of the vehicle intelligent agent, enabling the vehicle intelligent agent to generate real-time driving decisions based on sensor data, vehicle status and navigation information, and execute closed-loop vehicle control.

[0010] S3. Collect online learning data during the closed-loop interaction process, continuously record interaction data during the interaction between the vehicle intelligent agent and the simulation world, and asynchronously transmit the interaction data to the background learning system;

[0011] S4. Generate automatic supervision signals required for online learning. The background learning system sends a supervision request to the automatic supervision system, which generates supervision signals based on the interaction data and associates the supervision signals with the corresponding interaction data to form online training samples.

[0012] S5. Perform background online learning and experience management. The background learning system performs model updates based on the online training samples and historical experience samples, and manages the experience samples.

[0013] S6. Perform model update verification and asynchronous deployment. Verify the online updated model and asynchronously update the qualified model to the front-end execution process of the vehicle intelligent agent based on the verification results.

[0014] To further explain, step S1 specifically includes:

[0015] S11. Start the main process, which initializes the simulation world, test map, test route, traffic signals, road topology, background traffic participants, vehicle agents, and system operating parameters.

[0016] S12. Configure sensor interfaces, vehicle status interfaces, navigation interfaces, control interfaces, front-end inference interfaces, interaction recording interfaces, and model update interfaces for the vehicle intelligent agent;

[0017] S13. Establish a system layer, an intelligent agent layer, and a learning layer, wherein the system layer is used to perform system initialization, lifecycle management, and global signal coordination; the intelligent agent layer is used to perform the front-end closed-loop operation of the vehicle intelligent agent; and the learning layer is used to perform background learning, automatic supervision, experience management, and model update control.

[0018] S14. Establish an inter-process communication mechanism to transmit sensor data, vehicle status, driving decisions, interactive data, supervision signals, training samples, and model update information between the simulation world, vehicle intelligent agent, background learning system, and automatic supervision system.

[0019] S15. Set the simulation world to advance with continuous time semantics so that the time advancement of the simulation world does not wait for the front-end execution process or the back-end learning process of the vehicle intelligent agent to be completed.

[0020] S16. Initialize the unified signal system, which is used to broadcast global control signals during scene initialization, scene reset, simulation start, abnormal interruption, learning consolidation, model update and system termination.

[0021] S17. Initialize the data structures required for online self-evolutionary learning, wherein the data structures include at least one of the following: learning queue, experience buffer, model version record, sample priority record, and interaction log.

[0022] To further explain, step S2 specifically involves:

[0023] S21. Start the sensor data acquisition process of the vehicle intelligent agent and acquire multimodal sensor data asynchronously. The multimodal sensor data includes at least one of images, lidar point clouds, inertial measurement data, global navigation satellite system data, vehicle status data, and navigation commands.

[0024] S22. Start the data processing process and perform time alignment, coordinate transformation, normalization, feature encoding or a combination thereof on the multimodal sensor data to obtain a unified input representation;

[0025] S23. Start the front-end inference process, which reads the latest available unified input representation and calls the current front-end execution model to generate real-time driving decisions;

[0026] S24. The front-end execution model is the current champion model, which is used to generate real-time driving decisions during the operation of the vehicle intelligent agent.

[0027] S25, the real-time driving decision includes at least one of the following: target speed, future trajectory, waypoint sequence, local path, behavior decision, steering control quantity, throttle control quantity, and braking control quantity;

[0028] S26. Start the underlying vehicle control process, which generates a vehicle control command based on the real-time driving decision and applies the vehicle control command to the vehicle intelligent agent.

[0029] S27. The sensor data acquisition process, data processing process, front-end inference process and underlying vehicle control process run in an asynchronous manner to ensure that the front-end execution process of the vehicle intelligent agent does not block the progress of the simulation world.

[0030] S28. When the front-end inference process does not output new real-time driving decisions in a timely manner, the underlying vehicle control process generates vehicle control commands based on the most recent effective driving decision, safety degradation strategy, trajectory extrapolation strategy, smooth braking strategy, or lane keeping strategy.

[0031] To further explain, step S3 specifically involves:

[0032] S31. During the closed-loop interaction between the vehicle intelligent agent and the simulation world, continuously record the input, output and control execution process of the vehicle intelligent agent to form interactive data;

[0033] S32, the interactive data includes at least one of the following: unified input representation, vehicle pose, vehicle speed, vehicle acceleration, navigation command, historical trajectory, real-time driving decision, vehicle control command, simulation timestamp, physical wall clock timestamp, scene identifier, route identifier, traffic participant status, and traffic signal status;

[0034] S33. Assign an interaction record identifier to the interaction data. The interaction record identifier is used to associate input data, model output, control execution result, supervision signal, training sample, sample priority and learning state.

[0035] S34. Write the interaction data into the learning queue, and have the background learning system read the interaction data in the learning queue in an asynchronous consumer manner;

[0036] S35. The process of writing and reading the interactive data does not block the front-end reasoning process of the vehicle intelligent agent, the underlying vehicle control process, and the simulation world propulsion process.

[0037] To further explain, step S4 specifically involves:

[0038] S41. The background learning system reads the interaction data from the learning queue and sends a supervision request to the automatic supervision system.

[0039] S42. The automatic monitoring system generates a monitoring signal based on the monitoring request, the monitoring signal including a target velocity monitoring signal and a future trajectory monitoring signal;

[0040] S43. The background learning system pairs the supervision signal with the corresponding interaction data to construct online training samples;

[0041] S44. The online training samples include at least one of the following: model input, supervision label, sample timestamp, interaction record identifier, and sample priority field.

[0042] To further explain, in step S42, the automatic monitoring system generates the monitoring signal using at least one of the following methods:

[0043] S411. Local privileged expert supervision method based on simulator privileged information, which generates supervision signals by using map information, traffic participant status, traffic signal status, drivable area information, collision risk information or expert trajectory information provided by the simulator.

[0044] S412. A multimodal large language model integrated supervision method based on observable inputs, which generates supervision signals by utilizing observable sensor data, vehicle status, navigation commands and scene context of vehicle intelligent agents;

[0045] S413. A hybrid supervision method based on local privileged expert supervision and multimodal large language model integrated supervision, which selects or merges supervision signals from different sources according to preset conditions.

[0046] To further explain, the specific process of the multimodal large language model integration supervision method in step S412 is as follows:

[0047] S421. Construct a two-layer parallel inference architecture, which includes a multi-agent parallel layer and an integrated expert parallel layer.

[0048] S422. In the multi-agent parallel layer, multiple independent multimodal large language model agents are created to process supervised generation tasks corresponding to different interactive data in parallel.

[0049] S423. In the parallel layer of integrated experts, each multimodal large language model agent calls multiple multimodal large language model experts to generate multiple candidate supervision results for the same interactive data.

[0050] S424. Perform structured parsing, anomaly filtering, and validity verification on the multiple candidate supervision results;

[0051] S425. Perform consistency aggregation on the candidate supervision results that have passed the validity check to obtain the final supervision signal;

[0052] S426. The final supervision signal is returned to the background learning system for constructing online training samples.

[0053] To further explain, the specific process of consistent aggregation in step S425 is as follows:

[0054] S431. Extract candidate target velocities from multiple candidate supervision results, perform robust statistics on candidate target velocities, perform outlier detection based on median absolute deviation, screen outlier candidate velocities, and obtain the mainstream candidate set of target velocities.

[0055] S432. Select the original candidate target velocity that is close to the consensus center from the mainstream candidate set of target velocities as the final target velocity supervision signal;

[0056] S433. Extract candidate future trajectories from multiple candidate supervision results, and map each candidate future trajectory to a trajectory feature used to characterize the geometric trend of the trajectory.

[0057] S434. Based on the trajectory features, perform robust statistics on the candidate future trajectories, perform outlier detection based on median absolute deviation, filter out outlier candidate trajectories, and obtain the mainstream candidate set of future trajectories.

[0058] S435. Perform point-by-point coordinate averaging, weighted averaging, or robust averaging on the candidate future trajectories in the mainstream candidate set of future trajectories to obtain the final future trajectory supervision signal.

[0059] S436. Combine the final target velocity monitoring signal and the final future trajectory monitoring signal into the final monitoring signal corresponding to the current interactive data.

[0060] To further explain, the multimodal large language model integration supervision method in step S412 also includes an automatic supervision signal consistency verification process, specifically:

[0061] S441. Obtain the multimodal large language model integration supervision signal and the reference supervision signal. The multimodal large language model integration supervision signal is the final supervision signal obtained by the consensus aggregation. The reference supervision signal includes at least one of the following: simulator privilege supervision signal, rule security module output signal, supervision signal generated by manual takeover, historical high confidence supervision signal, and multi-expert consensus supervision signal.

[0062] S442. Based on the integration of future trajectories in the supervision signal and future trajectories in the reference supervision signal using a multimodal large language model, calculate the trajectory direction consistency index. The trajectory direction consistency index is determined based on at least one of the following: the angle between future trajectory direction vectors, cosine similarity, turning trend consistency, or trajectory tangential direction consistency.

[0063] S443. Based on the target velocity in the integrated supervision signal and the target velocity in the reference supervision signal of the multimodal large language model, calculate the velocity trend consistency index. The velocity trend consistency index is determined according to at least one of the following: acceleration trend consistency, deceleration trend consistency, velocity change direction consistency, or velocity range consistency among target velocities.

[0064] S444. Determine the semantic reliability of the multimodal large language model integration supervision signal based on the trajectory direction consistency index and the velocity trend consistency index;

[0065] S445. Based on the semantic reliability, perform retention, weight reduction, correction, or discard processing on the integrated supervision signal of the multimodal large language model.

[0066] To further explain, step S5 specifically involves:

[0067] S51. Establish a short-term experience buffer and a long-term experience buffer, wherein the short-term experience buffer is used to store newly arrived online training samples, and the long-term experience buffer is used to store historical experience samples.

[0068] S52. Calculate the sample priority for the online training samples and write the newly arrived online training samples and their sample priorities into the short-term experience buffer.

[0069] S53. The background learning system performs hybrid replay sampling based on short-term and long-term experience buffers to obtain experience samples for training.

[0070] S54. In the background learning system, the online update model is used as the challenger model, and the challenger model is trained and updated based on the sampled experience samples and their supervision signals.

[0071] S55. During the challenger model training process, the front-end execution flow of the vehicle agent is not blocked, and the current front-end execution model, which is the champion model, continues to generate real-time closed-loop driving decisions.

[0072] To further explain, the calculation process of sample priority in step S52 includes:

[0073] S511. Perform forward computation on the online training samples using the current champion model or a frozen copy of the current champion model;

[0074] S512. Calculate the reference loss corresponding to the current champion model based on the supervision signal of the online training samples;

[0075] S513. Use the reference loss or the value obtained based on the reference loss as the initial sample priority for online training samples;

[0076] S514. The online training samples, initial sample priority, and champion reference loss are associated and stored in a short-term experience buffer.

[0077] To further explain, the specific process of mixed playback sampling in step S53 is as follows:

[0078] S521. Calculate the sum of sample priorities in the short-term experience buffer and the long-term experience buffer, respectively.

[0079] S522. Determine the buffer selection probability based on the sample priority of the short-term experience buffer and the long-term experience buffer;

[0080] S523. Select a target buffer from a short-term experience buffer or a long-term experience buffer according to the buffer selection probability.

[0081] S524. Select empirical samples within the target buffer based on sample priority;

[0082] S525. Calculate the importance sampling weights based on the sampling probability of empirical samples;

[0083] S526. Use the importance sampling weights to correct the training bias introduced by non-uniform sampling.

[0084] To further explain, the management of experience samples in step S5 also includes an experience buffer consolidation process, specifically:

[0085] S531. After completing the model update verification, merge the experience samples in the short-term experience buffer into the long-term experience buffer.

[0086] S532. Determine whether the merged long-term experience buffer exceeds the preset capacity;

[0087] S533. When the merged long-term experience buffer does not exceed the preset capacity, all experience samples are retained.

[0088] S534. When the merged long-term experience buffer exceeds the preset capacity, sample forgetting processing is performed.

[0089] S535. Clear the short-term experience buffer and enter the next online learning cycle.

[0090] To further explain, the sample forgetting process in step S534 is an inverse priority probabilistic forgetting process, specifically including:

[0091] S541. Calculate the discard probability based on the sample priority of the empirical sample, wherein the discard probability is negatively correlated with the sample priority;

[0092] S542. Select samples to be discarded from the empirical sample set that exceeds the capacity limit based on the discard probability;

[0093] S543. Delete the samples to be discarded to obtain the updated long-term experience buffer.

[0094] To further explain, step S6 specifically involves:

[0095] S61. When the preset update conditions are met, the model update verification process is triggered.

[0096] S62. Calculate the validation results of the online updated model based on the empirical samples in the empirical buffer;

[0097] S63. Compare the verification result with the reference result corresponding to the current front-end execution model;

[0098] S64. When the online update model meets the preset deployment conditions, the online update model will be determined as the new front-end execution model.

[0099] S65. When the online update model does not meet the preset deployment conditions, it is prohibited to deploy it to the front-end execution process of the vehicle intelligent agent.

[0100] S66. Update at least one of the following based on the verification results: model version record, experience sample status, and sample priority.

[0101] To further explain, the model update verification process in step S61 adopts a champion-challenger safety gating mechanism, specifically including:

[0102] S611. Calculate the challenger loss for all or part of the experience samples in the short-term and long-term experience buffers using the updated challenger model.

[0103] S612, Read the champion reference loss stored in the corresponding experience sample;

[0104] S613. Summarize the challenger's loss and the champion's reference loss to obtain the challenger's global loss and the champion's global loss.

[0105] S614. Compare the global loss of the challenger and the global loss of the champion, and determine whether the challenger model meets the preset promotion conditions based on the comparison results.

[0106] S615. When the challenger model meets the promotion conditions, the challenger model will be promoted to the new champion model, and a model update notification will be sent to the front-end inference process.

[0107] S616. When the challenger model does not meet the promotion criteria, roll back the challenger model and keep the current champion model as the front-end execution model.

[0108] To further explain, the specific process of challenger model promotion and rollback in steps S615 and S616 is as follows:

[0109] S621. When the challenger model meets the promotion conditions, copy, map, register or save the challenger model's model parameters as the new champion model parameters.

[0110] S622. Recalculate the loss of all or part of the experience samples in the experience buffer using the new champion model;

[0111] S623. Update the sample priority and champion reference loss of the corresponding empirical sample based on the recalculated loss;

[0112] S624. When the challenger model does not meet the promotion conditions, roll back at least one of the challenger model's model parameters, optimizer state, and learning rate scheduler state to the state corresponding to the current champion model.

[0113] S625. After the challenger model is rolled back, the update results of the current challenger model are prohibited from entering the front-end execution process of the vehicle intelligent agent.

[0114] To further explain, the specific process of asynchronous deployment in step S6 is as follows:

[0115] S631. The front-end inference process listens for model update notifications from the back-end learning system or unified signal system.

[0116] S632. When the front-end inference process receives a model update notification, it reads the model version number, model weight path, parameter pointer, or shared memory identifier of the new front-end execution model.

[0117] S633, The front-end inference process loads new model weights or switches to new model parameters between adjacent inference cycles;

[0118] S634. During model switching, sensor data acquisition, data processing, underlying vehicle control, and simulation world advancement remain uninterrupted.

[0119] S635. After the model switch is completed, the front-end inference process continues to generate real-time driving decisions using the new front-end execution model.

[0120] The beneficial effects of this invention are:

[0121] 1. This invention utilizes a multimodal large language model integrated supervision method to automatically generate target speed supervision signals and future trajectory supervision signals based on the interaction data generated during the closed-loop operation of the vehicle intelligent agent. This transforms unannotated online interaction data into online training samples that can be used for model updates, reducing the dependence of online learning on human ground truth labels and overcoming the problem of missing ground truth labels in real-world applications of online learning.

[0122] 2. This invention generates candidate supervision results in parallel through multiple multimodal large language model agents or experts, and performs structured parsing, anomaly filtering, validity verification and consistency aggregation on the candidate supervision results. At the same time, it combines trajectory direction consistency index and velocity trend consistency index to verify the semantic reliability of the supervision signal, thereby reducing the impact of abnormal supervision signals on the model training process.

[0123] 3. This invention uses a dual-buffering real-time continuous learning mechanism with short-term and long-term experience buffers to manage newly arrived online training samples and historical experience samples respectively. It also combines hybrid replay sampling, sample priority calculation and inverse priority probability forgetting processing to maintain the ability to remember historical scenes while learning new scene experience.

[0124] 4. This invention adopts a champion-challenger safety gating mechanism to control the verification, promotion and deployment of the model, avoiding the direct deployment of the performance-degraded model to the vehicle agent. At the same time, the asynchronous mechanism decouples the background model training, model verification and front-end model switching, thereby completing the seamless model update without affecting the continuity of the vehicle agent's front-end driving execution.

[0125] 5. This invention can be applied to end-to-end online learning of autonomous driving models, closed-loop continuous optimization of autonomous driving models, continuous learning of embodied intelligent agents, and self-evolutionary learning for robot safety in complex dynamic environments. Attached Figure Description

[0126] Figure 1 A schematic diagram of the overall process of this invention;

[0127] Figure 2 Diagram of the multimodal large language model integration supervision architecture of this invention;

[0128] Figure 3 Diagram of the double-buffered experience management mechanism of this invention;

[0129] Figure 4 Diagram of the Champion-Challenger security gating mechanism of this invention. Detailed Implementation

[0130] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0131] like Figure 1 As shown, this invention is an online self-evolving learning method and system for autonomous driving based on an asynchronous architecture. In this embodiment, the asynchronous closed-loop online interactive environment for autonomous driving is built based on the CARLA simulator. The asynchronous closed-loop interactive logic, unified signal system, communication interface, data acquisition, background learning system, and automatic supervision system are implemented based on Python, including the following steps:

[0132] S1. Establish an asynchronous closed-loop online interactive environment for autonomous driving, initialize the simulation world, vehicle intelligent agent, background learning system, automatic supervision system and system communication interface, and configure each system unit to operate collaboratively in an asynchronous manner;

[0133] S2. Run the asynchronous front-end execution process of the vehicle intelligent agent, enabling the vehicle intelligent agent to generate real-time driving decisions based on sensor data, vehicle status and navigation information, and execute closed-loop vehicle control.

[0134] S3. Collect online learning data during the closed-loop interaction process, continuously record interaction data during the interaction between the vehicle intelligent agent and the simulation world, and asynchronously transmit the interaction data to the background learning system;

[0135] S4. Generate automatic supervision signals required for online learning. The automatic supervision system generates supervision signals based on the interaction data and associates the supervision signals with the corresponding interaction data to form online training samples.

[0136] S5. Perform background online learning and experience management. The background learning system performs model updates based on the online training samples and historical experience samples, and manages the experience samples.

[0137] S6. Perform model update verification and asynchronous deployment. Verify the online updated model and asynchronously update the qualified model to the front-end execution process of the vehicle intelligent agent based on the verification results.

[0138] To further explain, step S1 specifically includes:

[0139] Establish an asynchronous closed-loop online interactive environment for autonomous driving, initialize the simulation world, vehicle intelligent agent, background learning system, automatic supervision system and system communication interface, and configure each system unit to operate collaboratively in an asynchronous manner;

[0140] S11. Start the main process, which initializes the simulation world, test map, test route, traffic signals, road topology, background traffic participants, vehicle agents, and system operating parameters.

[0141] S12. Configure sensor interfaces, vehicle status interfaces, navigation interfaces, control interfaces, front-end inference interfaces, interaction recording interfaces, and model update interfaces for the vehicle intelligent agent;

[0142] S13. Establish a system layer, an intelligent agent layer, and a learning layer, wherein the system layer is used to perform system initialization, lifecycle management, and global signal coordination; the intelligent agent layer is used to perform the front-end closed-loop operation of the vehicle intelligent agent; and the learning layer is used to perform background learning, automatic supervision, experience management, and model update control.

[0143] S14. Establish an inter-process communication mechanism to transmit sensor data, vehicle status, driving decisions, interactive data, supervision signals, training samples, and model update information between the simulation world, vehicle intelligent agent, background learning system, and automatic supervision system.

[0144] S15. Set the simulation world to advance with continuous time semantics so that the time advancement of the simulation world does not wait for the front-end execution process or the back-end learning process of the vehicle intelligent agent to be completed.

[0145] S16. Initialize the unified signal system, which is used to broadcast global control signals during scene initialization, scene reset, simulation start, abnormal interruption, learning consolidation, model update and system termination.

[0146] S17. Initialize the data structures required for online self-evolutionary learning, wherein the data structures include at least one of the following: learning queue, experience buffer, model version record, sample priority record, and interaction log;

[0147] To further explain, step S2 specifically involves:

[0148] The asynchronous front-end execution process of the vehicle intelligent agent enables the vehicle intelligent agent to generate real-time driving decisions based on sensor data, vehicle status and navigation information, and execute closed-loop vehicle control.

[0149] S21. Start the sensor data acquisition process of the vehicle intelligent agent and acquire multimodal sensor data asynchronously. The multimodal sensor data includes at least one of images, lidar point clouds, inertial measurement data, global navigation satellite system data, vehicle status data, and navigation commands.

[0150] S22. Start the data processing process and perform time alignment, coordinate transformation, normalization, feature encoding or a combination thereof on the multimodal sensor data to obtain a unified input representation;

[0151] S23. Start the front-end inference process, which reads the latest available unified input representation and calls the current front-end execution model to generate real-time driving decisions;

[0152] S24. The front-end execution model is the current champion model, which is used to generate real-time driving decisions during the operation of the vehicle intelligent agent.

[0153] S25, the real-time driving decision includes at least one of the following: target speed, future trajectory, waypoint sequence, local path, behavior decision, steering control quantity, throttle control quantity, and braking control quantity;

[0154] S26. Start the underlying vehicle control process, which generates a vehicle control command based on the real-time driving decision and applies the vehicle control command to the vehicle intelligent agent.

[0155] S27. The sensor data acquisition process, data processing process, front-end inference process and underlying vehicle control process run in an asynchronous manner to ensure that the front-end execution process of the vehicle intelligent agent does not block the progress of the simulation world.

[0156] S28. When the front-end inference process does not output new real-time driving decisions in a timely manner, the underlying vehicle control process generates vehicle control commands based on the most recent effective driving decision, safety degradation strategy, trajectory extrapolation strategy, smooth braking strategy, or lane keeping strategy.

[0157] To further explain, step S3 specifically involves:

[0158] Collect online learning data during the closed-loop interaction process, continuously record interaction data during the interaction between the vehicle intelligent agent and the simulation world, and asynchronously transmit the interaction data to the background learning system;

[0159] S31. During the closed-loop interaction between the vehicle intelligent agent and the simulation world, continuously record the input, output and control execution process of the vehicle intelligent agent to form interactive data;

[0160] S32, the interactive data includes at least one of the following: unified input representation, vehicle pose, vehicle speed, vehicle acceleration, navigation command, historical trajectory, real-time driving decision, vehicle control command, simulation timestamp, physical wall clock timestamp, scene identifier, route identifier, traffic participant status, and traffic signal status;

[0161] S33. Assign an interaction record identifier to the interaction data. The interaction record identifier is used to associate input data, model output, control execution result, supervision signal, training sample, sample priority and learning state.

[0162] S34. Write the interaction data into the learning queue, and have the background learning system read the interaction data in the learning queue in an asynchronous consumer manner;

[0163] S35. The process of writing and reading the interactive data does not block the front-end reasoning process of the vehicle intelligent agent, the underlying vehicle control process, and the simulation world propulsion process.

[0164] To further explain, step S4 specifically involves:

[0165] Automatic supervision signals required for online learning are generated by an automatic supervision system based on the interaction data, and the supervision signals are associated with the corresponding interaction data to form online training samples.

[0166] S41. The background learning system reads the interaction data from the learning queue and sends a supervision request to the automatic supervision system.

[0167] S42. The automatic monitoring system generates a monitoring signal based on the monitoring request, the monitoring signal including a target velocity monitoring signal and a future trajectory monitoring signal;

[0168] S43. The background learning system pairs the supervision signal with the corresponding interaction data to construct online training samples;

[0169] S44. The online training samples include at least one of the following: model input, supervision label, sample timestamp, interaction record identifier, and sample priority field;

[0170] To further explain, in step S42, the automatic monitoring system generates the monitoring signal using at least one of the following methods:

[0171] S411. Local privileged expert supervision method based on simulator privileged information, which generates supervision signals by using map information, traffic participant status, traffic signal status, drivable area information, collision risk information or expert trajectory information provided by the simulator.

[0172] S412. Based on the current position of the vehicle intelligent agent, the global navigation route, the road topology and the lane centerline, generate a candidate reference path within a preset distance range in front of the vehicle intelligent agent. This path extends along the global navigation route in priority. When an obstacle, lane blockage or traffic rule constraint is detected in front, an alternative candidate path can be generated based on adjacent drivable lanes, drivable areas or expert trajectories.

[0173] S413. Based on traffic signal status, stop line position, lane direction, lane boundary, drivable area information and road topology constraints, the candidate reference path is screened for legality, and candidate paths that cross non-drivable areas, violate lane direction, cross red light stop lines, cross stop sign constraint areas or are inconsistent with road topology are eliminated.

[0174] S414. Based on the status of surrounding traffic participants and collision risk information, conduct a safety assessment of the candidate reference path, calculate the predicted distance, time and collision or occupation area overlap relationship between the vehicle intelligent agent and surrounding traffic participants when moving along the candidate reference path, and eliminate candidate paths with collision risk exceeding a preset threshold, or perform deceleration, yielding, lane changing or local offset correction on the candidate path.

[0175] S415. Select the candidate path with the lowest cost from the candidate reference paths that meet the requirements of legality and security as the expert reference trajectory, and parameterize it in time to obtain the future trajectory supervision signal.

[0176] S416. Generate a target speed monitoring signal based on at least one of local speed limit, traffic signal status, distance to the vehicle in front, collision risk, road curvature, and expert trajectory speed; wherein, when there are no traffic rules and no collision risk, the target speed does not exceed the current road speed limit, but when there is a red light, stop sign, slow-moving vehicle in front, or potential collision risk ahead, the target speed is reduced according to the stopping distance, safe distance, or time to collision.

[0177] S417. Combine the future trajectory supervision signal and the target velocity supervision signal into a local privileged expert supervision signal, and then associate it with the corresponding interaction record identifier, simulation timestamp and sample status before returning it to the background learning system.

[0178] S418. A multimodal large language model integrated supervision method based on observable input, which generates supervision signals by utilizing observable sensor data, vehicle status, navigation instructions and scene context of vehicle intelligent agents;

[0179] S419. A hybrid supervision method based on local privileged expert supervision and multimodal large language model integrated supervision, which selects or merges supervision signals from different sources according to preset conditions;

[0180] To further explain, such as Figure 2 As shown, the specific process of the multimodal large language model integration supervision method in step S418 is as follows:

[0181] S421. Construct a two-layer parallel inference architecture, which includes a multi-agent parallel layer and an integrated expert parallel layer.

[0182] S422. Organize the sensor data observable by the vehicle intelligent agent, vehicle status, navigation instructions and scene context into a multimodal supervision request. The multimodal supervision request includes at least one of image frames, LiDAR projection map, BEV grid map, vehicle speed, vehicle acceleration, vehicle heading angle, historical trajectory, navigation target, local route, traffic signal status and relative status of surrounding traffic participants, and requires the multimodal large language model to output the target speed and future trajectory.

[0183] S423. In the multi-agent parallel layer, multiple independent multimodal large language model agents are created to process supervised generation tasks corresponding to different interactive data in parallel.

[0184] S424. In the parallel layer of integrated experts, each multimodal large language model agent calls multiple multimodal large language model experts to generate multiple candidate supervision results for the same interactive data.

[0185] S425. Perform structured parsing, anomaly filtering, and validity verification on the multiple candidate supervision results;

[0186] S426. Perform consistency aggregation on the candidate supervision results that have passed the validity check to obtain the final supervision signal;

[0187] S427. The final supervision signal is returned to the background learning system for constructing online training samples;

[0188] To further explain, the specific process of consistent aggregation in step S426 is as follows:

[0189] S431. Extract candidate target velocities from multiple candidate supervision results, perform robust statistics on candidate target velocities, perform outlier detection based on median absolute deviation, screen outlier candidate velocities, and obtain the mainstream candidate set of target velocities.

[0190] S432. Calculate the consensus center of the mainstream candidate set of target speeds, wherein the consensus center is the median, mean, weighted mean, or truncated mean of the mainstream candidate set of target speeds, and then select the original candidate target speed that is closest to the consensus center from the mainstream candidate set of target speeds as the final target speed monitoring signal.

[0191] S433. Extract candidate future trajectories from multiple candidate supervision results, and map each candidate future trajectory to a trajectory feature used to characterize the geometric trend of the trajectory.

[0192] S434. Based on the trajectory features, perform robust statistics on the candidate future trajectories, perform outlier detection based on median absolute deviation, filter out outlier candidate trajectories, and obtain the mainstream candidate set of future trajectories.

[0193] S435. Perform point-by-point coordinate averaging, weighted averaging, or robust averaging on the candidate future trajectories in the mainstream candidate set of future trajectories to obtain the final future trajectory supervision signal.

[0194] S436. Combine the final target velocity monitoring signal and the final future trajectory monitoring signal into the final monitoring signal corresponding to the current interactive data;

[0195] To further explain, the multimodal large language model integration supervision method in step S418 also includes an automatic supervision signal consistency verification process, specifically:

[0196] S441. Obtain the multimodal large language model integration supervision signal and the reference supervision signal. The multimodal large language model integration supervision signal is the final supervision signal obtained through the consensus aggregation. The reference supervision signal includes at least one of the following: simulator privileged supervision signal, rule safety module output signal, supervision signal generated by manual takeover, historical high-confidence supervision signal, and multi-expert consensus supervision signal. The simulator privileged supervision signal refers to the reference supervision signal generated based on the map truth value, traffic participant truth value state, traffic signal truth value state, drivable area truth value, collision risk truth value, or expert trajectory provided by the simulator. The rule safety module output signal refers to the reference supervision signal generated by the rule safety module based on traffic rules, safety distance rules, collision risk thresholds, etc. The reference supervision signals generated by road speed limits, traffic signal status, lane boundaries, drivable area constraints, or vehicle dynamics constraints; the supervision signals generated by manual takeover refer to the reference supervision signals generated when a human driver, safety officer, or remote operator takes over or corrects vehicle control during the operation of the vehicle intelligent agent; the historical high-confidence supervision signals refer to the supervision signals that have passed consistency verification, safety rule verification, manual confirmation, or model verification in historical interaction data; the multi-expert consensus supervision signals refer to the reference supervision signals obtained by voting, weighted fusion, averaging, robust statistics, or consistency aggregation after multiple expert models, rule modules, simulator privileged experts, or human supervision sources generate candidate supervision results for the same interaction data respectively.

[0197] S442. Based on the integration of future trajectories in the supervision signal and future trajectories in the reference supervision signal using a multimodal large language model, calculate the trajectory direction consistency index. The trajectory direction consistency index is determined based on at least one of the following: the angle between future trajectory direction vectors, cosine similarity, turning trend consistency, or trajectory tangential direction consistency.

[0198] S443. Based on the target velocity in the integrated supervision signal and the target velocity in the reference supervision signal of the multimodal large language model, calculate the velocity trend consistency index. The velocity trend consistency index is determined according to at least one of the following: acceleration trend consistency, deceleration trend consistency, velocity change direction consistency, or velocity range consistency among target velocities.

[0199] S444. Determine the semantic reliability of the multimodal large language model integration supervision signal based on the trajectory direction consistency index and the velocity trend consistency index;

[0200] S445. Based on the semantic reliability, perform retention, weight reduction, correction, or discard processing on the integrated supervision signal of the multimodal large language model;

[0201] To further explain, step S5 specifically involves:

[0202] The system performs online learning and experience management in the background. Based on the online training samples and historical experience samples, the background learning system updates the model and manages the experience samples, such as... Figure 3 As shown;

[0203] S51. Establish a short-term experience buffer and a long-term experience buffer, wherein the short-term experience buffer is used to store newly arrived online training samples, and the long-term experience buffer is used to store historical experience samples.

[0204] S52. Calculate the sample priority for the online training samples and write the newly arrived online training samples and their sample priorities into the short-term experience buffer.

[0205] S53. The background learning system performs hybrid replay sampling based on short-term and long-term experience buffers to obtain experience samples for training.

[0206] S54. In the background learning system, the online update model is used as the challenger model, and the challenger model is trained and updated based on the sampled experience samples and their supervision signals.

[0207] S55. During the challenger model training process, the front-end execution flow of the vehicle agent is not blocked, and the current front-end execution model, which is the champion model, continues to generate real-time closed-loop driving decisions.

[0208] To further explain, the calculation process of sample priority in step S52 includes:

[0209] S511. Perform forward computation on the online training samples using the current champion model or a frozen copy of the current champion model;

[0210] S512. Calculate the reference loss corresponding to the current champion model based on the supervision signal of the online training samples;

[0211] S513. Use the reference loss or the value obtained based on the reference loss as the initial sample priority for online training samples;

[0212] S514. The online training samples, initial sample priority, and champion reference loss are associated and stored in a short-term empirical buffer.

[0213] To further explain, the specific process of mixed playback sampling in step S53 is as follows:

[0214] S521. Calculate the sum of sample priorities in the short-term experience buffer and the long-term experience buffer, respectively.

[0215] S522. Based on the sum of sample priorities of the short-term experience buffer and the sum of sample priorities of the long-term experience buffer, a weighted normalization method is used to determine the buffer selection probability; where the probability of the short-term experience buffer being selected as the target buffer is... The weighted priority of the short-term experience buffer and its proportion of the sum of the weighted priorities of the two experience buffers represent the probability that the long-term experience buffer will be selected as the target buffer. The weighted priority of the long-term experience buffer and its proportion of the sum of the weighted priorities of the two experience buffers are calculated as follows:

[0216]

[0217]

[0218] in, and These represent the sum of sample priorities in the short-term and long-term experience buffers, respectively. and These represent the preset weighting coefficients for the short-term and long-term experience buffers, respectively.

[0219] S523. Select a target buffer from a short-term experience buffer or a long-term experience buffer according to the buffer selection probability.

[0220] S524. Select empirical samples within the target buffer based on sample priority;

[0221] S525. Calculate the importance sampling weights based on the sampling probability of empirical samples; for empirical samples... If it is located in the target buffer First, based on the selection probability of the target buffer and the priority sampling probability of the empirical sample within the target buffer, the empirical sample is determined. Overall sampling probability:

[0222]

[0223] in, Representing empirical samples The overall sampling probability, Represents a buffer The probability of being selected as the target buffer. Representing empirical samples Sample priority, This represents the priority sampling intensity parameter; then, the importance sampling weight is calculated based on the overall sampling probability.

[0224]

[0225] in Representing empirical samples Importance sampling weights This represents the total number of empirical samples in the short-term and long-term empirical buffers. Indicates the importance sampling correction intensity coefficient;

[0226] S526. To avoid training instability caused by excessive weights, the importance sampling weights are normalized, and the training loss corresponding to the empirical samples is weighted using the normalized importance sampling weights, thereby correcting the training bias introduced by non-uniform sampling.

[0227] To further explain, the management of experience samples in step S5 also includes an experience buffer consolidation process, specifically:

[0228] S531. After completing the model update verification, merge the experience samples in the short-term experience buffer into the long-term experience buffer.

[0229] S532. Determine whether the merged long-term experience buffer exceeds the preset capacity;

[0230] S533. When the merged long-term experience buffer does not exceed the preset capacity, all experience samples are retained.

[0231] S534. When the merged long-term experience buffer exceeds the preset capacity, sample forgetting processing is performed.

[0232] S535. Clear the short-term experience buffer and enter the next online learning cycle;

[0233] To further explain, the sample forgetting process in step S534 is an inverse priority probabilistic forgetting process, specifically including:

[0234] S541. Calculate the discard probability of the empirical samples based on their sample priority, wherein the discard probability is negatively correlated with the sample priority; specifically, for the candidate empirical sample set participating in the forgetting process... empirical samples Its probability of being discarded is:

[0235]

[0236] in, Representing empirical samples The probability of discarding, Representing empirical samples Sample priority, Indicates the forgetting intensity parameter;

[0237] S542. Select samples to be discarded from the empirical sample set that exceeds the capacity limit based on the discard probability;

[0238] S543. Delete the samples to be discarded to obtain the updated long-term experience buffer;

[0239] To further explain, step S6 specifically involves:

[0240] Perform model update verification and asynchronous deployment, verify the online updated model, and asynchronously update the qualified model to the front-end execution process of the vehicle intelligent agent based on the verification results;

[0241] S61. When the preset update conditions are met, the model update verification process is triggered.

[0242] S62. Calculate the validation results of the online updated model based on the empirical samples in the empirical buffer;

[0243] S63. Compare the verification result with the reference result corresponding to the current front-end execution model;

[0244] S64. When the online update model meets the preset deployment conditions, the online update model will be determined as the new front-end execution model.

[0245] S65. When the online update model does not meet the preset deployment conditions, it is prohibited to deploy it to the front-end execution process of the vehicle intelligent agent.

[0246] S66. Update at least one of the following based on the verification results: model version record, empirical sample status, and sample priority;

[0247] To further explain, such as Figure 4 As shown, the model update verification process in step S61 adopts a champion-challenger safety gating mechanism, specifically including:

[0248] S611. Calculate the challenger loss for all or part of the experience samples in the short-term and long-term experience buffers using the updated challenger model.

[0249] S612, Read the champion reference loss stored in the corresponding experience sample;

[0250] S613. Summarize the challenger's loss and the champion's reference loss to obtain the challenger's global loss and the champion's global loss.

[0251] S614. Compare the global loss of the challenger and the global loss of the champion, and determine whether the challenger model meets the preset promotion conditions based on the comparison results.

[0252] S615. When the challenger model meets the promotion conditions, the challenger model will be promoted to the new champion model, and a model update notification will be sent to the front-end inference process.

[0253] S616. When the challenger model does not meet the promotion conditions, roll back the challenger model and keep the current champion model as the front-end execution model.

[0254] To further explain, the specific process of challenger model promotion and rollback in steps S615 and S616 is as follows:

[0255] S621. When the challenger model meets the promotion conditions, copy, map, register or save the challenger model's model parameters as the new champion model parameters.

[0256] S622. Recalculate the loss of all or part of the experience samples in the experience buffer using the new champion model;

[0257] S623. Update the sample priority and champion reference loss of the corresponding empirical sample based on the recalculated loss;

[0258] S624. When the challenger model does not meet the promotion conditions, roll back at least one of the challenger model's model parameters, optimizer state, and learning rate scheduler state to the state corresponding to the current champion model.

[0259] S625. After the challenger model is rolled back, the update results of the current challenger model are prohibited from entering the front-end execution process of the vehicle intelligent agent.

[0260] To further explain, the specific process of asynchronous deployment in step S6 is as follows:

[0261] S631. The front-end inference process listens for model update notifications from the back-end learning system or unified signal system.

[0262] S632. When the front-end inference process receives a model update notification, it reads the model version number, model weight path, parameter pointer, or shared memory identifier of the new front-end execution model.

[0263] S633, The front-end inference process loads new model weights or switches to new model parameters between adjacent inference cycles;

[0264] S634. During model switching, sensor data acquisition, data processing, underlying vehicle control, and simulation world advancement remain uninterrupted.

[0265] S635. After the model switch is completed, the front-end inference process continues to generate real-time driving decisions using the new front-end execution model.

[0266] The detailed descriptions listed above are merely specific descriptions of feasible embodiments of the present invention, and are not intended to limit the scope of protection of the present invention. All equivalent methods or modifications that do not depart from the technology of the present invention should be included within the scope of protection of the present invention.

Claims

1. An online self-evolutionary learning method for autonomous driving based on an asynchronous architecture, characterized in that, include: S1. Establish an asynchronous closed-loop online interactive environment for autonomous driving, initialize the simulation world, vehicle intelligent agent, background learning system, automatic supervision system and communication interface, and configure each system unit to operate collaboratively in an asynchronous manner; S2. Run the asynchronous front-end execution process of the vehicle intelligent agent, enabling the vehicle intelligent agent to generate real-time driving decisions based on sensor data, vehicle status and navigation information, and execute closed-loop vehicle control. S3. Collect online learning data during the closed-loop interaction process, continuously record interaction data during the interaction between the vehicle intelligent agent and the simulation world, and asynchronously transmit the interaction data to the background learning system; S4. Generate automatic supervision signals required for online learning. The background learning system sends a supervision request to the automatic supervision system, which generates supervision signals based on the interaction data and associates the supervision signals with the corresponding interaction data to form online training samples. S5. Perform background online learning and experience management. The background learning system performs model updates based on the online training samples and historical experience samples, and manages the experience samples. S6. Perform model update verification and asynchronous deployment. Verify the online updated model and asynchronously update the qualified model to the front-end execution process of the vehicle intelligent agent based on the verification results.

2. The online self-evolutionary learning method for autonomous driving based on asynchronous architecture according to claim 1, characterized in that, Step S1 specifically involves: S11. Initialize the simulation world, test map, test route, traffic signals, road topology, background traffic participants, vehicle agents, and operating parameters; S12. Configure sensor interfaces, vehicle status interfaces, navigation interfaces, control interfaces, front-end inference interfaces, interaction recording interfaces, and model update interfaces for the vehicle intelligent agent; S13. Establish a hierarchical architecture system consisting of a system layer, an intelligent agent layer, and a learning layer. The system layer is used to perform system initialization, lifecycle management, and global signal coordination. The intelligent agent layer is used to perform the front-end closed-loop operation of the vehicle intelligent agent. The learning layer is used to perform background learning, automatic supervision, experience management, and model update control. S14. Establish an inter-process communication mechanism to transmit sensor data, vehicle status, driving decisions, interactive data, supervision signals, training samples, and model update information between the simulation world, vehicle intelligent agent, background learning system, and automatic supervision system. S15. Set the simulation world to advance with continuous time semantics, so that the time advancement of the simulation world is completed independently of the front-end execution process or the back-end learning process of the vehicle intelligent agent. S16. Initialize the unified signal system, which is used to broadcast global control signals during scene initialization, scene reset, simulation start, abnormal interruption, learning consolidation, model update and system termination. S17. Initialize the data structures required for online self-evolutionary learning, wherein the data structures include at least one of the following: learning queue, experience buffer, model version record, sample priority record, and interaction log.

3. The online self-evolutionary learning method for autonomous driving based on asynchronous architecture according to claim 2, characterized in that, Step S4 specifically involves: S41. The background learning system reads the interaction data from the learning queue and sends a supervision request to the automatic supervision system. S42. The automatic monitoring system generates a monitoring signal based on the monitoring request, the monitoring signal including a target velocity monitoring signal and a future trajectory monitoring signal; S43. The background learning system pairs the supervision signal with the corresponding interaction data to construct online training samples; S44. The online training samples include at least one of the following: model input, supervision label, sample timestamp, interaction record identifier, and sample priority field; The method for generating the monitoring signal in step S42 includes the following: S411. Local privileged expert supervision method based on simulator privileged information, which generates supervision signals by using map information, traffic participant status, traffic signal status, drivable area information, collision risk information or expert trajectory information provided by the simulator. S412. A multimodal large language model integrated supervision method based on observable inputs, which generates supervision signals by utilizing observable sensor data, vehicle status, navigation commands and scene context of vehicle intelligent agents; S413. A hybrid supervision method based on local privileged expert supervision and multimodal large language model integrated supervision, which selects or merges supervision signals from different sources according to preset conditions.

4. The online self-evolutionary learning method for autonomous driving based on asynchronous architecture according to claim 3, characterized in that, The specific process of the multimodal large language model integration supervision method in S412 is as follows: S421. Construct a two-layer parallel inference architecture, which includes a multi-agent parallel layer and an integrated expert parallel layer. S422. In the multi-agent parallel layer, multiple independent multimodal large language model agents are created to process supervised generation tasks corresponding to different interactive data in parallel. S423. In the parallel layer of integrated experts, each multimodal large language model agent calls multiple multimodal large language model experts to generate multiple candidate supervision results for the same interactive data. S424. Perform structured parsing, anomaly filtering, and validity verification on the multiple candidate supervision results; S425. Perform consistency aggregation on the candidate supervision results that have passed the validity check to obtain the final supervision signal; S426. The final supervision signal is returned to the background learning system for constructing online training samples.

5. The online self-evolutionary learning method for autonomous driving based on asynchronous architecture according to claim 4, characterized in that, The specific process of consistent aggregation in step S425 is as follows: S431. Extract candidate target velocities from multiple candidate supervision results, perform robust statistics on candidate target velocities, perform outlier detection based on median absolute deviation, screen outlier candidate velocities, and obtain the mainstream candidate set of target velocities. S432. Select the original candidate target velocity that is close to the consensus center from the mainstream candidate set of target velocities as the final target velocity supervision signal; S433. Extract candidate future trajectories from multiple candidate supervision results, and map each candidate future trajectory to a trajectory feature used to characterize the geometric trend of the trajectory. S434. Based on the trajectory features, perform robust statistics on the candidate future trajectories, perform outlier detection based on median absolute deviation, filter out outlier candidate trajectories, and obtain the mainstream candidate set of future trajectories. S435. Perform point-by-point coordinate averaging, weighted averaging, or robust averaging on the candidate future trajectories in the mainstream candidate set of future trajectories to obtain the final future trajectory supervision signal. S436. Combine the final target velocity monitoring signal and the final future trajectory monitoring signal into the final monitoring signal corresponding to the current interactive data.

6. The online self-evolutionary learning method for autonomous driving based on asynchronous architecture according to claim 5, characterized in that, The multimodal large language model integration supervision method in step S412 also includes an automatic supervision signal consistency verification process, specifically: S441. Obtain the multimodal large language model integration supervision signal and the reference supervision signal. The multimodal large language model integration supervision signal is the final supervision signal obtained by the consensus aggregation. The reference supervision signal includes at least one of the following: simulator privilege supervision signal, rule security module output signal, supervision signal generated by manual takeover, historical high confidence supervision signal, and multi-expert consensus supervision signal. S442. Based on the integration of future trajectories in the supervision signal and future trajectories in the reference supervision signal using a multimodal large language model, calculate the trajectory direction consistency index. The trajectory direction consistency index is determined based on at least one of the following: the angle between future trajectory direction vectors, cosine similarity, turning trend consistency, or trajectory tangential direction consistency. S443. Based on the target velocity in the integrated supervision signal and the target velocity in the reference supervision signal of the multimodal large language model, calculate the velocity trend consistency index. The velocity trend consistency index is determined according to at least one of the following: acceleration trend consistency, deceleration trend consistency, velocity change direction consistency, or velocity range consistency among target velocities. S444. Determine the semantic reliability of the multimodal large language model integration supervision signal based on the trajectory direction consistency index and the velocity trend consistency index; S445. Based on the semantic reliability, perform retention, weight reduction, correction, or discard processing on the integrated supervision signal of the multimodal large language model.

7. The online self-evolutionary learning method for autonomous driving based on asynchronous architecture according to claim 6, characterized in that, Step S5 specifically involves: S51. Establish a short-term experience buffer and a long-term experience buffer, wherein the short-term experience buffer is used to store newly arrived online training samples, and the long-term experience buffer is used to store historical experience samples. S52. Calculate the sample priority for the online training samples and write the newly arrived online training samples and their sample priorities into the short-term experience buffer. S53. The background learning system performs hybrid replay sampling based on short-term and long-term experience buffers to obtain experience samples for training. S54. In the background learning system, the online update model is used as the challenger model, and the challenger model is trained and updated based on the sampled experience samples and their supervision signals. S55. During the challenger model training process, the front-end execution flow of the vehicle agent is not blocked, and the current front-end execution model, which is the champion model, continues to generate real-time closed-loop driving decisions. The process of calculating sample priority in S52 includes: S511. Perform forward computation on the online training samples using the current champion model or a frozen copy of the current champion model; S512. Calculate the reference loss corresponding to the current champion model based on the supervision signal of the online training samples; S513. Use the reference loss or the value obtained based on the reference loss as the initial sample priority for online training samples; S514. The online training samples, initial sample priority, and champion reference loss are associated and stored in a short-term empirical buffer. The specific process of mixed playback sampling in S53 is as follows: S521. Calculate the sum of sample priorities in the short-term experience buffer and the long-term experience buffer, respectively. S522. Determine the buffer selection probability based on the sample priority of the short-term experience buffer and the long-term experience buffer; S523. Select a target buffer from a short-term experience buffer or a long-term experience buffer according to the buffer selection probability. S524. Select empirical samples within the target buffer based on sample priority; S525. Calculate the importance sampling weights based on the sampling probability of empirical samples; S526. Use the importance sampling weights to correct the training bias introduced by non-uniform sampling.

8. The online self-evolutionary learning method for autonomous driving based on asynchronous architecture according to claim 7, characterized in that, The management of experience samples in S5 includes experience buffer consolidation, specifically as follows: S531. After completing the model update verification, merge the experience samples in the short-term experience buffer into the long-term experience buffer. S532. Determine whether the merged long-term experience buffer exceeds the preset capacity; S533. When the merged long-term experience buffer does not exceed the preset capacity, all experience samples are retained. S534. When the merged long-term experience buffer exceeds the preset capacity, sample forgetting processing is performed. S535. Clear the short-term experience buffer and enter the next online learning cycle; The process of performing template forgetting handling in S534 includes: S541. Calculate the discard probability based on the sample priority of the empirical sample, wherein the discard probability is negatively correlated with the sample priority; S542. Select samples to be discarded from the empirical sample set that exceeds the capacity limit based on the discard probability; S543. Delete the samples to be discarded to obtain the updated long-term experience buffer.

9. The online self-evolutionary learning method for autonomous driving based on asynchronous architecture according to claim 8, characterized in that, Step S6 specifically involves: S61. When the preset update conditions are met, the model update verification process is triggered. S62. Calculate the validation results of the online updated model based on the empirical samples in the empirical buffer; S63. Compare the verification result with the reference result corresponding to the current front-end execution model; S64. When the online update model meets the preset deployment conditions, the online update model will be determined as the new front-end execution model. S65. When the online update model does not meet the preset deployment conditions, it is prohibited to deploy it to the front-end execution process of the vehicle intelligent agent. S66. Update at least one of the following based on the verification results: model version record, empirical sample status, and sample priority; The model update and validation process employs a champion-challenger safety gating mechanism, specifically including: S611. Calculate the challenger loss for all or part of the experience samples in the short-term and long-term experience buffers using the updated challenger model. S612, Read the champion reference loss stored in the corresponding experience sample; S613. Summarize the challenger's loss and the champion's reference loss to obtain the challenger's global loss and the champion's global loss. S614. Compare the global loss of the challenger and the global loss of the champion, and determine whether the challenger model meets the preset promotion conditions based on the comparison results. S615. When the challenger model meets the promotion conditions, the challenger model will be promoted to the new champion model, and a model update notification will be sent to the front-end inference process. S616. When the challenger model does not meet the promotion conditions, roll back the challenger model and keep the current champion model as the front-end execution model. The specific process for the challenger model to be promoted or rolled back is as follows: S621. When the challenger model meets the promotion conditions, copy, map, register or save the challenger model's model parameters as the new champion model parameters. S622. Recalculate the loss of all or part of the experience samples in the experience buffer using the new champion model; S623. Update the sample priority and champion reference loss of the corresponding empirical sample based on the recalculated loss; S624. When the challenger model does not meet the promotion conditions, roll back at least one of the challenger model's model parameters, optimizer state, and learning rate scheduler state to the state corresponding to the current champion model. S625. After the challenger model is rolled back, the update results of the current challenger model are prohibited from entering the front-end execution process of the vehicle intelligent agent.

10. A system for implementing the online self-evolutionary learning method for autonomous driving based on an asynchronous architecture as described in any one of claims 1-9, characterized in that, include: The autonomous driving closed-loop simulation asynchronous architecture module, background learning module, automatic supervision module, and communication module; The asynchronous closed-loop online interaction module for autonomous driving is used to implement steps S1-S3 and S6. The automatic monitoring module is used to generate monitoring signals in step S4; The background learning system is used to associate the supervision signal with the corresponding interactive data to form online training samples in step S4. And in S5, model updates are performed based on the online training samples and historical experience samples, and experience samples are managed; The communication module is used to enable communication between modules and between different parts within a module, and to transfer data.

Images