Method for generating corner case based on global big language model proxy

Abstract

The invention discloses a method for generating a corner case based on a global large language model agent, and the method comprises the steps: obtaining road scene data, selecting a plurality of key agents in a current vehicle distance range, extracting a plurality of most critical key frames in the whole track of each key agent, and enabling the key frames to have the maximum kinematics entropy; encoding the trajectory of the key agent into a structured tuple; calculating a time-space relationship between the current vehicle and the key agent according to the structured tuple, and identifying an extreme situation scene and the key agent executing a corresponding behavior in the extreme situation scene; determining the type of the extreme condition and agent coordination required for executing the key agent of the corresponding behavior; generating agent cue words based on the extreme condition type and the required agent coordination; and based on the agent prompt words, modifying behaviors of corresponding key agents in the key frames through a large language model, performing track synthesis, and generating corner cases in combination with road scene data. The corner case is generated by analyzing the aggregated scene data, so that the expandability of the corner case is realized.

Description

Technical Field

[0001] The present application relates to the field of testing of intelligent driving systems for new energy vehicles, and in particular to a method for generating corner cases based on a global large language model agent. Background Art

[0002] With the rapid development of the new energy vehicle industry, intelligent driving technology, as a core technology for improving the safety and efficiency of new energy vehicles, has become a current research focus. However, ensuring the safety and reliability of intelligent driving systems in complex and changing real-world traffic environments remains a key scientific issue that needs to be addressed. Due to their large size and high passenger capacity, new energy vehicles face higher safety requirements in actual operation. The limitations of traditional testing methods are particularly prominent when faced with extreme, rare, and challenging scenarios such as "corner cases."

[0003] Corner cases are essentially driving scenarios that, while extremely unlikely to occur, could potentially lead to serious consequences. For new energy vehicles, these scenarios often include complex urban intersections, congested traffic, unexpected obstacles, or emergency handling in severe weather conditions. These scenarios pose greater challenges to vehicle planning algorithms. Effectively verifying the safety and reliability of new energy vehicles in extreme situations has become an unavoidable core issue in both technological research and development and practical applications. However, existing testing methods struggle to cover all potential complex scenarios due to high costs, time consumption, and the difficulty of large-scale data collection. They also suffer from significant deficiencies in scalability and customization. Summary of the Invention

[0004] The purpose of this application is to provide a method for generating corner cases based on a global large language model agent, using a real-world road scene dataset to transform challenging extreme scenarios into a scalable test benchmark, significantly improving the robustness and adaptability of planning algorithms in complex scenarios.

[0005] In a first aspect of the present application, a method for generating corner cases based on a global large language model agent is provided, comprising:

[0006] (a) obtaining road scene data, selecting several key agents within the current vehicle distance range, and extracting several most critical key frames in the entire trajectory of each key agent, wherein each of the several key frames has the largest kinematic entropy;

[0007] (b) encoding the trajectories of the several key agents into structured tuples;

[0008] (c) calculating the spatiotemporal relationship between the current vehicle and the plurality of key agents based on the structured tuple, and identifying extreme case scenarios and key agents that execute corresponding behaviors in the extreme case scenarios;

[0009] (d) determining an extreme case type according to the identified extreme case scenario, and determining the agent coordination required for key agents to perform corresponding behaviors in the extreme case scenario;

[0010] (e) generating agent prompt words for the global large language model agent based on the corner case type and the required agent coordination; and

[0011] (f) Based on the agent prompt words, the behavior of the corresponding key agent in the key frame is modified through a large language model and trajectory synthesis is performed, and corner cases are generated in combination with the road scene data.

[0012] In a preferred example, in step (a), the distance range is a local spatial range with the current vehicle as the center and a radius of R meters.

[0013] In a preferred embodiment, the number of the selected key agents is 8 to 10.

[0014] In a preferred embodiment, the number of the selected key frames is 8 to 15.

[0015] In another preferred embodiment, R is 10 meters to 20 meters.

[0016] In a preferred embodiment, the step (f) further comprises:

[0017] (f1) Smoothing the synthesized trajectory using cubic spline interpolation.

[0018] In a preferred example, the kinematic entropy is velocity variance, acceleration variance, or the sum of velocity variance and acceleration variance.

[0019] In a preferred embodiment, the step (a) further comprises: sorting the key frames in the entire trajectory of each key agent according to kinematic entropy, and extracting the first n key frames with the largest kinematic entropy in each key agent.

[0020] In another preferred embodiment, the step (a) further comprises: sorting the key frames in the entire trajectory of each key agent according to speed variance, and extracting the first n key frames with the largest speed variance in each key agent.

[0021] In another preferred embodiment, the step (a) further comprises: sorting the key frames in the entire trajectory of each key agent according to the acceleration variance, and extracting the first n key frames with the largest acceleration variance in each key agent.

[0022] In another preferred embodiment, the step (a) further comprises: sorting the key frames of each key agent according to the sum of velocity variance and acceleration variance, and extracting the first n key frames with the largest acceleration variance in each key agent.

[0023] In a preferred embodiment, the road scene data includes a NuPlan dataset, and step (a) further includes:

[0024] (a1) Segmenting the road scene data.

[0025] In a preferred embodiment, the segmentation includes randomly selecting a number of scenes from each scene type in the NuPlan dataset;

[0026] In a preferred embodiment, the segmentation includes extracting several worst performing scenes from each scene type in the NuPlan dataset.

[0027] In a preferred example, the segmentation includes sorting by NR-CLS scores, and selecting k scenes with the lowest NR-CLS scores as the worst performing scenes.

[0028] In a preferred example, the extreme situation types include lane intrusion or sudden braking.

[0029] In a second aspect of the present application, an interactive simulation testing method for an intelligent driving system is provided. The testing method includes using the corner cases generated by the aforementioned method as simulation test cases for the intelligent driving system.

[0030] In another preferred embodiment, corner cases generated based on individual large language model agents are used as simulation test cases for the intelligent driving system.

[0031] In a third aspect of the present application, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer-executable instructions, and the computer-executable instructions implement the steps in the aforementioned method when executed by a processor.

[0032] It should be understood that within the scope of the present invention, the above-mentioned technical features of the present invention and the technical features described in detail below (such as in the embodiments) can be combined with each other to form new or preferred technical solutions. Due to space limitations, they will not be listed here one by one. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] To more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. It should be understood that the drawings described below are merely some implementation examples of the present invention, and those skilled in the art can also derive other implementation examples based on these drawings without inventive effort.

[0034] Figure 1 This is a flowchart of a method for generating corner cases based on a global large language model agent in one embodiment of the present application.

[0035] Figure 2 Schematic diagram of an innovative simulation test method SCP according to one embodiment of the present application.

[0036] Figure 3 These are three routes designed in an urban driving scenario according to one embodiment of the present application, as well as the performance of the driver's role.

[0037] Figure 4 These are different road scenes established according to an embodiment of the present application. DETAILED DESCRIPTION

[0038] In the following description, many technical details are provided to help readers better understand this application. However, those skilled in the art will understand that even without these technical details and various changes and modifications based on the following embodiments, the technical solutions claimed in this application can be implemented.

[0039] Through extensive and in-depth research, the inventors have proposed a top-down approach for constructing scalable corner cases. Based on an existing dataset, keyframes of key agents with the largest kinematic entropy are extracted and encoded into structured tuples. Based on the spatiotemporal relationships between the structured tuples and key agents, extreme cases and their associated key agents are identified. Trajectory synthesis is then performed based on a large language model to generate corner cases that extend the original dataset. Intelligent driving systems tested through this scalable corner case simulation approach can improve their responsiveness in extreme scenarios or complex traffic environments.

[0040] Specifically, this application proposes an innovative simulation testing method based on a large language model: the Scenario Creation Planner (SCP). This method leverages the powerful generation capabilities of the large language model to generate scalable and customizable corner cases through natural language instructions, greatly improving the efficiency and flexibility of corner case generation.

[0041] Based on the SCP method, this application proposes two important technical achievements: a corner case benchmark tool (SCP-NuPlan) and a corner case simulator (SCP-LimSim). First, SCP-NuPlan transforms the classic intelligent driving planning task dataset NuPlan into a set of diverse and challenging corner case benchmarks, providing a new standard and testing tool for the evaluation of intelligent driving systems of new energy vehicles. Experimental results show that when tested on the SCP-NuPlan dataset, the performance of the intelligent driving planning algorithm decreases significantly, revealing the shortcomings of the algorithm in complex and extreme scenarios. Secondly, SCP-LimSim generates customized corner cases through natural language descriptions, becoming a flexible simulation tool that can perform highly customized tests in different scenarios, further enhancing the emergency response capabilities of new energy vehicles in complex traffic environments.

[0042] This application has at least the following beneficial effects and advantages:

[0043] (a) SCP not only effectively reduces reliance on large amounts of real-world data but also flexibly adjusts the complexity and diversity of generated scenarios based on testing requirements, overcoming the limitations of traditional simulation testing methods in terms of contextual relevance, customization, and scalability. The introduction of the SCP method marks a new stage in intelligent driving simulation testing, significantly improving the comprehensiveness and accuracy of testing in extreme scenarios, particularly for new energy vehicles.

[0044] (b) This application successfully overcomes the bottleneck of traditional simulation testing in generating corner cases through the innovative introduction of a large language model, proposing a low-cost, efficient, and customizable testing solution. The development of SCP-NuPlan and SCP-LimSim provides new benchmarks and tools for validating intelligent driving systems for new energy vehicles, driving the development of the industry. Through these innovative achievements, new energy vehicles will see further improvements in safety, reliability, and intelligence, providing more solid support for the popularization of green transportation and the promotion and application of intelligent driving technology.

[0045] (c) This invention provides a new solution for simulation testing of intelligent driving systems for new energy vehicles, effectively improving the coverage and accuracy of the test. In particular, it can generate highly customized extreme scenarios in complex planning tasks, thereby better verifying the system's ability to cope with special conditions.

[0046] As used herein, the terms "agent," "vehicle," and "car" may have the same meaning and be used interchangeably, referring to a vehicle traveling in a road scenario. The terms "extreme scenario," "extreme situation," and "complex situation" may have the same meaning and be used interchangeably. The terms "extreme case," "corner case," and "edge case" may have the same meaning and be used interchangeably.

[0047] In order to make the objectives, technical solutions and advantages of this application clearer, the implementation methods of this application will be further described in detail below with reference to the accompanying drawings.

[0048] The first aspect of the present application provides a method for generating corner cases based on an individual large language model agent, the process of which is as follows: Figure 1 As shown, the method includes the following steps:

[0049] (a) Obtain road scene data, select several key agents within the current vehicle distance range, and extract the most critical key frames in the entire trajectory of each key agent, where the key frames each have the largest kinematic entropy.

[0050] (b) Encoding the trajectories of several key agents into structured tuples.

[0051] (c) Based on the structured tuples, the spatiotemporal relationship between the current vehicle and several m key agents is calculated to identify extreme case scenarios and the key agents that perform corresponding behaviors in the extreme case scenarios.

[0052] (d) Determine the extreme case type based on the identified extreme case scenarios and determine the agent coordination required for the key agents to perform the corresponding behaviors in the extreme case scenarios.

[0053] (e) Generate agent prompt words for the global large language model agent based on the corner case type and the required agent coordination.

[0054] (f) Based on the agent prompt words, the behavior of the corresponding key agent in the key frame is modified through the large language model and trajectory synthesis is performed, and corner cases are generated by combining road scene data.

[0055] In order to better understand the technical solution of the present application, the following is an illustration with reference to specific examples. The details listed in the examples are mainly for ease of understanding and are not intended to limit the scope of protection of the present application.

[0056] The core of this application is a proposed LLM-driven corner case generation method (SCP) for intelligent driving, aiming to enhance the ability of intelligent driving systems to cope with extreme scenarios through simulation testing. The SCP framework aims to generate corner cases specific to planning tasks through two complementary LLM interaction paradigms: a global agent and an individual agent approach. First, the global agent adopts a top-down approach for scalability. Leveraging a large language model, the global agent generates corner cases by analyzing aggregated scenario data (coordinates, timestamps, agent states, etc.). This approach focuses on system-level coordination, selecting the optimal agent and time window for trajectory modification while maintaining scenario consistency. Individual agents, on the other hand, adopt a bottom-up approach for customization. Specifically, individual agents implement real-time behavior control via natural language instructions to generate corner cases. Each agent's decision-making process is integrated into the intent-driven context through explicit natural language instructions, allowing for customized driving behavior. This dual-paradigm architecture enables SCP to achieve scalable customization: the global agent ensures scenario-wide feasibility, while the individual agents allow for localized behavior specification. Furthermore, the framework is inherently data-agnostic, requiring neither scenario-specific training nor predefined behavior templates. Natural language serves as both a control interface and a consistency constraint, explicitly incorporating human domain knowledge through prompt engineering.

[0057] Based on the SCP framework, this application has achieved two important interim results: SCP-NuPlan and SCP-LimSim. Figure 2 As shown in Figure 2, SCP-NuPlan implements a global agent approach for large-scale automatic corner case benchmark generation. SCP-LimSim implements an individual agent approach for generating corner cases for interactive scene creation. These two applications share the core technical component of language-mediated trajectory modification but differ in the scope of LLM intervention and optimization objectives.

[0058] Global Proxy

[0059] like Figure 2 As shown in the previous section, we construct SCP-NuPlan on an existing dataset using a global LLM agent, modifying vehicle trajectories using global information to achieve scalability. Our benchmark generation system implements the global agent paradigm through a three-stage process aligned with the SCP framework.

[0060] Context Encoding

[0061] Global information is constructed by extracting spatiotemporal context from the original NuPlan scene. First, a group of m key agents (e.g., 8 to 10, preferably 10) that are close to the ego vehicle (e.g., within a radius of 10 to 20 meters) are selected, because extreme cases usually arise from sudden or unreasonable changes in the trajectories of surrounding vehicles. Next, the most critical top n key frames (e.g., 8 to 15, preferably 10) in the entire trajectory of each agent are extracted. These frames have the largest kinematic entropy because they can effectively capture significant changes in the trajectory between the ego vehicle and the agent. Kinematic entropy is the velocity variance, acceleration variance, or the sum of velocity variance and acceleration variance. When extracting key frames, they are sorted according to the kinematic entropy, and the top n key frames with the largest kinematic entropy are selected. It should be understood that a different number of key frames can be extracted for each key agent. Finally, the trajectory of the key agent is encoded as a structured tuple<agent_id,type,(x,y,θ)t> , t∈T, ensuring a compact and comprehensive temporal representation of the agent's motion process. agent_id represents the agent identifier, type represents the agent type in the traffic context, such as car, pedestrian, or bicycle, θ represents the current vehicle's head angle, t represents the current frame, and T represents the trajectory time window set, which is the time from the start to the end of the scene, usually around 8-15 seconds.

[0062] LLM Center for Reasoning

[0063] Specific requirements are imposed on the LLM to generate extreme case scenarios. First, a relational analysis is performed to identify the most appropriate extreme case scenarios and the agents best suited to execute them by computing the spatiotemporal relationships between the ego vehicle and surrounding agents. Next, intervention planning is performed to determine the optimal extreme case types—such as lane incursions or sudden braking—and to specify the agent coordination required to introduce these extreme cases. "Extreme cases" are rare and complex scenarios that the autonomous driving system must reliably handle to ensure safety under real-world conditions. Directly collecting extreme case data is often costly, time-consuming, and insufficiently scalable. Some extreme case types include: lane cutting, sudden braking, S-shaped maneuvers, emergency vehicles, sudden breakdowns, and aircraft forced landings. Other extreme case types include non-priority forced merges, running the light at intersections, and sudden crossings under obstructed vision. Agent coordination refers to how the actions of the ego vehicle and surrounding agents are coordinated when introducing extreme cases. Agent coordination can include predicting interactive behavioral patterns between agents, spatial yielding strategies, and coordinated speed control. For example, introducing extreme cases involves coordinating the trajectories of surrounding vehicles to create situations such as lane incursions or sudden braking. Maintaining scenario consistency: When modifying agent trajectories to generate extreme situations, ensure that the entire scenario still conforms to the laws of physics and real-world logic. This may include ensuring that agents do not collide with each other unreasonably, or that their behavior remains within acceptable limits (even for aggressive driving). System-level coordination: Aggregated scenario data (such as coordinates, timestamps, agent states, etc.) is analyzed through LLM to coordinate agent trajectories at the system level to generate extreme situations. Finally, trajectory synthesis is performed to generate modified agent trajectories under predefined constraints.

[0064] Physical Constraint Enforcement

[0065] Cubic spline interpolation is used to smooth the trajectory output by the LLM, ensuring seamless transitions between keyframes. This method effectively minimizes sudden changes in the trajectory and maintains the continuity of motion, thereby enhancing the realism of the scene.

[0066] The extreme case benchmark SCP-NuPlan implemented in this paper covers 6,382 scenarios across NuPlan's 14 challenge types, involving 1,403 logs in the NuPlan test set and mini split, while ensuring the preservation of extreme cases through a non-reactive evaluation protocol.

[0067] Individual Agent

[0068] like Figure 2 As shown in the following section, SCP-LimSim is built on the existing simulator through individual LLM agents, and customization is achieved through natural language-based individual control.

[0069] SCP-Limsim: Individual Control for Customization

[0070] The SCP-LimSim framework focuses on providing flexible and highly customizable corner case generation capabilities by controlling individual agents. Unlike SCP-NuPlan, which focuses on global information integration, SCP-LimSim emphasizes generating edge cases from an individual perspective and allows users to customize the agent's driving behavior, such as vehicle type and driver characteristics. SCP-LimSim achieves fine-grained control over agent behavior through a large language model.

[0071] Context Encoding

[0072] Individual information is constructed by integrating key contextual elements. First, as shown in Table 1, situational context is introduced by encoding relevant environmental information into natural language. Next, behavioral context is provided in two ways: defining driver roles and providing navigation strategies. In SCP-Limsim, the present invention implements eight different driver roles (as shown in Table 2), each of which provides specific behavioral instructions through customized prompts. Finally, the action space is constrained by predefining five available actions (as shown in Table 3), thereby ensuring the realism and controllability of agent behavior.

[0073] Table 1 Encoding of environmental information (including vehicle and road status) into natural language in SCP-LimSim.

[0074]

[0075] Table 2 Predefined driver roles in SCP-LimSim

[0076]

[0077]

[0078] Table 3 Predefined available actions in SCP-LimSim.

[0079] action describe To the left Switch lanes to the left of the current lane Turn right Switch lanes to the right of the current lane idle Stay in your lane and maintain your current speed accelerate Accelerate in your current lane slow down Slow down in your current lane

[0080] LLM Center for Reasoning

[0081] Parallel decision-making is achieved using distributed LLM agents. First, intent generation is performed to map the character configuration to the corresponding driving strategy. Next, parameter estimation is performed. Parameter estimation refers to the LLM-centric inference phase of SCP-LimSim. Due to the difficulty of large language models (LLMs) in accurately processing numerical values, the LLM is allowed to return the range of acceleration and steering angles:

[0082] Acceleration:

[0083] Steering angle:

[0084] Finally, by generating natural language explanatory justifications for each action, we ensure an explainable decision-making process, providing transparency and making the agent's behavior and policy execution clearer.

[0085] Physical Constraint Enforcement

[0086] Physical constraints are enforced by converting language specifications into executable trajectories. The acceleration range generated by the LLM is mapped to the Intelligent Driving Model (IDM) parameters and the IDM model is integrated. The vehicle's acceleration It is controlled by the following formula:

[0087]

[0088] in, represents the vehicle's acceleration, v is the current speed, v0 is the desired speed for the lane, Δv is the speed difference between the current vehicle and the agent, s is the actual distance between the current vehicle and the preceding vehicle, s0 is the minimum safe following distance, δ is the acceleration factor, a is the vehicle's maximum acceleration capability under ideal conditions (i.e., without interference from the preceding vehicle), and b is the maximum comfortable deceleration in an emergency. The maximum value of the acceleration range is defined as a, and the minimum value is represented by b. LLM determines the acceleration range by calculating the vehicle's acceleration.

[0089] Next, the final steering angle is determined by first checking whether the steering direction is valid within the defined action space and then calculating the average value over a given angle range. Finally, the Frenet framework trajectory optimization from LimSim is applied to generate an optimal path that complies with real-world driving constraints and maintains smooth motion.

[0090] Experimental verification

[0091] Table 4 shows the reproduction results on the NuPlan mini segmentation. Nu represents NuPlan, and NuMini represents the NuPlan mini segmentation. The Non-Reactive Closed-Loop Score (NR-CLS) is used as the evaluation metric. NR-CLS measures the vehicle's ability to navigate smoothly in corner cases without relying on the perception module. A higher NR-CLS indicates better performance. This result illustrates that smaller segmentations lead to greater variance.

[0092] Table 4. Reproduction results on NuPlan mini segmentation.

[0093]

[0094] Through experiments, the effectiveness and customization capabilities of the SCP framework in different algorithms and driving models were verified.

[0095] SCP-NuPlan Experiment

[0096] We first tested multiple planning models on the SCP-NuPlan benchmark to demonstrate the effectiveness of SCP. The experiments used planTF, an open-source model that performed well in the 2023 NuPlan Challenge. For comparison, we selected the classic UrbanDriver and the CNN-based RasterModel and compared them using the same testing protocol as planTF. The testing protocol includes two scenario selection methods: Test14-random and Test14-hard. Test14-random randomly selects 20 scenarios of each scenario type from NuPlan, while Test14-hard selects the 20 worst-performing scenarios for evaluation. We primarily focus on NR-CLS, which is used to assess the difficulty of edge case generation. A higher NR-CLS score achieved by a planner indicates better model performance, while a lower NR-CLS score indicates worse performance in that scenario. Test14-hard sorts the planners by NR-CLS score, selecting the k (e.g., 20) scenarios with the lowest NR-CLS scores for evaluation.

[0097] Inspired by NuPlan's provision of a specialized small split for simple testing or development, we conducted pilot experiments with SCP-NuPlan-Mini and SCP-NuPlan-Air. SCP-NuPlan-Mini contains only 186 Test14-random scenes and 7 Test14-hard scenes. Concerns about underrepresentational performance due to the small split led to this experiment. The experimental results (Table 4) show that planTF's performance on the small split is consistent with reported results, while UrbanDriver and RasterModel demonstrate superior performance, with RasterModel even surpassing planTF's replication performance. These results confirm that small data volumes can lead to underrepresentational data. To balance the convenience of testing on small datasets with data representativeness, we created a medium-sized benchmark, SCP-NuPlan-Air, which includes SCP-NuPlan-Mini and 336 additional logs from the test split, for a total of 400 logs and 2,000 modified scenes, including 249 Test14-random scenes and 89 Test14-hard scenes. The complete SCP-NuPlan consists of 1403 logs from the test and mini-splits, and 6382 modified scenarios, including 262 Test14-random scenarios and 272 Test14-hard scenarios.

[0098] The present invention evaluates the performance of RasterModel, Urban-Driver and planTF, and the results are shown in Table 5. Table 5 shows the results on SCP-NuPlan-Mini, SCP-NuPlan-Air and SCP-NuPlan. SNu stands for SCP-NuPlan. The non-reactive closed loop score (NR-CLS) is used as the evaluation metric. A higher NR-CLS indicates better performance. The experiments show that SCP-NuPlan does bring higher challenges to the planning model, especially on larger benchmarks, where the performance of the three models all declines. In the Test14-random scenario, SCP-NuPlan significantly reduces the model performance, while SCP-NuPlan-Mini and SCP-NuPlan-Air also reduce the model performance, proving that these two smaller splits are still effective extreme case benchmarks in fast testing.

[0099] Table 5 Results on SCP-NuPlan-Mini, SCP-NuPlan-Air, and SCP-NuPlan.

[0100]

[0101] It's worth noting that some scenarios in the NuPlan dataset are inherently difficult, especially under the Test14-hard setting, which already includes some challenging edge cases. Comparing the results of SCP-NuPlan with those of Test-hard reveals that under both Test14-random and Test14-hard, SCP-NuPlan's experimental results are consistently equal to or lower than the original results. This demonstrates that SCP-NuPlan successfully raises the NuPlan benchmark to a higher level of difficulty, at least meeting the standards of Test14-hard. Furthermore, SCP-NuPlan's scalability for difficult cases provides researchers with more challenging and extensive testing scenarios, possessing significant application value.

[0102] SCP-LimSim Experiment

[0103] SCP-LimSim leverages the advanced simulation platform LimSim++, seamlessly integrating SUMO's robust traffic flow modeling and CARLA's high-fidelity rendering to achieve highly controllable multi-agent driving behavior simulation. In this validation, SCP-LimSim's customized driving capabilities in a busy urban environment were tested by controlling different driving characters. Agent control relies on the large-scale language model Qwen-turbo, and the present invention designed multiple behavioral metrics for evaluation:

[0104] Percentage completed: The proportion of the trip that was completed without deviation or collision.

[0105] Driving time: The time (in seconds) required for the vehicle to reach the destination.

[0106] Comfort score: The comfort score during vehicle operation, calculated as follows: Where as is the longitudinal acceleration, ad is the lateral acceleration, is the rate of change of longitudinal acceleration, is the rate of change of lateral acceleration.

[0107] Efficiency score: The calculation formula is in is the average speed of the current vehicle within 10 frames, vlimit is the speed limit of the lane, The lowest average speed of surrounding vehicles within 10 frames.

[0108] Safety Score: Records the time to collision (TTC) between your vehicle and other vehicles. The minimum TTC is the safety score.

[0109] Speed ​​Limit Penalty Points: Penalty for speeding. The lower the number, the more serious the speeding offense.

[0110] The present invention designs three routes in the urban driving scenario and tests various characters on them to demonstrate the diversity and customizability of agents in SCP-LimSim. Figure 3 (a) shows a top-down view of Routes A, B, and C, where Routes A is the easiest path, Routes B tests intersections and turns, and Routes C tests circular paths and multiple intersections. (b) shows the performance scores of different characters on the three routes. The Experienced character excels in all routes, balancing efficiency, safety, and comfort. In contrast, the Angry, Hasty, and Excited characters prioritize efficiency at the expense of safety and comfort. The Novice Female, Novice Male, and Patient characters are more cautious and ensure safety, albeit at the expense of efficiency.

[0111] The final comprehensive score is the sum of the three dimensions of comfort, efficiency, and safety, and points are deducted for running red lights, speeding, and collisions. The resulting score can fully reflect driving performance and compliance with traffic regulations. In the experiment, three urban routes of varying complexity were selected, with Route A being the simplest, Route B being the most challenging, and Route C being of medium difficulty. Figure 3 The results show that the performance of the same driver on the three routes is highly consistent, verifying the stability and controllability of SCP-LimSim. Next, the analysis focuses on the more difficult route B to demonstrate the performance of SCP-LimSim.

[0112] In the test of Route B, the experienced driving characters performed the best, being able to achieve a good balance between efficiency, safety and comfort, and demonstrating proficiency in handling complex urban traffic scenarios. In contrast, radical characters such as wild, impatient and excited performed well in efficiency, but had poor safety and comfort, reflecting their high-risk driving behavior characteristics. Novice female, novice male and patient characters performed more conservatively, with higher safety but lower efficiency, which is related to the degree of caution in driving behavior. It is worth noting that novice male characters performed more aggressively, while novice female characters tended to be conservative. In order to further verify the versatility of SCP-LimSim, the present invention constructed multi-agent simulation scenarios under multiple extreme situations, such as Figure 4 As shown, these scenarios include emergency avoidance, sudden queue cutting, forced landing (e.g., an aircraft making an emergency landing on a road), circuitous driving, repeated queue cutting, ambulance (e.g., giving way to an ambulance), sudden vehicle breakdown, and rude queue cutting. Generating these scenarios only requires modifying the driver role or navigation information, demonstrating the efficiency and flexibility of SCP-LimSim in customized generation.

[0113] This application proposes an interactive simulation testing method (SCP) for new energy vehicle intelligent driving systems, aiming to address the critical issues of corner case generation and verification. By introducing LLM, scalability and customizability are achieved for planning specific corner cases, promoting the efficiency and flexibility of simulation testing.

[0114] This application proposes two core platforms: SCP-NuPlan and SCP-LimSim. SCP-NuPlan leverages real-world datasets to transform challenging extreme scenarios into scalable test benches, significantly improving the robustness and adaptability of planning algorithms in complex scenarios. SCP-LimSim, through natural language input, enables highly customizable driving agent control, capable of generating diverse and complex driving environments to meet diverse testing requirements. These two platforms effectively address the shortcomings of traditional simulation testing in corner case coverage and scenario complexity, providing a more rigorous and comprehensive testing environment for the development and verification of intelligent driving systems for new energy vehicles.

[0115] In summary, the SCP framework, through its scalability, customizability, and efficiency, has significantly improved the performance of intelligent driving planning algorithms in handling rare and unpredictable driving situations. This innovative approach provides a powerful tool for interactive simulation testing of intelligent driving systems for new energy vehicles, and has significant scientific and practical value for improving the safety, reliability, and practical application capabilities of these systems.

[0116] This application also provides an interactive simulation testing method for an intelligent driving system for a new energy vehicle, using corner cases generated by the aforementioned method as simulation test cases for the intelligent driving system. An intelligent driving system tested using these corner cases can improve its performance in extreme scenarios or complex traffic environments.

[0117] Accordingly, the embodiments of the present application also provide a computer-readable storage medium having computer-executable instructions stored therein, which implement the various method embodiments of the present application when executed by a processor. Computer-readable storage media include permanent and non-permanent, removable and non-removable media that can implement information storage by any method or technology. The information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, read-only compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable storage media does not include transient computer-readable media (transitory media), such as modulated data signals and carrier waves.

[0118] In addition, an embodiment of the present application further provides a computer program product, which includes computer-executable instructions, and when the computer-executable instructions are executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0119] All documents mentioned in this specification are considered to be included in their entirety in the disclosure of this application so that they can be used as a basis for modification when necessary. In addition, it should be understood that the above description is only a preferred embodiment of this specification and is not intended to limit the scope of protection of this specification. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of one or more embodiments of this specification should be included in the scope of protection of one or more embodiments of this specification.

Claims

1. A method for generating corner cases based on a global large language model proxy, characterized in that include: (a) obtaining road scene data, selecting several key agents within the current vehicle distance range, and extracting several most critical key frames in the entire trajectory of each key agent, wherein each of the several key frames has the largest kinematic entropy; (b) encoding the trajectories of the several key agents into structured tuples; (c) calculating the spatiotemporal relationship between the current vehicle and the plurality of key agents based on the structured tuple, and identifying extreme case scenarios and key agents that execute corresponding behaviors in the extreme case scenarios; (d) determining an extreme case type according to the identified extreme case scenario, and determining the agent coordination required for key agents to perform corresponding behaviors in the extreme case scenario; (e) generating agent prompt words for the global large language model agent based on the extreme case type and the required agent coordination; (f) Based on the agent prompt words, the behavior of the corresponding key agent in the key frame is modified through a large language model and trajectory synthesis is performed, and corner cases are generated in combination with the road scene data.

2. The method according to claim 1, characterized in that The step (f) further comprises: (f1) Smoothing the synthesized trajectory using cubic spline interpolation.

3. The method according to claim 1, characterized in that The kinematic entropy is velocity variance, acceleration variance, or the sum of velocity variance and acceleration variance.

4. The method according to claim 1, wherein The road scene data includes a NuPlan dataset, and step (a) further includes: (a1) Segmenting the road scene data.

5. The method according to claim 4, characterized in that The segmentation includes randomly sampling several scenes from each scene type in the NuPlan dataset.

6. The method according to claim 4, characterized in that The segmentation includes extracting the worst performing scenes from each scene type in the NuPlan dataset.

7. The method according to claim 1, characterized in that Such extreme situation types include lane incursions or sudden braking.

8. An interactive simulation test method for an intelligent driving system, characterized in that: The method includes using a corner case generated by the method according to any one of claims 1 to 7 as a simulation test case for an intelligent driving system.

9. The method according to claim 8, characterized in that RasterModel, Urban-Driver and planTF are used to conduct interactive simulation tests on new energy vehicles.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, implement the steps of the method according to any one of claims 1 to 7.

Images