A Method and System for Generating Key Scenarios for Autonomous Driving Based on Large Language Models
By integrating large language models, driving rules, and reward mechanisms, the problems of low generation efficiency and insufficient rationality in existing technologies are solved, enabling efficient generation of reasonable key autonomous driving scenarios and improving the effectiveness of autonomous driving testing.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- INST OF SOFTWARE - CHINESE ACAD OF SCI
- Filing Date
- 2025-12-26
- Publication Date
- 2026-05-26
AI Technical Summary
Existing methods for generating critical scenarios based on adversarial behavior are inadequate in terms of generation efficiency and rationality. The generated scenarios are often too extreme and rarely occur under actual driving conditions, making them of little significance for autonomous driving testing.
By integrating a large language model (LLM), driving rules, and reward mechanisms, and through scene prediction and reward region segmentation, the LLM is guided to make reasonable behavioral decisions for the controlled vehicle, generating key scenarios.
It improves the efficiency and rationality of generating key scenarios, ensuring that the generated scenarios are more representative under actual driving conditions, and enhances the effectiveness of autonomous driving testing.
Smart Images
Figure CN122086754A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of autonomous driving technology, specifically to a method and system for generating key autonomous driving scenarios based on a large language model. Background Technology
[0002] In the field of autonomous driving testing, the generation of critical scenarios aims to efficiently identify and correct defects in driving algorithms, and is a key research problem in this field. Existing methods for generating critical scenarios are mainly divided into three categories: those based on traffic data, those based on expert knowledge, and those based on adversarial behavior.
[0003] Adversarial behavior (BV) approaches actively create dangerous scenarios by attacking autonomous driving systems. This method stands out due to its ability to generate dynamic and diverse scenarios, making it a hot research area. NADE and AuthSim are representative works of this type. These methods first establish an objective function for scenario risk assessment, then iteratively optimize the behavioral parameters of the background vehicle (BV) based on this function, encouraging the BV to actively attack the autonomous vehicle (AV), thereby generating critical scenarios. However, this method faces a major challenge: generating effective critical scenarios, i.e., maintaining the rationality of the BV attack while ensuring the efficiency of critical scenario generation.
[0004] The issue of attack justification in BV (Browser Vehicle) stems from the design of its objective function: it encourages BV to launch arbitrary attacks on AV (Autopilot Vehicle) while ignoring the justification of such attacks. This, to some extent, "condones" overly extreme and intentional attack behaviors. For example, a BV might intentionally collide with an AV from the side or begin changing lanes and cutting in at a very short distance ahead of the AV. These extreme adversarial behaviors would make a collision unavoidable for the AV regardless of its actions. Such scenarios are widespread in current methods. On the one hand, these methods are inefficient in generating critical scenarios. On the other hand, the generated critical scenarios lack justification; such scenarios rarely occur under actual natural driving conditions and have little significance for AV testing. Therefore, existing critical scenario generation methods suffer from both inefficiency and low justification.
[0005] As another significant leap forward in AI technology, LLM possesses a range of powerful capabilities, including semantic understanding, logical reasoning, and decision-making. Currently, there is considerable work combining LLM with the field of autonomous driving testing. However, LLMs are trained on large-scale, general datasets, which to some extent lacks the expertise required for autonomous driving testing. It is necessary to integrate expertise in autonomous driving testing to guide LLM reasoning and decision-making, thereby enhancing its domain-specific capabilities in autonomous driving testing. Summary of the Invention
[0006] To address the issue of the rationality of key scenarios, this invention discloses a method and system for generating key scenarios for autonomous driving based on a large language model. It integrates LLM, driving rules, reward mechanisms, and scenario prediction methods to efficiently generate key scenarios while improving the rationality of the generated scenarios, thereby solving the problems of low generation efficiency and unreasonable generated scenarios in current scenario generation methods.
[0007] To achieve the above objectives, the technical solution of the present invention includes the following:
[0008] A method for generating key scenarios for autonomous driving based on a large language model, the method comprising: In the current scenario, select a background vehicle as the controlled vehicle for the autonomous vehicle and extract the current scenario description information; Obtain the optional actions of the controlled vehicle and calculate the reward score for the controlled vehicle to perform each optional action; Acquire descriptions of several historical scenarios and their corresponding large language model thinking processes and decision-making results; The current scene description information, optional actions and the reward score for performing the optional actions, and historical scene description information are passed to the large language model to obtain the action decision of the autonomous vehicle.
[0009] Furthermore, in the current scenario, a background vehicle is selected as the controlled vehicle for the autonomous vehicle, including: Obtain the set safe distance; Among the background vehicles in front of the autonomous vehicle, select the one that is greater than the safe distance and closest to the autonomous vehicle as the controlled vehicle.
[0010] Furthermore, the current scene description information includes: the number of lanes in the scene, the distance between the current position of the autonomous vehicle and the destination, the state information of the autonomous vehicle, and the state information of the controlled vehicle and the background vehicles around the controlled vehicle; wherein, the state information includes: vehicle ID, longitudinal position, lane, speed and acceleration, and the background vehicles around the controlled vehicle refer to the background vehicles whose distance from the controlled vehicle is less than a set distance threshold.
[0011] Furthermore, the optional actions to be acquired from the controlled vehicle include: The system acquires the set safe driving rules and the possible actions of the autonomous vehicle; wherein the possible actions include: changing lanes to the left, maintaining lane position, changing lanes to the right, accelerating, and decelerating. Based on this safe driving rule, the possible actions of the controlled vehicle are filtered to obtain the optional actions of the controlled vehicle.
[0012] Furthermore, the reward score for each optional action performed by the controlled vehicle is calculated, including: Giving autonomous vehicles the ability to perform possible actions probability ; Based on the speed of the autonomous vehicle, the speed of the controlled vehicle, and the distance between the autonomous vehicle and the controlled vehicle, the longitudinal area between the autonomous vehicle and the controlled vehicle is divided into a relatively dangerous area, a relatively boundary area, and a relatively safe area, and a reward weight is assigned to the relatively dangerous area. Reward weight of relative boundary regions and the reward weight of the relatively safe area The relatively safe area also includes lanes not occupied by controlled vehicles. Calculate the optional actions to be performed by the controlled vehicle And autonomous vehicles may perform actions Subsequently, the overlapping area between the controlled vehicle and the relatively dangerous area Area overlapping with the relative boundary region Area overlapping with the relatively safe area ; Based on probability Reward weight Reward weight Reward weight and overlapping area Overlapping area Overlapping area The controlled vehicle executes optional actions. Reward points , Indicates the current scenario.
[0013] Furthermore, the reward score , Indicates the total number of possible actions.
[0014] Furthermore, the current scene description information, optional actions and the reward score for performing the optional action, and historical scene description information are passed to the large language model for the autonomous vehicle's action decision-making, including: Large language model prompts are generated based on system prompts, scene information prompts, and few-shot prompts. The system prompts include: the task description of the large language model, the role to be played, the thought process, the input template, and the output template. The scene information prompts include: the current scene description information, the optional actions, and the reward score for performing the optional actions. The few-shot prompts are generated based on historical scene description information. Large language model inference is performed on the prompts of the large language model to obtain the action decisions of the autonomous vehicle.
[0015] A system for generating key scenarios for autonomous driving based on a large language model, the method comprising: The scene operation and description module is used to select a background vehicle as the controlled vehicle for the autonomous vehicle in the current scene and extract the current scene description information; The action and reward calculation module is used to obtain the optional actions of the controlled vehicle and calculate the reward score for the controlled vehicle to perform each optional action. The historical information acquisition module is used to acquire several historical scene descriptions corresponding to the current scene description information; The action decision module is used to pass the current scene description information, optional actions and the reward score for performing the optional actions, and historical scene description information to the large language model to obtain the action decision of the autonomous vehicle.
[0016] An electronic device includes: a processor and a memory storing computer program instructions; the processor, when executing the computer program instructions, implements the method for generating key autonomous driving scenarios based on a large language model as described above.
[0017] A computer-readable storage medium, characterized in that the computer-readable storage medium stores computer program instructions, which, when executed by a processor, implement the method for generating key autonomous driving scenarios based on a large language model as described above.
[0018] Compared with the prior art, the present invention has at least the following beneficial effects.
[0019] This invention integrates a large language model, driving rules, reward mechanisms, and scene prediction to efficiently generate reasonable key scenarios. The effectiveness of this invention has been verified through a series of comprehensive evaluation methods, ensuring its reliability and practicality in real-world applications.
[0020] This invention innovatively introduces LLM (Limited Module Management) for behavioral decisions of controlled vehicles. It also designs scene prediction, reward region partitioning, and weighting methods to calculate the corresponding reward for each optional behavior of the TAV (Transcript Vehicle), thereby guiding the LLM in making behavioral decisions for the TAV. Ablation experiments show that, compared to simply using LLM for decision-making, incorporating scene prediction and reward mechanisms can significantly improve the generation efficiency of key scenes without affecting the rationality of the generated scenes. This proves the effectiveness of the scene prediction and reward mechanisms proposed in this invention. Furthermore, compared to existing key scene generation methods such as NADE and AuthSim, this invention achieves significant improvements in both scene generation efficiency and the rationality of the generated scenes, demonstrating the advantages of this invention over existing methods.
[0021] This invention applies probability weighting to the next action of the AV (Autopilot) to predict its driving state in the next time step, making attacks on the TAV (Traffic Availability Vehicle) more accurate and improving the efficiency of generating key scenarios. To clarify the respective driving states of the AV and TAV in the next time step, this invention predicts the scenario for each optional action of the TAV. For the TAV, each optional action corresponds to a change in its driving state (including position, lane, and speed). Each optional action is iterated over; if the TAV executes that action, its state in the next time step is calculated. For the AV, based on the experience that "vehicles mostly travel at relatively stable speeds and do not require frequent deceleration, acceleration, or lane changes," this invention assumes that the AV will maintain its current speed and lane with a high probability in the near future; that is, the AV will execute the "hold" action with a high probability, and other actions with a lower probability. The behavior of the AV in the next time step is predicted based on this probability.
[0022] This invention broadens the method for dividing three relative regions and weights the risks of different regions, allowing TAVs to enter more of the relative boundary regions and improving the rationality of the scenario. To accurately measure the risk of TAVs to AVs, this invention utilizes three layers of relative distance between AVs and TAVs in the longitudinal direction and conceptually extends the relative safety distance in the lateral direction. Specifically, compared to the lane where the AV is located, TAVs in the lanes to its left and right pose a significantly lower threat. First, three layers of relative regions in the longitudinal direction within the lane where the AV is located are obtained by dividing the region into relative danger, boundary, and safety distance. Simultaneously, the relative safety region is broadened to the two lanes to the left and right of the lane where the AV is located. Finally, considering the balance between the risk and rationality of the three layers of relative regions for the AV, a reward weighting is applied to the regions. Attached Figure Description
[0023] Figure 1 This is a flowchart of a method for generating key scenarios for autonomous driving based on a large language model.
[0024] Figure 2 It is a method of dividing regions into three relative layers. Detailed Implementation
[0025] The present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be noted that the embodiments described below are intended to facilitate the understanding of the present invention and do not limit it in any way.
[0026] The autonomous driving key scenario generation system based on a large language model of the present invention integrates a large language model, driving rules, reward mechanism, and scenario prediction to efficiently generate reasonable key scenarios. The system framework is as follows: Figure 1As shown in the diagram. Specifically, in this framework, at each time step, a TAV is selected from all BVs, and the LLM makes behavioral decisions for that TAV, while other BVs maintain a natural driving style. Then, this invention formulates a set of driving rules to eliminate irrational behavior of the autonomous vehicle, resulting in the TAV's optional behaviors. Next, this invention extends the three-layer relative region partitioning method, predicting the AV's behavioral probabilities through region weighting and calculating the reward corresponding to each available behavior of the TAV. Finally, cue word engineering integrates scene descriptions, optional behaviors, and corresponding rewards into the LLM to guide its behavioral decisions for the TAV, enabling the TAV to attack the AV, thereby generating critical scenarios to test the AV's security and robustness.
[0027] This invention mainly consists of a scene operation and description module, a behavior and reward calculation module, and an LLM-based behavior decision module. The following is a detailed description of each module.
[0028] I. Scene Operation and Description Module.
[0029] 1. Selection of Controlled Vehicle (TAV).
[0030] Within the framework of this invention, at the start of each time step, a TAV is selected from all BVs. The LLM (Limited Line Module) makes a decision on its actions, launching a certain degree of adversarial attack against the AV to test its safety, while the other BVs maintain a natural driving style. In this invention, at each time step, the closest BV that is at least one safe distance in front of the AV is preferentially selected as the TAV to fundamentally eliminate extreme collisions caused by TAVs too close to the AV. Specifically, to ensure the reasonableness of the generated scenario, this invention does not directly select the BV closest to the AV, but instead pre-sets a safe distance and selects the BV that is closer to the AV than this distance as the TAV.
[0031] 2. Scene description.
[0032] Extract the number of lanes, the distance between the AV and its destination, the driving status, the TAV and all surrounding BVs, including each vehicle's ID, longitudinal position, lane, speed, and acceleration. Use all of the above information as a description of the current scene.
[0033] II. Behavior and Reward Calculation Module.
[0034] 1. Filter the optional behaviors of TAV based on driving rules.
[0035] The driving rule-based optional behavior filter eliminates obviously infeasible and prohibited actions to reduce the risk of LLMs making incorrect decisions. The filter applies a comprehensive set of driving rules based on vehicle information for a specific scenario. It filters the "possible behaviors" of a TAV to obtain "optional behaviors." Specifically, the possible behaviors of a TAV include five actions: changing lanes to the left, maintaining lane position, changing lanes to the right, accelerating, and decelerating. After filtering, those possible behaviors that clearly do not comply with safe driving rules, such as those that are highly likely to cause a collision between the TAV and other BVs, will be filtered out, thus obtaining the optional behaviors.
[0036] 2. Scene prediction.
[0037] To calculate the potential risk of each optional action of the TAV, i.e., the reward corresponding to each action, this invention performs scenario prediction for each optional action, that is, comprehensively predicts the driving states of both the AV and the TAV to calculate the reward corresponding to the action. For the TAV, each optional action corresponds to a change in the TAV's driving state, such as position, lane, and speed. By iterating through each optional action, the state of the TAV in the next time step if it takes that action is calculated. For the AV, based on the experience that "vehicles mostly travel at relatively stable speeds without frequent deceleration, acceleration, or lane changes," this invention assumes that the AV will maintain its current speed and lane with a high probability in the next time step, and will perform lane change, acceleration, or deceleration actions with a low probability. Based on this, this invention calculates the probability of the AV performing actions such as maintaining, turning left, turning right, accelerating, and decelerating. Set them to [0.6, 0.1, 0.1, 0.1, 0.1] respectively.
[0038] 3. Reward Calculation.
[0039] 1) Reward area division and weighting.
[0040] This invention first extends the method of dividing three relative zones (relative danger, boundary, and safe zone) and weights them according to the degree of risk. Specifically, this invention utilizes the three-layer relative distance between AV and TAV in the longitudinal direction, and extends the relative safe distance in the lateral direction: compared with the lane where the AV is located, the TAVs in the lanes to its left and right obviously pose a relatively lower threat. Figure 2 As shown, the three relative zones within the lane are divided using longitudinal three-layer relative distances. Furthermore, the left and right lanes of the AV are also designated as relatively safe zones.
[0041] (1) Relative danger distance Two vehicles are traveling in the same lane, and the speed of the vehicle in front is v. The speed of the following car is If the distance between the vehicles in front and behind is less than this distance, even if the following vehicle immediately decelerates at maximum speed... Even with deceleration and a reaction time of 0, a collision will still occur. The calculation method is as follows: (2) Relative boundary distance Two vehicles are traveling in the same lane, and the speed of the vehicle in front is v. The speed of the following car is If the distance between the vehicles in front and behind is less than this distance, the vehicle in front maintains its current speed, and the vehicle behind reacts within the specified time. With maximum deceleration The distance at which deceleration is achieved to prevent a collision. The calculation method is as follows: in This refers to the maximum acceleration of the vehicle behind.
[0042] (3) Relative safe distance Two vehicles are traveling in the same lane, and the speed of the vehicle in front is v. The speed of the following car is If the distance between the vehicles in front and behind is less than this distance, the vehicle in front maintains its current speed, and the vehicle behind reacts within the specified time. With minimum deceleration The distance at which deceleration is achieved to prevent a collision. The calculation method is as follows: Then, this invention adjusts the reward weights of the three relative regions based on their respective risk levels. Generally, BVs (Border Vaults) appear with a high probability at the relative boundaries and relatively safe regions of AVs (Action Zones), posing minimal risk to the AV. To balance the generation efficiency of critical scenes with the reasonableness of the generated scenes, the weights of the relative regions need to be modified: obviously, the larger the overlap area between the TAV and the relatively dangerous region, the greater its risk to the AV. Conversely, the larger the overlap area between the TAV and the relatively safe region, the greater its risk to the AV. Considering that entering too much of the relatively dangerous distance can lead to overly extreme and intentional collisions, while entering too much of the relatively safe region significantly reduces the collision probability, this invention encourages TAVs to enter the relative boundary regions of the AV more often to balance the generation efficiency of critical scenes and the reasonableness of the generated scenes. This invention achieves this goal through a weighted method: adjusting the reward weights of the relatively dangerous, boundary, and safe regions. Set them to [0.1, 0.8, 0.1] respectively.
[0043] 2) Reward calculation method.
[0044] Based on the above-mentioned scene prediction, reward region division, and weighting methods, we obtain the following results for the current scene: Below, the optional behaviors of TAV Corresponding rewards The calculation method is as follows: in, This indicates five possible behaviors that AV can perform. Indicates the AV execution settings The probability of the behavior. These represent the reward weights for the relatively dangerous, boundary, and safe zones, respectively. They respectively represent the scenarios in which Next, TAV executes Behavior and AV execution After the action, the overlapping area between TAV and the three-layer relative area.
[0045] III. LLM-based behavioral decision-making module.
[0046] By integrating the above information through cue word engineering, LLM can guide behavioral decisions for TAV. LLM cue words consist of three main parts: system cue, contextual information cue, and few-sample cue.
[0047] The system prompts include the LLM's task description, its role, step-by-step reasoning method (thinking chain), input and output templates, etc. Specifically, this invention guides the LLM to play the role of an expert in the field of autonomous driving testing. Meanwhile, to avoid violating the LLM's value of "keeping things safe and not giving dangerous advice," its task is described as "generating critical autonomous driving scenarios to test the safety of the AV." The thinking chain includes "Your decision-making process should be as follows: 1. Based on the scenario description, fully understand the current driving scenario information. 2. Based on the available actions, determine all actions the TAV can take in the current scenario. 3. Based on the rewards, evaluate each action and its corresponding reward. 4. Select the action most likely to generate the critical scenario." Scenario information prompts include text about the current scenario information regarding the driving road and all vehicles specified in steps 1 and 2, optional actions, and their corresponding rewards. Few-sample prompts include five historical scenario descriptions and corresponding behavioral decisions. These historical scenarios are typical scenarios manually selected from random simulations. In these five historical scenarios, the TAV's behavioral decisions correspond to five optional actions, providing the LLM with comprehensive reasoning examples. The inference results for the five historical scenarios correspond one-to-one with the five optional actions to prevent the LLM from learning from the decision-making rules and avoid influencing its decision-making in the current scenario. Therefore, the LLM agent can effectively understand the task, provide reasonable inference, and more closely follow the output template.
[0048] In summary, this invention integrates LLM, driving rules, reward mechanisms, and scenario prediction to efficiently generate reasonable critical scenarios. Specifically, the framework selects a TAV at each time step. Then, based on a set of driving rules developed according to this invention, behaviors of the TAV that clearly do not comply with safe driving rules are eliminated, resulting in the TAV's optional behaviors. Next, this invention extends the three-layer relative region partitioning method, calculating the reward corresponding to each optional behavior of the TAV through region weighting and AV behavior probability prediction. Finally, through cue word engineering, all information, including scenario information, optional behaviors, and their reward information, is integrated into the LLM, guiding it to make behavioral decisions for the TAV, thereby generating critical scenarios to test the safety and robustness of the AV.
[0049] This invention achieves a balance between the efficiency and plausibility of critical scene generation by extending the three-layer relative region division method and weighting it according to the degree of risk, as well as predicting the driving state of the scene in the next time step. The former makes TAV attacks more accurate and improves the efficiency of critical scene generation; the latter allows TAVs to enter relative boundary regions more often, improving the plausibility of the scene.
[0050] This invention applies probability weighting to the next action of the AV (Autopilot), predicts the AV's driving state in the next time step, and calculates the reward corresponding to each optional action of the TAV. To calculate the potential risk of each optional action of the TAV, i.e., the reward corresponding to each action, this invention performs scenario prediction for each optional action, that is, comprehensively predicts the driving states of both the AV and the TAV to calculate the reward corresponding to the action. For the TAV, each optional action corresponds to a change in the TAV's driving state, such as position, lane, and speed. By iterating through each optional action, the state of the TAV in the next time step if it takes that action is calculated. For the AV, based on the experience that "vehicles mostly travel at relatively stable speeds without frequent deceleration, acceleration, or lane changes," this invention assumes that the AV will maintain its current speed and lane with a high probability in the next time step, and will perform lane changes, acceleration, or deceleration actions with a low probability.
[0051] This invention extends the method for dividing three relative regions (relative danger, boundary, and safe zone) and weights them according to the degree of risk. Specifically, this invention utilizes the three-layer relative distance between AV and TAV in the longitudinal direction and extends the relative safe distance in the lateral direction: compared to the lane where the AV is located, the TAVs in the lanes to its left and right pose a significantly lower threat. The three-layer relative regions within a lane are divided using the three-layer relative distance in the longitudinal direction. Furthermore, the two lanes to the left and right of the AV are also divided into relatively safe zones. Then, this invention adjusts the reward weights of each of the three-layer relative regions according to the degree of risk. Generally speaking, BVs will appear with a high probability in the relative boundary and relatively safe zone of the AV, posing minimal risk to the AV. To balance the generation efficiency of critical scenarios and the rationality of the generated scenarios, the weights of the relative regions need to be modified.
Claims
1. A method for generating key scenarios for autonomous driving based on a large language model, characterized in that, The method includes: In the current scenario, select a background vehicle as the controlled vehicle for the autonomous vehicle and extract the current scenario description information; Obtain the optional actions of the controlled vehicle and calculate the reward score for the controlled vehicle to perform each optional action; Acquire descriptions of several historical scenarios and their corresponding large language model thinking processes and decision-making results; The current scene description information, optional actions and the reward score for performing the optional actions, and historical scene description information are passed to the large language model to obtain the action decision of the autonomous vehicle.
2. The method according to claim 1, characterized in that, In the current scenario, select a background vehicle as the controlled vehicle for the autonomous vehicle, including: Obtain the set safe distance; Among the background vehicles in front of the autonomous vehicle, select the one that is greater than the safe distance and closest to the autonomous vehicle as the controlled vehicle.
3. The method according to claim 1, characterized in that, The current scene description information includes: the number of lanes in the scene, the distance between the current position of the autonomous vehicle and the destination, the status information of the autonomous vehicle, and the status information of the controlled vehicle and the background vehicles around the controlled vehicle; wherein, the status information includes: vehicle ID, longitudinal position, lane, speed and acceleration, and the background vehicles around the controlled vehicle refer to the background vehicles whose distance from the controlled vehicle is less than a set distance threshold.
4. The method according to claim 1, characterized in that, Optional actions to acquire from the controlled vehicle include: The system acquires the set safe driving rules and the possible actions of the autonomous vehicle; wherein the possible actions include: changing lanes to the left, maintaining lane position, changing lanes to the right, accelerating, and decelerating. Based on this safe driving rule, the possible actions of the controlled vehicle are filtered to obtain the optional actions of the controlled vehicle.
5. The method according to claim 4, characterized in that, Calculate the reward score for each optional action performed by the controlled vehicle, including: Giving autonomous vehicles the ability to perform possible actions probability ; Based on the speed of the autonomous vehicle, the speed of the controlled vehicle, and the distance between the autonomous vehicle and the controlled vehicle, the longitudinal area between the autonomous vehicle and the controlled vehicle is divided into a relatively dangerous area, a relatively boundary area, and a relatively safe area, and a reward weight is assigned to the relatively dangerous area. Reward weight of relative boundary regions and the reward weight of the relatively safe area The relatively safe area also includes lanes not occupied by controlled vehicles. Calculate the optional actions to be performed by the controlled vehicle And autonomous vehicles may perform actions Subsequently, the overlapping area between the controlled vehicle and the relatively dangerous area Area overlapping with the relative boundary region Area overlapping with the relatively safe area ; Based on probability Reward weight Reward weight Reward weight and overlapping area Overlapping area Overlapping area The controlled vehicle executes optional actions. Reward points , Indicates the current scenario.
6. The method according to claim 5, characterized in that, The reward points , Indicates the total number of possible actions.
7. The method according to claim 1, characterized in that, The current scene description information, optional actions and reward scores for performing those actions, and historical scene description information are passed to the large language model for the autonomous vehicle's action decision-making, including: Large language model prompts are generated based on system prompts, scene information prompts, and few-shot prompts. The system prompts include: the task description of the large language model, the role to be played, the thought process, the input template, and the output template. The scene information prompts include: the current scene description information, the optional actions, and the reward score for performing the optional actions. The few-shot prompts are generated based on historical scene description information. Large language model inference is performed on the prompts of the large language model to obtain the action decisions of the autonomous vehicle.
8. A system for generating key scenarios for autonomous driving based on a large language model, characterized in that, The method includes: The scene operation and description module is used to select a background vehicle as the controlled vehicle for the autonomous vehicle in the current scene and extract the current scene description information; The action and reward calculation module is used to obtain the optional actions of the controlled vehicle and calculate the reward score for the controlled vehicle to perform each optional action. The historical information acquisition module is used to acquire several historical scene descriptions corresponding to the current scene description information; The action decision module is used to pass the current scene description information, optional actions and the reward score for performing the optional actions, and historical scene description information to the large language model to obtain the action decision of the autonomous vehicle.
9. An electronic device, characterized in that, The electronic device includes: a processor and a memory storing computer program instructions; when the processor executes the computer program instructions, it implements the method for generating key autonomous driving scenarios based on a large language model as described in any one of claims 1-7.
10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer program instructions, which, when executed by a processor, implement the method for generating key autonomous driving scenarios based on a large language model as described in any one of claims 1-7.