Edge scene generation method and apparatus, device, storage medium, and program product

Abstract

The present application relates to an edge scene generation method, device, equipment, storage medium and program product, and relates to the field of vehicle algorithm testing. The method comprises: obtaining a scene data set of automatic driving algorithm test failure and an initial scene, a target feature combination related to test failure can be extracted from the scene data set, based on the target feature combination and the initial scene, the multi-modal processing capability of a large model can be applied to generate an edge scene meeting the test requirements. This way does not need to generate a model by a large amount of historical data self-training, but extracts key failure feature combinations based on the real failure behavior of the automatic driving algorithm, decouples the extraction and processing of the failure features from the traditional generation model, so as to generate an edge scene meeting the training requirements of the automatic driving algorithm by using the inference capability of the general large model, batch acquisition of the edge scene can be realized in less time and at lower cost, and the rapid iteration requirement of the automatic driving algorithm is met.

Description

Technical Field

[0001] This invention relates to the field of vehicle algorithm testing, specifically to an edge scene generation method, apparatus, device, storage medium, and program product. Background Technology

[0002] In the development and optimization of autonomous driving algorithms, the quality and coverage of training data affect the algorithm's safety and robustness. In particular, some complex edge scenarios are weak points that can cause autonomous driving algorithms to fail. Utilizing data from these edge scenarios for targeted training is an important means to improve the performance of autonomous driving algorithms.

[0003] However, edge cases occur very infrequently in the real world, representing a typical long-tail distribution. Relying solely on large-scale road surveys to obtain training data is not only costly but also time-consuming, making it difficult to meet the needs of rapid algorithm iteration. Summary of the Invention

[0004] One of the objectives of this invention is to provide an edge scene generation method, apparatus, device, storage medium, and program product to solve the problems of high difficulty and high cost in acquiring edge scenes required for autonomous driving algorithm training.

[0005] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0006] An edge scene generation method, comprising:

[0007] Obtain the dataset of scenarios where autonomous driving algorithm tests fail, as well as the initial scenarios used to test the autonomous driving algorithm;

[0008] Extract combinations of target features related to test failures from the scenario dataset;

[0009] Based on the combination of target features and the initial scene, the multimodal processing capabilities of the large model are applied to generate edge scenes that meet the testing requirements.

[0010] Furthermore, combinations of target features related to test failures are extracted from the scenario dataset, including:

[0011] For each of the N features in the scene dataset, discretize the feature and calculate the discretization distribution entropy.

[0012] For M types of features whose discretization distribution entropy is greater than a preset threshold, different types of discretization features are selected and combined to obtain a combination of target features related to test failure. M and N are both positive integers, and M is less than or equal to N.

[0013] Furthermore, for M features whose discretized distribution entropy exceeds a preset threshold, different types of discretized features are selected and combined to obtain a combination of target features related to test failure, including:

[0014] For M features whose discretization distribution entropy is greater than a preset threshold, different types of discretization features are selected and combined to obtain Q feature combinations.

[0015] Based on the frequency of occurrence of Q feature combinations in the scene dataset and / or the number of features within the combination, select K feature combinations from the Q feature combinations as target feature combinations, where Q and K are both positive integers and K is less than Q.

[0016] Furthermore, based on the frequency of occurrence of the Q feature combinations in the scene dataset and the number of features within each combination, K feature combinations are selected from the Q feature combinations as target feature combinations, including:

[0017] For the Q feature combinations whose frequency in the scene dataset is greater than a first threshold, sort them in descending order according to the number of features within each combination, and select the top K feature combinations after descending sorting as the target feature combinations; or...

[0018] For feature combinations with more than a second threshold among the Q feature combinations, sort them in descending order of their frequency of occurrence in the scene dataset, and select the top K feature combinations after descending sorting as the target feature combinations.

[0019] Furthermore, the initial scene contains at least one background vehicle. Based on the target feature combination and the initial scene, the multimodal processing capabilities of a large model are applied to generate an edge scene that meets the testing requirements, including:

[0020] Based on the combination of target features and the road structure of the initial scene, the multimodal processing capability of the large model is applied to generate the driving trajectory of the background vehicle.

[0021] Based on the initial scene and driving trajectory, generate edge scenes that meet the testing requirements.

[0022] Furthermore, based on the combination of target features and the road structure of the initial scene, the multimodal processing capabilities of a large model are applied to generate the driving trajectories of background vehicles, including:

[0023] Obtain the state information of the vehicle under test and the background vehicle in the initial scene;

[0024] Based on the road structure of the initial scene and the status information of the background vehicles, determine the reachable destination of the background vehicles;

[0025] Based on the combination of target features, road structure, and state information of the vehicle under test, the multimodal processing capability of the large model is applied to determine the target path endpoint of the background vehicle according to the reachable path endpoint.

[0026] Generate the driving trajectory of the background vehicle based on the destination of the target path.

[0027] Furthermore, the state information includes the vehicle's initial position and initial speed. Based on the road structure of the initial scene and the state information of the background vehicle, the reachable path endpoint of the background vehicle is determined, including:

[0028] Based on the road structure of the initial scene and the initial position of the background vehicle, determine all possible temporal trajectory points of the background vehicle;

[0029] Based on the initial speed of the background vehicle and the scene constraints of the initial scene, all possible temporal trajectory points of the background vehicle are filtered to obtain the reachable path endpoint of the background vehicle. The scene constraints include at least one of vehicle dynamics constraints, road speed limits, and speed probability distributions of similar scenes.

[0030] Furthermore, based on the combination of target features, road structure, and the state information of the vehicle under test, the multimodal processing capabilities of the large model are applied to determine the target path endpoint of the background vehicle according to the reachable path endpoint, including:

[0031] Generate a map with multiple reachable path endpoints based on the road structure;

[0032] Based on the combination of target features and the state information of the vehicle under test, model prompt words are generated. The model prompt words are used to guide the model to select the target path endpoint from multiple reachable path endpoints that will cause the vehicle under test to fail the test of the autonomous driving algorithm.

[0033] The model prompts and map are input into the large model for multimodal processing to obtain the target path endpoint output by the large model.

[0034] Furthermore, before generating a map marked with multiple reachable path endpoints based on the road structure, the process also includes:

[0035] Based on the preset test duration, all reachable path endpoints are filtered, and multiple reachable path endpoints within the test duration are retained;

[0036] A map is generated based on the road structure, and multiple reachable path endpoints are marked on the map, resulting in a map with multiple reachable path endpoints marked.

[0037] Furthermore, based on the combination of target features and the road structure of the initial scene, the multimodal processing capabilities of a large model are applied to generate the driving trajectories of background vehicles, including:

[0038] Obtain the state information of the vehicle under test and the background vehicle in the initial scene;

[0039] Based on the road structure of the initial scene and the state information of the background vehicles, the temporal trajectory sequence of the background vehicles is determined. Each temporal trajectory sequence contains multiple trajectory points with different time sequences and the real-time vehicle speed of the background vehicle at the trajectory point.

[0040] Based on the combination of target features, road structure, and state information of the vehicle under test, the multimodal processing capability of the large model is applied to determine at least some trajectory points and real-time vehicle speed of the background vehicle according to the time-series trajectory sequence.

[0041] The driving trajectory of the background vehicle is generated based on at least some trajectory points and real-time vehicle speed.

[0042] An edge scene generation device, comprising:

[0043] The acquisition module is used to acquire the dataset of scenarios where autonomous driving algorithm tests failed, as well as the initial scenarios used to test autonomous driving algorithms.

[0044] The feature extraction module is used to extract combinations of target features related to test failure from the scenario dataset;

[0045] The scene generation module is used to generate edge scenes that meet the testing requirements based on the combination of target features and the initial scene, and by applying the multimodal processing capabilities of a large model.

[0046] An electronic device includes: a processor, and a memory communicatively connected to the processor;

[0047] The memory stores the instructions that the computer executes;

[0048] The processor executes computer-executable instructions stored in memory to implement edge scene generation methods as described above.

[0049] A computer-readable storage medium includes: computer-executable instructions stored in the computer-readable storage medium, which, when executed by a processor, are used to implement the edge scene generation method as described above.

[0050] A computer program product includes a computer program that, when executed by a processor, implements an edge scene generation method as described above.

[0051] The beneficial effects of this invention are as follows: It eliminates the need for self-training a model using vast amounts of historical data. Instead, it extracts key failure feature combinations based on real-world failure behaviors of autonomous driving algorithms, decoupling the extraction and processing of failure features from traditional generative models. This allows the generation of edge scenarios that meet the training requirements of autonomous driving algorithms by leveraging the reasoning capabilities of a general-purpose large model. This approach enables the batch acquisition of edge scenarios with less time and lower cost, satisfying the rapid iteration needs of autonomous driving algorithms. Attached Figure Description

[0052] Figure 1 A flowchart illustrating an edge scene generation method provided for an exemplary embodiment of the present invention;

[0053] Figure 2 A flowchart illustrating the generation of background vehicle driving trajectories is provided as an exemplary embodiment of the present invention. Figure 1 ;

[0054] Figure 3 A flowchart illustrating the generation of background vehicle driving trajectories is provided as an exemplary embodiment of the present invention. Figure 2 ;

[0055] Figure 4 A schematic diagram of the training process of an autonomous driving algorithm provided as an exemplary embodiment of the present invention;

[0056] Figure 5 A schematic diagram of an edge scene trajectory generation process provided for an exemplary embodiment of the present invention;

[0057] Figure 6 A schematic diagram of the structure of an edge scene generation device provided for an exemplary embodiment of the present invention;

[0058] Figure 7 This is a schematic diagram of the structure of an electronic device provided as an exemplary embodiment of the present invention.

[0059] The accompanying drawings have illustrated specific embodiments of the invention, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the invention in any way, but rather to illustrate the concept of the invention to those skilled in the art through reference to particular embodiments. Detailed Implementation

[0060] The embodiments of the present invention will be described below with reference to the accompanying drawings and preferred embodiments. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be understood that the preferred embodiments are only for illustrating the present invention and not for limiting the scope of protection of the present invention.

[0061] It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of the present invention. Therefore, the drawings only show the components related to the present invention and are not drawn according to the actual number, shape and size of the components in the actual implementation. In the actual implementation, the form, quantity and proportion of each component can be arbitrarily changed, and the layout of the components may also be more complex.

[0062] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present invention. Rather, they are merely examples of apparatuses and methods consistent with some aspects of the invention as detailed in the appended claims.

[0063] The terms "comprising," "including," or any other variations thereof are intended to cover a non-exclusive inclusion, such that a process, method, product, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, product, or apparatus. Without further limitation, the presence of additional identical or equivalent elements in the process, method, product, or apparatus that includes elements is not excluded. For example, the use of terms such as "first," "second," etc., to indicate names does not imply any particular order.

[0064] Currently, autonomous driving algorithms based on deep learning technology heavily rely on large-scale, high-quality training data. In particular, training data based on edge scenarios plays a crucial role in the iteration and optimization of these algorithms. Edge scenarios refer to rare, complex traffic conditions that pose significant challenges to autonomous driving algorithms, causing them to malfunction.

[0065] Data collected from the real world exhibits a "long tail effect," meaning that most of the data is of low complexity and average quality, making it difficult to obtain sufficient edge-scene data for training autonomous driving algorithms.

[0066] To address the issues of scarce edge scene samples and difficulty in obtaining road data, edge scenes that meet training requirements can be generated virtually through simulation or other methods. For example, edge scenes can be generated based on reinforcement learning combined with adversarial neural networks such as GANs (Generative Adversarial Networks), or based on causal multi-directional particle swarm optimization and variational autoencoders. However, these methods of generating edge scenes themselves require a large number of edge scene samples to train the generative model. Not only is the sample requirement high, but the training cost of models such as GANs and variational autoencoders is also extremely high. In particular, with the iteration of autonomous driving algorithms, these models also need to be repeatedly trained to generate edge scenes that adapt to the iterative algorithms, resulting in a significant consumption of computing power and time costs in obtaining edge scenes.

[0067] To address the challenge of acquiring edge scenarios, this paper proposes a technical concept. Starting with failed scenarios from autonomous driving algorithm testing, it uses statistical analysis to identify the weaknesses of the algorithms, extracts feature combinations strongly correlated with algorithm failures, and leverages the multimodal processing capabilities of a large-scale model to generate edge scenarios targeting these weaknesses. This approach eliminates the need to rely on large-scale historical data to train generative models like GANs. Instead, it accurately extracts key failure feature combinations based on the failure behavior of autonomous driving algorithms in real-world scenarios, decoupling the extraction and processing of failure features from traditional generative models. This allows the generation of edge scenarios that meet the training requirements of autonomous driving algorithms to utilize the reasoning capabilities of a general-purpose large-scale model. Compared to traditional methods that generate edge scenarios by training generative models, this approach saves the computational resources required for training generative models, obtaining a sufficient number of edge scenarios for autonomous driving algorithm training in less time and at a lower cost.

[0068] The application scenarios described above are only partial examples. Those skilled in the art can expand the applications according to specific needs and scenarios, and the embodiments of the present invention do not impose specific limitations in this regard. The method according to an exemplary embodiment of the present invention will now be described with reference to the accompanying drawings.

[0069] Figure 1 This is a flowchart illustrating an edge scene generation method provided as an exemplary embodiment of the present invention. Figure 1 As shown, the method may include:

[0070] Step S101: Obtain the dataset of scenarios where the autonomous driving algorithm failed to be tested, as well as the initial scenario used to test the autonomous driving algorithm.

[0071] The scene dataset can be obtained from actual roads in the real world or from simulations in a virtual environment. The initial scene can be a scene suitable for testing autonomous driving algorithms and can be sampled from the real world.

[0072] For example, a test vehicle equipped with an autonomous driving algorithm can be driven on a real road. If the vehicle fails due to a collision or rollover under the decision of the autonomous driving algorithm, data such as scene map, vehicle trajectory, and environmental information (such as lighting conditions) can be recorded into the scene dataset.

[0073] A scene map can refer to the road topology, which defines the position, shape, lane centerline, and other geometric attributes of each lane, thus defining the drivable area for vehicles. A vehicle trajectory can be a time-series trajectory sequence sampled at a fixed frequency over fixed time segments, where each time step can contain the vehicle's instantaneous physical attributes (such as the vehicle's position, speed, acceleration, etc.).

[0074] Test scenarios for autonomous driving algorithms can include both the vehicle under test (V2T) and background vehicles. The V2T can refer to the vehicle on which the autonomous driving algorithm is deployed and used to make decisions for testing; it can also be called the primary vehicle or target vehicle. Background vehicles can refer to any other vehicle in the test scenario besides the V2T; they can also be called social vehicles. The vehicle trajectories in the dataset of failed test scenarios can include the trajectories of both the V2T and background vehicles.

[0075] For example, real-world samples can be taken of the vehicle under test and its surrounding traffic scene. The state changes of the vehicle under test and other surrounding background vehicles over time can be recorded, centered on the vehicle itself. The autonomous driving algorithm can be tested by replaying the autonomous driving dataset using a simulator. This involves keeping the trajectory data of all background vehicles unchanged and replacing the autonomous driving vehicle in the original dataset with the autonomous driving algorithm to be tested, thus verifying the algorithm's performance in that scenario. Through large-scale testing, the simulator can record and statistically analyze driving data from all instances of failed autonomous driving algorithm behavior, recording the characteristic data at the time of the failure, and constructing a dataset of failed test scenarios.

[0076] Step S102: Extract the target feature combination related to test failure from the scenario dataset.

[0077] The core of this step lies in using statistical methods to accurately screen out typical features and feature combinations that are strongly correlated with algorithm failure from a large number of failure scenarios, identify the weaknesses of autonomous driving algorithms, and provide a data foundation for the generation of edge scenarios.

[0078] Scene datasets can record various features that cause autonomous driving algorithms to fail, such as weather, lighting, and background vehicle behavior. Based on statistical modeling and analysis, typical features strongly correlated with algorithm failure can be identified. For example, features such as heavy rain, a vehicle cutting into the lane ahead, and close following distance to the vehicle ahead appear frequently in scene datasets. These features may be typical causes of algorithm failure. Combining these different types of features can yield target feature combinations related to test failure (e.g., a vehicle cutting into the lane ahead plus failure to maintain a safe distance).

[0079] Step S103: Based on the target feature combination and the initial scene, apply the multimodal processing capability of the large model to generate an edge scene that meets the testing requirements.

[0080] The large model can be a general-purpose model with multimodal processing capabilities. It can perform inference based on input prompts and multimodal data, generating inference results that meet the requirements of the prompts.

[0081] For example, the initial scene may include background vehicles in addition to the vehicle under test. A map can be generated based on the road topology of the initial scene and the positions of the vehicle under test and the background vehicles. Based on the combination of target features, prompts can be constructed to guide the large model in generating edge scenes. The prompts and the map are input into the large model, enabling it to infer edge scenes that cause the autonomous driving algorithm to fail with a high probability based on the combination of target features. For example, the large model can generate the trajectory of the background vehicle that causes the target vehicle to collide under low light conditions based on the prompt "referring to the combination of target features related to test failure" and the initial position of the background vehicle marked on the map. Then, based on the trajectory and the road topology of the initial scene, the low light conditions are used as the environmental conditions of the scene to generate edge scenes.

[0082] In the above embodiments, by acquiring the scenario dataset of failed autonomous driving algorithm tests and the initial scenario, target feature combinations related to test failures can be extracted from the scenario dataset. Based on the target feature combinations and the initial scenario, the multimodal processing capabilities of a large model can be applied to generate edge scenarios that meet the testing requirements. This approach does not require self-training the generative model using a large amount of historical data. Instead, it extracts key failure feature combinations based on the real failure behavior of the autonomous driving algorithm, decoupling the extraction and processing of failure features from the traditional generative model. This allows the inference capabilities of the general-purpose large model to generate edge scenarios that meet the training requirements of the autonomous driving algorithm. This method can acquire edge scenarios in batches with less time and lower cost, meeting the rapid iteration needs of autonomous driving algorithms.

[0083] In one embodiment, extracting a combination of target features related to test failure from a scenario dataset may include:

[0084] For N features in the scene dataset, each feature is discretized and the discretization distribution entropy is calculated. For M features whose discretization distribution entropy is greater than a preset threshold, different types of discretized features are selected and combined to obtain the target feature combination related to test failure.

[0085] Where M and N are both positive integers, and M is less than or equal to N. Discrete distribution entropy is an indicator that can measure the uncertainty of feature value distribution. The higher the entropy value of a feature, the more dispersed the feature value distribution of that feature in the scene dataset.

[0086] The features in the scene dataset may be discrete features (such as weather conditions such as sunny, rainy, or foggy) or continuous features (such as vehicle speed).

[0087] In this embodiment of the invention, continuous features such as vehicle speed in the scene dataset can be discretized and binned, with each bin representing a discrete interval. Data sample values ​​within each discrete interval are then statistically analyzed. For example, the vehicle speed of the test vehicle can be divided into discrete intervals such as [0, 30 km / h], (30 km / h, 60 km / h], (60 km / h, 90 km / h], and (90 km / h, 120 km / h]. Each discrete interval can be used as a separate discretized feature under the category of vehicle speed. Similarly, for the feature of a background vehicle cutting into the lane in front of the test vehicle (i.e., the test vehicle), multiple discrete intervals can be defined based on the distance between the background vehicle's entry point and the test vehicle, such as intervals where the entry point is between 0 and 5 meters, or between 10 and 15 meters.

[0088] For each feature, its discretized distribution entropy can be calculated based on the data sample values ​​within each discrete interval under that feature (e.g., how many samples of the test vehicle's speed are between 0 and 30 km / h, how many samples are between 30 and 60 km / h, etc. in the scene dataset). Then, the discretized distribution entropy of this feature can be compared with a preset threshold. If the distribution entropy is less than the threshold, it indicates that the feature's value is too unidirectional, and the feature can be removed. If there are N features, after removing features with distribution entropy less than the threshold, M features can be retained.

[0089] After obtaining M features using discretized distribution entropy, different types of discretized features (i.e., discrete intervals) can be selected and combined. For example, the speed of the vehicle under test and the entry of a background vehicle into the lane ahead are two different types of features. We can select the discrete interval (30km / h, 60km / h) from the discrete interval of the vehicle under test's speed, and then select the discrete interval "the distance between the entry point and the vehicle under test is 0 to 5 meters" from the discrete interval of the entry of a background vehicle into the lane ahead. Combining these two discrete intervals yields a feature combination related to test failure. Similarly, we can select three discrete interval combinations based on three different features, or four discrete interval combinations based on four different features, and so on, obtaining at most Q feature combinations. These feature combinations can be directly used as target feature combinations related to test failure, or some feature combinations can be eliminated using preset rules, retaining the remaining combinations as target feature combinations.

[0090] In the above embodiments, by calculating the discretized distribution entropy of features, redundant features can be eliminated by combining thresholds, ensuring that the generated target feature combinations are strongly correlated with the reasons for algorithm failure. By utilizing these target feature combinations that are strongly correlated with algorithm failure, large models can generate edge scenarios that have a higher probability of causing autonomous driving algorithm tests to fail, thereby improving the quality of edge scenarios.

[0091] In one embodiment, for M features whose discretized distribution entropy is greater than a preset threshold, different types of discretized features are selected and combined to obtain a target feature combination related to test failure, including:

[0092] For M features whose discretization distribution entropy is greater than a preset threshold, different types of discretization features are selected and combined to obtain Q feature combinations. Based on the frequency of occurrence of the Q feature combinations in the scene dataset and / or the number of features in the combination, K feature combinations are selected from the Q feature combinations as target feature combinations.

[0093] Where Q and K are both positive integers, and K is less than Q.

[0094] In this embodiment of the invention, after obtaining feature combinations composed of multiple discretized features under different types of features, these feature combinations can be filtered based on two dimensions: the frequency of occurrence of the combination in the scene dataset and the feature diversity within the combination, retaining feature combinations with higher priority. Here, feature diversity within a combination can refer to the number of features within the combination. For example, some feature combinations are obtained by combining one discretized feature from each of two different features; such combinations contain 2 discretized features, and their feature diversity can be represented as 2. Q feature combinations can be obtained by iterating through all discretized features of M types of features, covering combinations of 2 discretized features, combinations of 3 discretized features, ..., combinations of M discretized features.

[0095] For example, for Q feature combinations, they can be sorted in descending order according to the frequency of each feature combination in the scene dataset, and the top K of the descending sort results can be taken as the target feature combination.

[0096] For example, for Q feature combinations, they can be sorted in descending order according to the feature diversity within each combination, and the top K of the descending sort results can be taken as the target feature combination.

[0097] For example, for Q feature combinations, we can first filter them according to their frequency of occurrence in the scene dataset, retaining feature combinations whose frequency of occurrence is greater than the first threshold, and then sort these retained feature combinations in descending order according to the number of features in the combination, and select the top K feature combinations after descending sort as the target feature combinations.

[0098] For example, for Q feature combinations, we can first filter them according to the number of features in the combination, retain the feature combinations with the number of features in the combination greater than the second threshold, and then sort these retained feature combinations in descending order according to their frequency of occurrence in the scene dataset, and select the top K feature combinations after descending sort as the target feature combinations.

[0099] In the above embodiments, filtering based on the frequency of occurrence of feature combinations in the scene dataset and the number of features within the combination can further eliminate feature combinations that are weakly correlated with the failure of autonomous driving algorithm testing, improve the quality of target feature combinations, and thus improve the quality of edge scenes generated by large models.

[0100] The inventors further discovered that the driving trajectories of background vehicles in the test scenario have a significant impact on the testing of autonomous driving algorithms. In many cases, the autonomous driving algorithm fails because the driving trajectories of the background vehicles are not within the data range used for training the algorithm. When faced with a combination of unlearned background vehicle trajectories and the scene, the autonomous driving algorithm is prone to making incorrect decisions. Therefore, generating background vehicle trajectories that can cause the autonomous driving algorithm to fail, and using these trajectories to generate edge scenes, can improve the quality of edge scenes and their training effect on the autonomous driving algorithm.

[0101] In one embodiment, the initial scene contains at least one background vehicle. Based on the target feature combination and the initial scene, the multimodal processing capabilities of a large model are applied to generate an edge scene that meets the testing requirements, which may include:

[0102] Based on the combination of target features and the road structure of the initial scene, the multimodal processing capability of the large model is applied to generate the driving trajectory of the background vehicle; based on the initial scene and the driving trajectory, the edge scene that meets the test requirements is generated.

[0103] In this embodiment of the invention, a map is generated based on the road topology of the initial scene and the positions of the vehicle under test and the background vehicle. The target features are combined as part of the text prompt words. At the same time, the task description and scene description are added to the text prompt words to guide the large model to learn the relationship between the background vehicle trajectory and the failure of the autonomous driving algorithm and generate a high-risk trajectory that can trigger the failure of the autonomous driving algorithm and is applicable to the initial scene. Then, the edge scene is generated based on the trajectory and relevant information of the initial scene (such as environmental factors in the scene, the position of the vehicle under test, etc.).

[0104] For example, the task description could include: the large model needs to act as a "vehicle trajectory generation expert in edge scenarios," with the core objective of generating high-risk trajectories that can accurately trigger the failure of autonomous driving algorithms. The scenario description could then list in detail the specific attributes of the failure feature combinations, the relative positional relationship between the test vehicle and background vehicles, road speed limits, steering rules, and other relevant constraints, ensuring that the large model clearly understands the task boundaries and the core requirements of the scenario.

[0105] In the above embodiments, by constructing text prompts to guide the large model to generate the driving trajectory of background vehicles in the scene, the scene can be made to have background vehicle trajectories that target the weaknesses of the autonomous driving algorithm when generating edge scenes. This allows for targeted testing of the autonomous driving algorithm's decision-making ability in the face of different background vehicle trajectories, thereby improving the quality of edge scenes.

[0106] In one embodiment, such as Figure 2 As shown, based on the combination of target features and the road structure of the initial scene, the multimodal processing capabilities of a large model are applied to generate the driving trajectory of the background vehicle, which may include:

[0107] Step S201: Obtain the state information of the vehicle under test and the background vehicle in the initial scene.

[0108] The status information may include the position, speed, direction of travel, and power of the vehicle under test and the background vehicle in the initial scene.

[0109] For example, targeted sampling can be performed based on real traffic datasets covering various road conditions such as urban roads and highways to ensure that the sampling scenarios cover different road structures. Then, an initial scenario that can be used for simulation can be constructed based on the sampling scenarios, and the initial state of the initial scenario can be set. This initial state can include core parameters such as the real-time position coordinates, initial driving speed, and driving direction of the test vehicle and the background vehicle in the first frame of the simulation. At the same time, road structure information of the corresponding scenario can be extracted synchronously, including key data such as the number of lanes, lane boundary coordinates, and lane connections.

[0110] Step S202: Determine the reachable path endpoint of the background vehicle based on the road structure of the initial scene and the state information of the background vehicle.

[0111] For example, based on the road structure information of the initial scene and the initial position of the background vehicle, and with the lane centerline as the planning reference, a trajectory search algorithm can be used to calculate all possible driving trajectories of the vehicle that meet the constraints of vehicle dynamic characteristics and real physical rules within a preset time range in the future. Then, the trajectory points in these driving trajectories can be used as the reachable path endpoints (i.e., the reachable path endpoints) of the background vehicle.

[0112] Step S203: Based on the target feature combination, road structure and the state information of the vehicle under test, the multimodal processing capability of the large model is applied to determine the target path endpoint of the background vehicle according to the reachable path endpoint.

[0113] For example, the reachable path endpoints of the background vehicle obtained in the previous step can be incorporated into the map to enable the large model to understand the relationship between the background vehicle trajectory and the test vehicle in map form. Furthermore, the reasoning task of the large model can be changed through prompts, transforming it from generating the background vehicle trajectory out of thin air to selecting target path endpoints from known reachable path endpoints that have a high probability of triggering the failure of the autonomous driving algorithm.

[0114] Step S204: Generate the driving trajectory corresponding to the background vehicle based on the destination of the target path.

[0115] For example, the endpoint of the target path can be used as a constraint condition. Using a cubic spline interpolation algorithm, based on the trajectory points of the driving trajectory determined in step S202, trajectory smoothing interpolation is performed according to a preset time step to supplement the instantaneous velocity, acceleration and steering angle data between adjacent trajectory points, forming a complete background vehicle time sequence trajectory.

[0116] In the above embodiments, by changing the input parameters of the large model, the large model learns the possible reachable path endpoints. Based on this, the large model only needs to select from a finite number of known path endpoints to output the target path endpoint of the background vehicle, without having to plan the background vehicle's driving path out of thin air. This not only reduces the inference cost of the large model, but also reduces the impact of the large model's own inference performance on the background vehicle trajectory generation result, reduces the error in the model's inference process, and ensures the quality of the final generated edge scene.

[0117] In one embodiment, the state information includes the vehicle's initial position and initial speed. Based on the road structure of the initial scene and the state information of the background vehicle, the reachable path endpoint of the background vehicle is determined, including:

[0118] Based on the road structure of the initial scene and the initial position of the background vehicle, determine all possible temporal trajectory points of the background vehicle; based on the initial speed of the background vehicle and the scene constraints of the initial scene, filter all possible temporal trajectory points of the background vehicle to obtain the reachable path endpoint of the background vehicle.

[0119] The scenario constraints include at least one of the following: vehicle dynamics constraints, road speed limits, and vehicle speed probability distributions for similar scenarios.

[0120] In this embodiment of the invention, based on the road topology and initial vehicle position of the initial scene, a trajectory search algorithm can calculate all possible driving trajectories of the vehicle within a preset future time range. These trajectories can be stored as a standardized sequence of trajectory points, with each trajectory point containing information such as a timestamp, two-dimensional coordinates, instantaneous speed, and steering angle, ensuring the continuity and accuracy of the trajectory. Then, the trajectory data that does not meet these conditions can be filtered based on vehicle dynamics constraints (such as the maximum achievable vehicle speed) and road speed limits (such as a speed not exceeding 60 km / h).

[0121] In some possible implementations, the probability distribution of vehicle speeds in real-world scenarios similar to the initial scenario can be obtained. This distribution represents the probability of a vehicle traveling at a specific speed in similar scenarios. Based on this distribution, trajectory points corresponding to background vehicle speeds with a probability less than a preset threshold can be eliminated from the possible temporal trajectory points. For example, a driving trajectory obtained through a trajectory search algorithm requires the vehicle to continuously travel at an average speed of less than 2 km / h. This speed has an extremely low probability of occurring in similar scenarios, so the trajectory can be eliminated.

[0122] In one embodiment, based on the combination of target features, road structure, and state information of the vehicle under test, and applying the multimodal processing capabilities of a large model, the target path endpoint of the background vehicle is determined according to the reachable path endpoint, which may include:

[0123] A map with multiple reachable path endpoints is generated based on the road structure; model prompts are generated based on the combination of target features and the state information of the vehicle under test. The model prompts are used to guide the model to select the target path endpoint from the multiple reachable path endpoints that would cause the vehicle under test to fail the autonomous driving algorithm test; the model prompts and the map are input into a large model for multimodal processing to obtain the target path endpoints output by the large model.

[0124] Through the above steps, a map can be generated based on the road topology of the initial scene and the positions of the test vehicle and the background vehicle. Then, the reachable path endpoints determined in the previous step are marked on the generated map, so that the large model can perceive the reachable path endpoints of the background vehicle by parsing the map. At the same time, combined with prompt words, the large model can be guided to filter the reachable path endpoints on the map and output the target path endpoints with a high probability of triggering the failure of the autonomous driving algorithm to generate the driving trajectory of the background vehicle.

[0125] In some possible implementations, before generating a map marked with multiple reachable path endpoints based on the road structure, the following may also be included:

[0126] All reachable path endpoints are filtered according to the preset test duration, and multiple reachable path endpoints within the test duration are retained; a map is generated based on the road structure, and the multiple retained reachable path endpoints are marked on the map, resulting in a map marked with multiple reachable path endpoints.

[0127] The simulation testing requirements for autonomous driving algorithms may have requirements on the test duration. For example, testing the decision-making performance of the autonomous driving algorithm within 1 minute. However, the reachable path endpoint determined in the above embodiment may require the background vehicle to travel for more than 1 minute to reach. For such path endpoints, the test requirements can be obtained in advance and eliminated according to the test duration.

[0128] In one embodiment, such as Figure 3 As shown, based on the combination of target features and the road structure of the initial scene, the multimodal processing capabilities of a large model are applied to generate the driving trajectories of background vehicles, including:

[0129] Step S301: Obtain the state information of the vehicle under test and the background vehicle in the initial scene.

[0130] Step S302: Determine the temporal trajectory sequence of the background vehicles based on the road structure of the initial scene and the state information of the background vehicles.

[0131] Each time-series trajectory sequence contains multiple trajectory points with different time sequences and the real-time vehicle speed of the background vehicle at each trajectory point. For determining the time-series trajectory sequence of the background vehicle, refer to the above-described embodiment for determining the reachable end point of the background vehicle's path.

[0132] Step S303: Based on the target feature combination, road structure and the state information of the vehicle under test, the multimodal processing capability of the large model is applied to determine at least some trajectory points and real-time vehicle speed of the background vehicle according to the time-series trajectory sequence.

[0133] Step S304: Generate the driving trajectory of the background vehicle based on at least some trajectory points and real-time vehicle speed.

[0134] In this embodiment of the invention, a trajectory sequence containing the time-series trajectory points of the background vehicle and the corresponding vehicle speed can be input into the large model, enabling the large model to learn more complex driving behaviors of the background vehicle. Combined with the previously extracted target feature combinations related to algorithm test failures, the large model can not only output the path endpoint of the background vehicle, but also output the time-series trajectory points and corresponding vehicle speeds of the background vehicle before reaching the path endpoint. Thus, by designing the speed at which the background vehicle passes by, interference is generated with the vehicle under test, increasing the probability of causing the autonomous driving algorithm to fail and generating higher quality edge scenes.

[0135] The following description, in conjunction with the accompanying drawings, illustrates some possible application examples of the edge scene generation method according to embodiments of the present invention.

[0136] Figure 4 This is a schematic diagram of the training process of an autonomous driving algorithm provided as an exemplary embodiment of the present invention. Figure 5 This is a schematic diagram of an edge scene trajectory generation process provided for an exemplary embodiment of the present invention.

[0137] like Figure 4 As shown, starting with the failure scenario of autonomous driving algorithm testing, the weak points of the autonomous driving algorithm can be identified through statistical analysis, the feature combinations that are strongly correlated with the algorithm failure can be extracted, and the multimodal processing capability of the large model can be combined to generate edge scenarios targeting the algorithm's weak points.

[0138] For example, when generating edge scenes, it can be done according to Figure 5 The process shown involves constructing prompt words based on the combination of target features and the initial scene description, generating a map based on the road topology of the initial scene and marking the reachable path endpoints of background vehicles, and then inputting the prompt words and map into a large model to guide the large model to output the target path endpoints that have a high probability of triggering the failure of the autonomous driving algorithm. With the target path endpoints as constraints, a complete trajectory can be generated through a trajectory interpolation algorithm.

[0139] Then, the generated edge scene can be adapted and converted according to the simulator's input format requirements, including scene parameters, trajectory data, and map information, and deployed in the simulation environment. The scene legality and consistency dual verification process is initiated. Dynamic constraint verification is achieved by comparing the instantaneous acceleration, deceleration, and steering angular velocity of social vehicles during actual driving with the preset thresholds. Traffic rule verification can be performed by checking whether vehicles have violated regulations through the simulator's built-in rule detection module. Collision verification ensures that the scene has no logical conflicts or safety hazards. Subsequently, the autonomous driving algorithm to be optimized is integrated into the simulation scene, and multiple repeated test tasks are set up to synchronously collect all-dimensional driving data such as the algorithm's perception data, decision commands, control outputs, and vehicle driving trajectories. The key time nodes and data characteristics when the algorithm fails (such as collisions, path deviations, and decision delays) are recorded.

[0140] Based on the driving data collected from the simulation, failure-related data such as decision-making bias samples and perception miss detection samples in edge scenarios are extracted from the algorithm. A new round of targeted training is then conducted on the algorithm to improve its environmental adaptability and risk response capabilities in high-risk edge scenarios. After training, the optimized autonomous driving algorithm is placed back into the original generated edge scenario for testing under the same conditions. By comparing the failure rate and trajectory tracking accuracy of the algorithm before and after optimization, its ability to overcome the original edge scenario is verified. Finally, an iterative optimization loop is initiated and repeated. Figure 4 The process shown in S1~S5 involves re-mining failure feature combinations based on the new failure data of the optimized algorithm to generate more targeted edge scenarios. The algorithm performance is continuously iterated and refined until its failure rate in the target edge scenario is lower than the preset threshold, thus meeting the security requirements of practical applications.

[0141] Figure 6 This is a schematic diagram of an edge scene generation device provided as an exemplary embodiment of the present invention. (See diagram below.) Figure 6 As shown, the edge scene generation device 600 may include:

[0142] The acquisition module 601 is used to acquire the scenario dataset of failed autonomous driving algorithm tests and the initial scenario for testing the autonomous driving algorithm.

[0143] Feature extraction module 602 is used to extract a combination of target features related to test failure from the scene dataset;

[0144] The scene generation module 603 is used to generate edge scenes that meet the testing requirements based on the combination of target features and the initial scene, and by applying the multimodal processing capabilities of a large model.

[0145] In some possible implementations, the feature extraction module 602 can also be used to: discretize each of the N features in the scene dataset and calculate the discretization distribution entropy; for the M features whose discretization distribution entropy is greater than a preset threshold, select different types of discretized features to combine to obtain a combination of target features related to test failure, where M and N are both positive integers and M is less than or equal to N.

[0146] In some possible implementations, the feature extraction module 602 can also be used to: select different types of discrete features for M features whose discrete distribution entropy is greater than a preset threshold, and combine them to obtain Q feature combinations; select K feature combinations from the Q feature combinations as target feature combinations based on the frequency of occurrence of the Q feature combinations in the scene dataset and / or the number of features in the combination, where Q and K are both positive integers and K is less than Q.

[0147] In some possible implementations, the feature extraction module 602 can also be used to: sort the feature combinations among the Q feature combinations whose frequency of occurrence in the scene dataset is greater than a first threshold in descending order according to the number of features within the combination, and select the top K feature combinations after descending order as target feature combinations; or, sort the feature combinations among the Q feature combinations whose number of features within the combination is greater than a second threshold in descending order according to the frequency of occurrence in the scene dataset, and select the top K feature combinations after descending order as target feature combinations.

[0148] In some possible implementations, the scene generation module 603 can also be used to: generate the driving trajectory of the background vehicle based on the combination of target features and the road structure of the initial scene, applying the multimodal processing capabilities of the large model; and generate an edge scene that meets the testing requirements based on the initial scene and the driving trajectory.

[0149] In some possible implementations, the scene generation module 603 can also be used to: acquire the state information of the test vehicle and the background vehicle in the initial scene; determine the reachable path endpoint of the background vehicle based on the road structure of the initial scene and the state information of the background vehicle; based on the combination of target features, road structure and the state information of the test vehicle, apply the multimodal processing capability of the large model to determine the target path endpoint of the background vehicle based on the reachable path endpoint; and generate the driving trajectory corresponding to the background vehicle based on the target path endpoint.

[0150] In some possible implementations, the scene generation module 603 can also be used to: determine all possible temporal trajectory points of the background vehicle based on the road structure of the initial scene and the initial position of the background vehicle; and filter all possible temporal trajectory points of the background vehicle based on the initial speed of the background vehicle and the scene constraints of the initial scene to obtain the reachable path endpoint of the background vehicle. The scene constraints include at least one of vehicle dynamics constraints, road speed limits, and speed probability distributions of similar scenes.

[0151] In some possible implementations, the scene generation module 603 can also be used to: generate a map marked with multiple reachable path endpoints based on the road structure; generate model prompt words based on the combination of target features and the state information of the vehicle under test, the model prompt words being used to guide the model to select the target path endpoint from the multiple reachable path endpoints that would cause the vehicle under test to fail in testing the autonomous driving algorithm; input the model prompt words and the map into a large model for multimodal processing to obtain the target path endpoints output by the large model.

[0152] In some possible implementations, the scene generation module 603 can also be used to: filter all reachable path endpoints according to a preset test duration, and retain multiple reachable path endpoints within the test duration; generate a map according to the road structure, and mark the multiple retained reachable path endpoints on the map to obtain a map marked with multiple reachable path endpoints.

[0153] In some possible implementations, the scene generation module 603 can also be used to: acquire the state information of the test vehicle and the background vehicle in the initial scene; determine the temporal trajectory sequence of the background vehicle based on the road structure of the initial scene and the state information of the background vehicle, wherein each temporal trajectory sequence contains multiple trajectory points with different time sequences and the real-time vehicle speed of the background vehicle at the trajectory points; based on the target feature combination, road structure and the state information of the test vehicle, apply the multimodal processing capability of the large model to determine at least some trajectory points and real-time vehicle speed of the background vehicle according to the temporal trajectory sequence; and generate the driving trajectory of the background vehicle based on at least some trajectory points and real-time vehicle speed.

[0154] The edge scene generation device provided in this embodiment is used to execute the technical solution in any of the foregoing method embodiments. Its implementation principle and technical effect are similar, and will not be described again here.

[0155] It should be understood that the above-described device embodiments are merely illustrative, and the device of the present invention can also be implemented in other ways. For example, the division of units / modules in the above embodiments is only a logical functional division, and there may be other division methods in actual implementation. For example, multiple units, modules, or components may be combined, or integrated into another system, or some features may be ignored or not executed.

[0156] Furthermore, unless otherwise specified, the functional units / modules in the various embodiments of the present invention can be integrated into one unit / module, or each unit / module can exist physically separately, or two or more units / modules can be integrated together. The integrated units / modules described above can be implemented in hardware or as software program modules.

[0157] Figure 7 This is a schematic diagram of the structure of an electronic device provided as an exemplary embodiment of the present invention. For example... Figure 7 As shown, the electronic device 70 includes:

[0158] Processor 71, memory 72, and communication interface 73;

[0159] The memory 72 is used to store the executable instructions of the processor 71; the executable instructions can be instructions that the computer can execute.

[0160] The processor 71 is configured to execute the technical solutions in any of the foregoing method embodiments by executing executable instructions.

[0161] Optionally, the memory 72 can be either standalone or integrated with the processor 71.

[0162] Optionally, when the memory 72 is a device independent of the processor 71, the electronic device 70 may further include:

[0163] Bus 74, memory 72 and communication interface 73 are connected to processor 71 through bus 74 and complete communication with each other. Communication interface 73 is used to communicate with other devices.

[0164] Optionally, the communication interface 73 can be implemented using a transceiver. The communication interface is used to enable communication between the database access device and other devices (e.g., clients, read-write databases, and read-only databases). The memory may include random access memory (RAM) and may also include non-volatile memory, such as at least one disk drive.

[0165] Bus 74 can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of representation, only one line is used in the diagram, but this does not imply that there is only one bus or one type of bus.

[0166] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0167] The electronic device is used to execute the technical solutions in any of the foregoing method embodiments. Its implementation principle and technical effect are similar, and will not be described again here.

[0168] This invention also provides a readable storage medium, which can be a computer-readable storage medium storing a computer program thereon. When the computer program is executed by a processor, it implements the technical solution provided in any of the foregoing method embodiments.

[0169] This invention also provides a computer program product, including a computer program, which, when executed by a processor, is used to implement the technical solutions provided in any of the foregoing method embodiments.

[0170] Those skilled in the art will understand that all or part of the steps of the above-described method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When executed, the program performs the steps of the above-described method embodiments; and the aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks.

[0171] In the above embodiments, the descriptions of each embodiment have their own emphasis. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments. The technical features of the above embodiments can be combined arbitrarily. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as the combination of these technical features does not contradict each other, it should be considered within the scope of this specification.

[0172] Other embodiments of the invention will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This invention is intended to cover any variations, uses, or adaptations of the invention that follow the general principles of the invention and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of the invention are indicated by the following claims.

[0173] The above embodiments are merely preferred embodiments provided to fully illustrate the present invention, and the scope of protection of the present invention is not limited thereto. Equivalent substitutions or modifications made by those skilled in the art based on the present invention are all within the scope of protection of the present invention.

Claims

1. A method for generating edge scenes, characterized in that, include: Obtain a dataset of scenarios where autonomous driving algorithm tests fail and an initial scenario for testing the autonomous driving algorithm. The scenario dataset contains features of multiple classes, and the initial scenario contains at least one background vehicle. Based on the frequency of occurrence of features in the scenario dataset, different types of features are combined to obtain a combination of target features related to test failure. Based on the road structure of the initial scene and the status information of the background vehicle, determine the reachable path endpoint of the background vehicle; Based on the target feature combination, the road structure, and the state information of the vehicle under test, the large model is instructed by prompt words to determine the target path endpoint of the background vehicle according to the reachable path endpoint, and to generate the driving trajectory of the background vehicle according to the target path endpoint. The large model is a model with multimodal processing capabilities. Based on the initial scene and the driving trajectory, an edge scene that meets the testing requirements is generated.

2. The method according to claim 1, characterized in that, The step of combining different types of features based on the frequency of occurrence of features in the scene dataset to obtain a target feature combination related to test failure includes: For the N features in the scenario dataset, each feature is discretized, and the discretization distribution entropy is calculated. For M features whose discretization distribution entropy is greater than a preset threshold, different types of discretization features are selected and combined according to the frequency of occurrence of the features to obtain a target feature combination related to test failure. M and N are both positive integers, and M is less than or equal to N.

3. The method according to claim 2, characterized in that, For the M features whose discretized distribution entropy is greater than a preset threshold, different types of discretized features are selected and combined according to the frequency of occurrence of the features to obtain a target feature combination related to test failure, including: For M features whose discretization distribution entropy is greater than a preset threshold, different types of discretization features are selected and combined to obtain Q feature combinations. Based on the frequency of occurrence of the Q feature combinations in the scene dataset and / or the number of features within the combinations, K feature combinations are selected from the Q feature combinations as target feature combinations, where Q and K are both positive integers and K is less than Q.

4. The method according to claim 3, characterized in that, Based on the frequency of occurrence of the Q feature combinations in the scene dataset and the number of features within each combination, K feature combinations are selected from the Q feature combinations as target feature combinations, including: For the feature combinations among the Q feature combinations whose frequency of occurrence in the scene dataset is greater than a first threshold, sort them in descending order according to the number of features within the combination, and select the top K feature combinations after descending sorting as the target feature combinations; or, For the feature combinations among the Q feature combinations, where the number of features within the combination is greater than the second threshold, sort them in descending order according to their frequency of occurrence in the scene dataset, and select the top K feature combinations after descending sorting as the target feature combinations.

5. The method according to any one of claims 1 to 4, characterized in that, The state information includes the vehicle's initial position and initial speed. Determining the reachable path endpoint of the background vehicle based on the road structure of the initial scene and the state information of the background vehicle includes: Based on the road structure of the initial scene and the initial position of the background vehicle, determine all possible temporal trajectory points of the background vehicle; Based on the initial speed of the background vehicle and the scene constraints of the initial scene, all possible temporal trajectory points of the background vehicle are filtered to obtain the reachable path endpoint of the background vehicle. The scene constraints include at least one of vehicle dynamics constraints, road speed limits, and speed probability distributions of similar scenes.

6. The method according to any one of claims 1 to 4, characterized in that, The step of determining the target path endpoint of the background vehicle based on the target feature combination, the road structure, and the state information of the vehicle under test, by instructing the large model through prompt words, includes: A map is generated based on the road structure, marking the endpoints of multiple reachable paths; Based on the target feature combination and the state information of the vehicle under test, a model prompt word is generated. The model prompt word is used to guide the model to select a target path endpoint from multiple reachable path endpoints that causes the vehicle under test to fail in testing the autonomous driving algorithm. The model prompts and the map input model are subjected to multimodal processing to obtain the target path endpoint output by the large model.

7. The method according to claim 6, characterized in that, Before generating a map marked with multiple reachable path endpoints based on the road structure, the method further includes: All reachable path endpoints are filtered according to a preset test duration, and multiple reachable path endpoints within the test duration are retained. A map is generated based on the road structure, and multiple reachable path endpoints are marked on the map to obtain a map marked with multiple reachable path endpoints.

8. The method according to any one of claims 1 to 4, characterized in that, The process of determining the target path endpoint of the background vehicle based on the target feature combination, the road structure, and the state information of the vehicle under test, by instructing the large model through prompt words, and generating the driving trajectory of the background vehicle based on the target path endpoint, includes: Acquire the state information of the vehicle under test and the background vehicle in the initial scene; Based on the road structure of the initial scene and the state information of the background vehicle, the temporal trajectory sequence of the background vehicle is determined. Each temporal trajectory sequence contains multiple trajectory points with different time sequences and the real-time vehicle speed of the background vehicle at the trajectory point. Based on the target feature combination, the road structure, and the state information of the vehicle under test, the multimodal processing capability of the large model is applied to determine at least some trajectory points and real-time vehicle speed of the background vehicle according to the time-series trajectory sequence. The driving trajectory of the background vehicle is generated based on at least some of the trajectory points and the real-time vehicle speed.

9. An edge scene generation device, characterized in that, include: The acquisition module is used to acquire a dataset of scenarios where autonomous driving algorithm tests failed and an initial scenario for testing the autonomous driving algorithm. The scenario dataset contains features of multiple classes, and the initial scenario contains at least one background vehicle. The feature extraction module is used to combine different types of features based on the frequency of occurrence of features in the scene dataset to obtain a combination of target features related to test failure. The scene generation module is used to determine the reachable path endpoint of the background vehicle based on the road structure of the initial scene and the state information of the background vehicle; it is also used to instruct the large model to determine the target path endpoint of the background vehicle based on the target feature combination, the road structure, and the state information of the vehicle under test, by using prompt words, and to generate the driving trajectory of the background vehicle based on the target path endpoint, wherein the large model is a model with multimodal processing capabilities; and it is used to generate an edge scene that meets the testing requirements based on the initial scene and the driving trajectory.

10. An electronic device, characterized in that, include: A processor, and a memory communicatively connected to the processor; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory to implement the method as described in any one of claims 1 to 8.

11. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the method as described in any one of claims 1 to 8.

12. A computer program product, characterized in that, Includes a computer program that, when executed by a processor, implements the method of any one of claims 1 to 8.

Images