Scene generation method and device for automatic driving simulation test, storage medium, terminal and computer program product

Abstract

A scene generation method and device for an automatic driving simulation test, a storage medium, a terminal, and a computer program product, the method comprising: determining at least one initial scene, each initial scene comprising actual poses and actual speeds of an automatic driving vehicle and at least one dynamic obstacle at a plurality of sampling moments; determining at least one key obstacle from each initial scene; for each key obstacle, taking actual poses and actual speeds of the key obstacle, the automatic driving vehicle and other key obstacles at the target sampling moment and a first number of sampling moments before the target sampling moment as initial input data; executing trajectory prediction operation based on the initial input data to obtain a predicted trajectory of the key obstacle after the target sampling moment; and based on the predicted trajectory of each key obstacle in the initial scene, constructing and obtaining a generated test scene. According to the scheme, the generated scene containing diversified dynamic obstacle behaviors can be obtained based on the initial scene.

Description

Technical Field

[0001] The present invention relates to the field of autonomous driving technology, and in particular to a scene generation method and device, storage medium, terminal, and computer program product for autonomous driving simulation testing. Background Art

[0002] In the actual driving process, autonomous vehicles often encounter scenarios where they interact with dynamic obstacles such as pedestrians, motor vehicles, and non-motor vehicles (among which pedestrians and non-motor vehicles are vulnerable road users). Especially for unmanned sanitation vehicles (for example, unmanned street sweepers), since their main operating environment is non-motor vehicle lanes, residential areas, industrial parks, etc., the types of dynamic obstacles and their behavior types are more complex and diverse, and there is a greater probability of violating traffic rules, which leads to an increased risk of traffic accidents. Accordingly, higher requirements are placed on the safety performance of the obstacle avoidance, automatic braking, path planning, and other test algorithms or systems of autonomous vehicles.

[0003] In the prior art, there are two main testing schemes for the performance of the algorithm to be tested of an autonomous driving vehicle: one is to use fake obstacles (for example, robots) that simulate real obstacles for testing, which has a weak ability to simulate the diverse behaviors of various dynamic obstacles in the actual driving environment; the other is an autonomous driving simulation test scheme.

[0004] Conventional simulation test solutions record the surrounding driving scenes during the actual driving of the autonomous vehicle (for example, the camera records images or video data, and the laser sensor records point cloud data) and replay them, extract or sample the actual behavior data of dynamic obstacles from the recorded data (for example, the actual position, direction, speed, etc. at different sampling times), and construct a test scene for algorithm simulation testing. However, in the test scene obtained by the above method, the driving trajectory (or walking trajectory) of the dynamic obstacle is fixed and unchanged, and the types of interaction with the autonomous vehicle are still relatively limited, which cannot cover the various behaviors of dynamic obstacles in actual driving. Therefore, the test effect is also limited, and it is difficult to find loopholes or deficiencies in the algorithm. Summary of the invention

[0005] One of the purposes of an embodiment of the present invention is to provide a scenario generation for autonomous driving simulation testing, which can automatically generate test scenarios with multiple possibilities based on a given initial scenario to obtain a simulation test scenario containing diverse and complex dynamic obstacle behaviors.

[0006] To solve the above technical problems, an embodiment of the present invention provides a scene generation method for autonomous driving simulation testing, comprising the following steps: determining at least one initial scene, each initial scene including the actual position and actual speed of an autonomous driving vehicle and at least one dynamic obstacle at multiple sampling moments; for each initial scene, determining at least one key obstacle from the dynamic obstacles contained in the initial scene; for each key obstacle in the initial scene, selecting a single target sampling moment from the multiple sampling moments, and then using the actual position and actual speed of the key obstacle, the autonomous driving vehicle and the remaining key obstacles in the initial scene at the target sampling moment and a first number of sampling moments before the target sampling moment as initial input data; performing a trajectory prediction operation based on the initial input data to obtain a predicted trajectory of the key obstacle after the target sampling moment; and constructing a generated test scene corresponding to the initial scene based on the predicted trajectory of each key obstacle in the initial scene.

[0007] Optionally, the performing trajectory prediction operation is to iteratively perform multiple rounds of trajectory prediction, including: in a first round of trajectory prediction, performing trajectory prediction based on the initial input data to obtain multiple predicted trajectory segments of the key obstacle in the first round, wherein each predicted trajectory segment includes the predicted position and predicted speed of the key obstacle at a second number of future moments after the target sampling moment; in each subsequent round of trajectory prediction, selecting a single target trajectory segment from the multiple predicted trajectory segments obtained in the previous round, and selecting a single target moment from each future moment corresponding to the target trajectory segment, and then using the key obstacle at the target moment and the first moment before it to perform trajectory prediction. The predicted position and speed at a number of moments, the actual position and speed of the autonomous driving vehicle and the remaining key obstacles in the initial scene at the target moment and the first number of moments before the target moment are used as input data for the current round, and trajectory prediction is performed based on the input data to obtain multiple predicted trajectory segments of the key obstacle in the current round; until the preset iterative trajectory prediction termination condition is met, the actual position and speed of the key obstacle at the target sampling moment and the first number of sampling moments before the target moment, and the target trajectory segments selected in each round are used to form the predicted trajectory of the key obstacle after the target sampling moment.

[0008] Optionally, performing trajectory prediction based on the input data includes: iteratively performing one or more rounds of routing selection based on the input data to obtain routing information of the key obstacle; performing trajectory prediction based on the input data and the routing information to obtain the multiple predicted trajectory segments of the key obstacle.

[0009] Optionally, the input data also includes the positions of multiple lane segments in the actual driving scene to which the autonomous driving vehicle belongs and the topological relationship between the lane segments; multiple rounds of routing selection are iteratively performed based on the input data, including: in a first round of routing selection, determining a lane segment that is closest to the predicted position of the key obstacle at the target time among the multiple lane segments as the target lane segment obtained in the first round; in each subsequent round of routing selection, selecting a single lane segment from the lane segments that are sequentially located after the target lane segment obtained in the previous round according to the sequence of the lane segments in the topological relationship as the target lane segment obtained in the current round; until a preset iterative routing selection termination condition is met, using the target lane segments obtained in each round to form a lane segment sequence as the routing information of the key obstacle.

[0010] Optionally, in each round of trajectory prediction, trajectory prediction based on the input data is implemented by using a pre-trained trajectory prediction model, and the trajectory prediction model includes a route generation sub-model and a trajectory generation sub-model; wherein the route generation sub-model is used to iteratively perform one or more rounds of route selection based on the input data to obtain the routing information of the key obstacle; and the trajectory generation sub-model is used to perform trajectory prediction based on the input data and the routing information to obtain the multiple predicted trajectory segments of the key obstacle.

[0011] Optionally, the pre-trained trajectory prediction model is trained in the following manner: determining a trajectory prediction model to be trained, the trajectory prediction model to be trained comprising a route generation sub-model and a trajectory generation sub-model; obtaining a sample scene, the sample scene comprising the actual position and actual speed of a sample autonomous driving vehicle and at least one sample dynamic obstacle at multiple sample sampling moments; for each sample dynamic obstacle in the sample scene, selecting a single target sample sampling moment from the multiple sample sampling moments, and using the actual position and actual speed of the sample dynamic obstacle, the sample autonomous driving vehicle and the remaining sample dynamic obstacles in the sample scene at the target sample sampling moment and at the first sample number of sample sampling moments before the target sample sampling moment as the training data corresponding to the sample dynamic obstacle; constructing a training data set using the training data corresponding to multiple sample dynamic obstacles, and using the actual position of each sample dynamic obstacle at the second sample number of sample sampling moments after the target sample sampling moment as the target position sequence, and iteratively training the trajectory prediction model to be trained to obtain the pre-trained trajectory prediction model.

[0012] Optionally, each round of iterative training includes: inputting the training data corresponding to each sample dynamic obstacle into the route generation sub-model to obtain predicted sample routing information of each sample dynamic obstacle; inputting the training data and predicted sample routing information corresponding to each sample dynamic obstacle into the trajectory generation sub-model to obtain multiple predicted sample trajectories of the sample dynamic obstacle, wherein each predicted sample trajectory includes the sample predicted position of the sample dynamic obstacle at a second sample number of future sample moments after the target sample sampling moment, and the sample predicted positions at the second sample number of future sample moments form a single sample predicted position sequence; respectively calculating the difference values ​​between the multiple sample predicted position sequences corresponding to the multiple predicted sample trajectories and the target position sequence, and determining the minimum difference value among the difference values; and using the sum of the multiple minimum difference values ​​corresponding to the multiple sample dynamic obstacles as the loss value after the current round of iterative training.

[0013] Optionally, for each initial scene, at least one key obstacle is determined from the dynamic obstacles contained in the initial scene, including: selecting one or more first sampling moments from the multiple sampling moments; for each dynamic obstacle in the initial scene, calculating the distance between the actual position of the autonomous driving vehicle and the actual position of the dynamic obstacle at each first sampling moment; averaging the distances at each first sampling moment to obtain a first average distance corresponding to the dynamic obstacle; and selecting from the initial scene each dynamic obstacle whose first average distance is less than a first preset safety distance as the key obstacle.

[0014] Optionally, determining at least one initial scene includes: obtaining multiple candidate initial scenes, each candidate initial scene containing the actual posture and actual speed of the autonomous driving vehicle and at least one dynamic obstacle at multiple sampling moments; determining the number of types of dynamic obstacles contained in each candidate initial scene, and selecting candidate initial scenes with a number of types greater than a first preset number as preliminary screening initial scenes; for each dynamic obstacle in the preliminary screening initial scene, selecting one or more second sampling moments from the multiple sampling moments, calculating the distance between the actual position of the autonomous driving vehicle and the actual position of the dynamic obstacle at each second sampling moment, and then averaging the distances at each second sampling moment to obtain a second average distance corresponding to the dynamic obstacle; counting the number of dynamic obstacles in each preliminary screening initial scene whose second average distance is less than the second preset safety distance, recorded as the number of interactive targets; and selecting the preliminary screening initial scene with the number of interactive targets greater than the second preset number to obtain the at least one initial scene.

[0015] The embodiment of the present invention further provides a scene generation device for autonomous driving simulation testing, comprising: an initial scene determination module, used to determine at least one initial scene, each initial scene comprising the actual posture and actual speed of the autonomous driving vehicle and at least one dynamic obstacle at multiple sampling moments, wherein the posture comprises position and orientation; a key obstacle determination module, used to determine, for each initial scene, at least one key obstacle from the dynamic obstacles contained in the initial scene; an initial input data determination module, used to select, for each key obstacle in the initial scene, a single target sampling moment from the multiple sampling moments, and then use the actual posture and actual speed of the key obstacle, the autonomous driving vehicle and the remaining key obstacles in the initial scene at the target sampling moment and a first number of sampling moments before the target sampling moment as initial input data;

[0016] A trajectory prediction module is used to perform a trajectory prediction operation based on the initial input data to obtain a predicted trajectory of the key obstacle after the target sampling time; a scene construction module is used to construct a generated test scene corresponding to the initial scene based on the predicted trajectory of each key obstacle in the initial scene.

[0017] An embodiment of the present invention also provides a storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of the above-mentioned scene generation method for autonomous driving simulation testing are executed.

[0018] An embodiment of the present invention also provides a terminal, including a memory and a processor, wherein the memory stores a computer program that can be run on the processor, and when the processor runs the computer program, the steps of the above-mentioned scene generation method for autonomous driving simulation testing are executed.

[0019] An embodiment of the present invention also provides a computer program product, including a computer program, which, when executed by a processor, executes the steps of the above-mentioned scenario generation method for autonomous driving simulation testing.

[0020] Compared with the prior art, the technical solution of the embodiment of the present invention has the following beneficial effects:

[0021] In an embodiment of the present invention, for each initial scene containing a dynamic obstacle, for a key dynamic obstacle selected therein, a single target sampling moment is selected from multiple sampling moments to which actual driving data of the key dynamic obstacle belongs, and then the actual posture and actual speed of the key obstacle, the autonomous driving vehicle and the remaining key obstacles in the initial scene at the target sampling moment and a first number of sampling moments before the target sampling moment are used as initial input data; one or more rounds of iterative trajectory prediction operations are performed based on the initial input data to obtain a predicted trajectory of the key obstacle after the target sampling moment; and based on the predicted trajectories of the key obstacles in the initial scene, a generated test scene corresponding to the initial scene is constructed.

[0022] Since there are multiple possibilities for selecting the target sampling moment for each key obstacle, and there are also multiple possibilities for the predicted trajectory of the trajectory prediction operation, even for the same dynamic key obstacle, there is an opportunity to obtain a variety of predicted trajectories, and the same initial scene can contain multiple key obstacles, and there can be many combinations of predicted trajectories of different key obstacles. Therefore, for the same initial scene, there is an opportunity to obtain complex and rich test scenarios, which meets the diverse dynamic behavior requirements of autonomous driving simulation tests for dynamic obstacles other than the vehicle, and improves the reliability and effectiveness of safety verification of the algorithm to be tested. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 is a flow chart of a scene generation method for autonomous driving simulation testing in an embodiment of the present invention;

[0024] Figure 2 yes Figure 1 A flowchart of a specific implementation of step S11;

[0025] Figure 3 yes Figure 1 A flowchart of a specific implementation of step S14;

[0026] Figure 4 is a flow chart of a method for training a trajectory prediction model in an embodiment of the present invention;

[0027] Figure 5 It is a structural schematic diagram of a scene generation device for autonomous driving simulation testing in an embodiment of the present invention. DETAILED DESCRIPTION

[0028] In order to make the above-mentioned objects, features and beneficial effects of the present invention more obvious and understandable, specific embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0029] Reference Figure 1 , Figure 1 The present invention is a flowchart of a method for generating a scene for an autonomous driving simulation test in an embodiment of the present invention. The method can be applied to a terminal with a simulation test function, including but not limited to a computer, a mobile phone, a tablet computer, a vehicle-mounted terminal, a server, a cloud platform, etc.

[0030] The method may include steps S11 to S15:

[0031] Step S11: determining at least one initial scene, each initial scene including actual postures and actual speeds of the autonomous driving vehicle and at least one dynamic obstacle at multiple sampling moments, wherein the posture includes position and orientation;

[0032] Step S12: for each initial scene, determining at least one key obstacle from the dynamic obstacles included in the initial scene;

[0033] Step S13: for each key obstacle in the initial scene, a single target sampling moment is selected from the multiple sampling moments, and then the actual position and actual speed of the key obstacle, the autonomous driving vehicle and the remaining key obstacles in the initial scene at the target sampling moment and a first number of sampling moments before the target sampling moment are used as initial input data;

[0034] Step S14: performing a trajectory prediction operation based on the initial input data to obtain a predicted trajectory of the key obstacle after the target sampling time;

[0035] Step S15: Based on the predicted trajectories of the key obstacles in the initial scene, a generated test scene corresponding to the initial scene is constructed.

[0036] For ease of understanding, the relevant concepts in this application are first explained below.

[0037] The scenario of autonomous driving simulation test can specifically refer to an overall dynamic description of the comprehensive interaction process between the autonomous driving vehicle and other dynamic obstacles (for example, motor vehicles, non-motor vehicles, pedestrians, animals, etc.), static environmental elements (for example, lanes, traffic facilities, weather conditions, etc.), and static obstacles (for example, trash cans, buildings, public entertainment facilities, vehicles parked on the roadside, etc.) within a certain time and space range.

[0038] The initial scene (also referred to as a "seed scene") can be a test scene constructed by recording the actual driving environment around the autonomous vehicle during actual driving (for example, using a camera to record scene images or video data, or a laser sensor to record scene point cloud data) and playing back the scene, and extracting or sampling the required data from the recorded data. In the present application, the initial scene at least includes the actual driving data of the autonomous driving vehicle extracted or sampled, and the actual driving data of at least one dynamic obstacle, and the actual driving data includes the actual posture and actual speed at multiple sampling times, and the actual posture may include position and orientation.

[0039] Furthermore, in combination with actual application requirements, the initial scene may also include other appropriate data, such as the type identification of dynamic obstacles, the type identification of autonomous driving vehicles, etc., and may also include static environmental element data and static obstacle data. The static environmental element data may include, but is not limited to: the coordinates of multiple edge points constituting the lane line, the positions, lane segment identifications, topological relationships, etc. of multiple lane segments divided by the lane line, the positions and features of traffic facilities (for example, "0" represents a red light, "1" represents a green light), and meteorological condition identifications (such as sunny, cloudy, rainy, etc.). The static obstacle data may include, but is not limited to, the type identification, position, contour information, etc. of static obstacles.

[0040] Among them, the actual driving environment of the autonomous driving vehicle may include but is not limited to residential areas, parking lots, parks, and urban / suburban roads. For autonomous driving sanitation vehicles (for example, unmanned road sweepers), the main driving scenarios usually include industrial parks, residential areas, non-motorized vehicle lanes, etc. The characteristics of the aforementioned environments are that there are many vulnerable road users such as pedestrians and non-motorized vehicles, and the interaction with autonomous driving vehicles is complex and diverse. Therefore, the safety performance requirements for the relevant algorithms to be tested for autonomous driving vehicles are more stringent, and accordingly, the complexity and diversity of the test scenarios for the algorithms to be tested are also higher, especially for the diversity and complexity of the behavior data of the interactive targets contained in the test scenarios (mainly referring to: dynamic obstacles that interact with autonomous driving vehicles).

[0041] In the specific implementation of step S11, the at least one initial scene may be a scene screened from a plurality of candidate initial scenes. For example, candidate scenes containing a large number of dynamic obstacle types and / or numbers may be screened; for another example, candidate scenes containing a large number of dynamic obstacles that have interaction risks or potential interaction risks with the autonomous driving vehicle may be screened.

[0042] In a specific implementation, each initial scene may include, in addition to the actual position and actual speed of the autonomous driving vehicle and at least one dynamic obstacle at multiple sampling times, other appropriate data required for subsequent trajectory prediction and construction of a generated test scene, such as the aforementioned static environment element data and static obstacle data.

[0043] Regarding a non-limiting implementation scheme for screening the initial scene, refer to Figure 2 and the detailed description below.

[0044] Reference Figure 2 , Figure 2 yes Figure 1 A flowchart of a specific implementation of step S11 in the embodiment. In this implementation, the determining of at least one initial scene may specifically include steps S111 to S115.

[0045] In step S111, a plurality of candidate initial scenes are obtained, each candidate initial scene including the actual position and actual speed of the autonomous driving vehicle and at least one dynamic obstacle at a plurality of sampling moments.

[0046] In step S112, the number of types of dynamic obstacles contained in each candidate initial scene is determined, and the candidate initial scenes whose number of types is greater than a first preset number are selected as the initial screening scenes.

[0047] In step S113, for each dynamic obstacle in the initial screening scene, one or more second sampling moments are selected from the multiple sampling moments, and the distance between the actual position of the automatic driving vehicle and the actual position of the dynamic obstacle at each second sampling moment is calculated, and then the distances at each second sampling moment are averaged to obtain a second average distance corresponding to the dynamic obstacle.

[0048] In step S114, the number of dynamic obstacles whose second average distance is less than the second preset safety distance in each initial screening scene is counted and recorded as the number of interactive targets.

[0049] In step S115, a pre-screened initial scenario with a number of interactive targets greater than a second preset number is selected to obtain the at least one initial scenario.

[0050] It is understandable that the number of types of dynamic obstacles and the second average distance can characterize the interaction complexity or interaction risk between the candidate initial scene and the autonomous driving vehicle to a certain extent. Generally speaking, the more types of dynamic obstacles and / or the smaller the second average distance, the more types and times of interaction behaviors (such interaction behaviors may include, for example, slowing down to give way, overtaking, parking to avoid obstacles, etc.) between the dynamic obstacles in the candidate initial scene and the autonomous driving vehicle, and the higher the safety risk. Therefore, the embodiment of the present invention can improve the complexity and interaction risk (or antagonism) of the final generated scene by adopting the above-mentioned scheme to screen and obtain the initial scene, which is more helpful to test and evaluate the security and reliability of the algorithm to be tested during the simulation test, and the easier it is to find algorithm loopholes or problems.

[0051] Continue to refer to Figure 1 In the specific implementation of step S12, for each initial scene, at least one key obstacle is determined from the dynamic obstacles contained in the initial scene.

[0052] In a specific implementation, dynamic obstacles whose driving trajectories are not interrupted (ie, data integrity is guaranteed) may be screened from the initial scene as key obstacles.

[0053] In another specific implementation, a specific type of dynamic obstacle can be selected as the key obstacle in combination with the type of autonomous driving vehicle and the characteristics of its conventional operating environment. The types of dynamic obstacles may include, but are not limited to, motor vehicles, non-motor vehicles, pedestrians, etc. For example, for an autonomous driving sanitation vehicle whose conventional operating environment is an industrial park or residential area, dynamic obstacles of the types of pedestrians and non-motor vehicles are selected as key obstacles; for another example, for an autonomous driving vehicle whose conventional operating environment is an elevated road or highway, dynamic obstacles of the type of motor vehicles are selected as key obstacles.

[0054] In another specific implementation, dynamic obstacles with higher interaction risks may be screened as key obstacles, specifically: one or more first sampling moments are selected from the multiple sampling moments; for each dynamic obstacle in the initial scene, the distance between the actual position of the autonomous driving vehicle and the actual position of the dynamic obstacle at each first sampling moment is calculated; the distances at each first sampling moment are averaged to obtain a first average distance corresponding to the dynamic obstacle; and each dynamic obstacle whose first average distance is less than the first preset safety distance is selected from the initial scene as the key obstacle.

[0055] The selection, setting and related technical principles of the one or more first sampling moments and the first preset safety distance can refer to the above Figure 2 The relevant contents of steps S113 and S114 are not repeated here.

[0056] It should be pointed out that the specific method for determining key obstacles is not limited to the above scheme. In specific implementation, other appropriate schemes can be used to determine the key obstacles in each initial scene in combination with actual application requirements.

[0057] In the specific implementation of step S13, for different key obstacles in the initial scene, the single target sampling moments selected from the multiple sampling moments may be the same or different. The selection method may be, for example, random selection or selection using a preset sequence number; wherein the random selection is randomly selected from each sampling moment after the first number, and the preset sequence number is greater than the first number.

[0058] For example, the multiple sampling moments are t1, t2, t3, t4, t5, ... n , the first number is set to 2 (only as a non-limiting example), for a key obstacle A in the initial scene, the selected target sampling time is t4, then the initial input data includes: the actual position and actual speed of the key obstacle A at the three sampling times t2, t3, and t4, the actual position and actual speed of the autonomous driving vehicle at t2, t3, and t4, and the actual position and actual speed of the remaining key obstacles in the initial scene at t2, t3, and t4.

[0059] Furthermore, the initial input data may also include other appropriate data required for subsequent trajectory prediction operations, for example, static environment element data and static obstacle data included in the initial scene.

[0060] In a specific implementation of step S14, a trajectory prediction operation is performed based on the initial input data to obtain a predicted trajectory of the key obstacle after the target sampling time.

[0061] In a specific implementation, an existing conventional trajectory prediction algorithm can be used, the initial input data is used as the algorithm input data and a single-step trajectory prediction is performed, and a single predicted trajectory or multiple predicted trajectories can be predicted; then the predicted single trajectory is used directly as the predicted trajectory of the key obstacle after the target sampling moment; or, from the multiple predicted trajectories obtained by the single-step trajectory prediction, a predicted trajectory is randomly selected or selected using preset conditions as the predicted trajectory of the key obstacle after the target sampling moment. In another specific implementation, an iterative scheme can be used to implement trajectory prediction, specifically in combination with Figure 3 and the description below.

[0062] Reference Figure 3 , Figure 3 yes Figure 1Flow chart of a specific implementation of step S14 in the embodiment. In this implementation, step S14 may include S141 to S143.

[0063] Step S141: In the first round of trajectory prediction, trajectory prediction is performed based on the initial input data to obtain multiple predicted trajectory segments of the key obstacle in the first round, wherein each predicted trajectory segment includes a predicted position and predicted speed of the key obstacle at a second number of future moments after the target sampling moment.

[0064] In a specific implementation, the second number can be appropriately set according to actual needs, and its value can be the same as or different from the first number. As a non-limiting example, the second number is set to 5. Continuing with the above embodiment, each predicted trajectory segment can include the predicted position and predicted speed of the key obstacle A at 5 future moments after t4 (i.e., t5, t6, t7, t8, t9).

[0065] Step S142: In each subsequent round of trajectory prediction, a single target trajectory segment is selected from the multiple predicted trajectory segments obtained in the previous round, and a single target moment is selected from each future moment corresponding to the target trajectory segment, and then the predicted position and predicted speed of the key obstacle at the target moment and a first number of moments before it, and the actual position and actual speed of the autonomous driving vehicle and the remaining key obstacles in the initial scene at the target moment and a first number of moments before it are used as input data for the current round, and trajectory prediction is performed based on the input data to obtain multiple predicted trajectory segments of the key obstacle in the current round.

[0066] Among them, the method of selecting a single target trajectory segment from multiple predicted trajectory segments obtained in the previous round can be random selection to maximize the possibility and diversity of the final predicted trajectory; or, it can be selected according to preset screening conditions to increase the predictability or controllability of the final predicted trajectory.

[0067] For more details about determining the input data, please refer to the above content about determining the initial input data, which will not be repeated here.

[0068] Step S143: until the preset iterative trajectory prediction termination condition is met, the actual position and actual speed of the key obstacle at the target sampling time and the first number of sampling times before it, as well as the target trajectory segments selected in each round, are used to form a predicted trajectory of the key obstacle after the target sampling time.

[0069] The iterative trajectory prediction termination condition may be a preset number of trajectory prediction iterations, or a preset total number of predicted poses or predicted velocities, or other appropriate termination conditions.

[0070] Furthermore, since the second number of future moments predicted in the previous round may overlap with the second number of future moments predicted in the current round, for example, the predicted poses and predicted velocities of t5, t6, t7, t8, and t9 are predicted in the previous round, and the predicted velocities of t8, t9, and t10 are predicted in the current round, the predicted poses and predicted velocities of t11, t12, t13, and t14 are predicted in the current round. 10 ,t 11 ,t 12 The predicted position and speed of t8 and t9 are predicted in both the previous round and the current round. For the overlapping moments in the target trajectory segments selected in two adjacent rounds, the application can use the predicted position and speed of each overlapping moment obtained by the later round prediction.

[0071] Compared with the single-step trajectory prediction scheme to obtain the predicted trajectory of each key obstacle, in the embodiment of the present invention, by adopting the above-mentioned multi-round iterative trajectory prediction scheme, since in each round of iteration, there are multiple possible results for selecting a single target trajectory segment from multiple predicted trajectory segments obtained by the previous round of prediction, and there are also multiple possible results for selecting a single target moment from each future moment corresponding to the target trajectory segment, therefore, the diversity of the predicted trajectory of each key obstacle finally obtained can be greatly improved, and there can be multiple key obstacles in each initial scene, and there can be multiple combinations of various predicted trajectories of different key obstacles. Therefore, complex and diverse generation test scenarios can be generated based on a single initial scene.

[0072] Furthermore, in the above-mentioned iterative trajectory prediction scheme, performing trajectory prediction based on the input data may specifically include an iterative routing selection step and a trajectory prediction step: iteratively performing one or more rounds of routing selection based on the input data to obtain routing information of the key obstacle; performing trajectory prediction based on the input data and the routing information to obtain the multiple predicted trajectory segments of the key obstacle.

[0073] Furthermore, the input data also includes the positions of multiple lane segments in the actual driving scene to which the autonomous driving vehicle belongs and the topological relationship between the lane segments, and multiple rounds of routing selection are iteratively performed based on the input data, including: in a first round of routing selection, determining a lane segment that is closest to the predicted position of the key obstacle at the target time among the multiple lane segments as the target lane segment obtained in the first round; in each subsequent round of routing selection, selecting a single lane segment from the lane segments that are sequentially located after the target lane segment obtained in the previous round according to the sequence of the lane segments in the topological relationship as the target lane segment obtained in the current round; until a preset iterative routing selection termination condition is met, using the target lane segments obtained in each round to form a lane segment sequence as the routing information of the key obstacle.

[0074] The multiple lane segments may be obtained by segmenting one or more lanes in the actual driving scene to which the autonomous driving vehicle belongs. The topological relationship between the lane segments may specifically include: adjacency relationship, association relationship, etc. between the lane segments. The topological relationship may indicate the sequence between the lane segments. For example, lane segment A is followed by lane segments B, C, and D, and lane segment C is followed by lane segments E and F. The sequence between the lane segments may be determined based on the sequence of geographical locations of the lane segments in the actual driving environment of the autonomous driving vehicle, or the sequence of the lane segments that the autonomous driving vehicle passes through.

[0075] The routing termination condition may be a preset number of routing iterations, a preset total number of target lane segments, or other appropriate termination conditions.

[0076] In a specific implementation, in each round of trajectory prediction, performing trajectory prediction based on the input data may be implemented using a pre-trained trajectory prediction model, wherein the trajectory prediction model includes a route generation sub-model and a trajectory generation sub-model; wherein the route generation sub-model is used to iteratively perform one or more rounds of route selection based on the input data to obtain the route information of the key obstacle; and the trajectory generation sub-model is used to perform trajectory prediction based on the input data and the routing information to obtain the multiple predicted trajectory segments of the key obstacle.

[0077] In the embodiment of the present invention, compared with directly performing trajectory prediction on input data using an existing conventional trajectory prediction algorithm or a single trajectory prediction model (i.e., a model that does not include a routing selection operation), this embodiment introduces an iterative routing selection step in each trajectory prediction (i.e., adding a routing generation sub-model to the trajectory prediction model). On the one hand, it can provide global routing planning information for trajectory prediction, reduce the randomness of the prediction, and improve the controllability of the prediction result; on the other hand, compared with a single-round routing selection, a multi-round iterative routing selection scheme can be used to increase the diversity of the generated routing information and further obtain a variety of predicted trajectories.

[0078] The following combination Figure 4 The scheme for training to obtain the pre-trained trajectory prediction model is described in detail.

[0079] Reference Figure 4 , Figure 4 4 is a flow chart of a method for training a trajectory prediction model in an embodiment of the present invention. The training method may include the following steps S41 to S44.

[0080] In step S41, a trajectory prediction model to be trained is determined, wherein the trajectory prediction model to be trained includes a route generation sub-model and a trajectory generation sub-model.

[0081] In a specific implementation, the structures of the route generation sub-model and the trajectory generation sub-model can be selected from but not limited to a recurrent neural network structure (RNN), a long short-term memory network structure (LSTM), a one-dimensional convolutional neural network structure (1D-CNN), and the like.

[0082] In step S42, a sample scene is obtained, where the sample scene includes the actual position and actual speed of a sample autonomous driving vehicle and at least one sample dynamic obstacle at multiple sample sampling moments.

[0083] Regarding the method for obtaining the sample scene, you can refer to the method for obtaining the initial scene mentioned above, which will not be repeated here.

[0084] In step S43, for each sample dynamic obstacle in the sample scene, a single target sample sampling moment is selected from the multiple sample sampling moments, and the actual posture and actual speed of the sample dynamic obstacle, the sample autonomous driving vehicle and the remaining sample dynamic obstacles in the sample scene at the target sample sampling moment and the first sample number of sample sampling moments before the target sample sampling moment are used as training data corresponding to the sample dynamic obstacle.

[0085] In step S44, a training data set is constructed using training data corresponding to a plurality of sample dynamic obstacles, and the actual position of each sample dynamic obstacle at a second number of sample sampling moments after the target sample sampling moment is used as a target position sequence to iteratively train the trajectory prediction model to be trained, so as to obtain the pre-trained trajectory prediction model.

[0086] Furthermore, in step S44, each round of iterative training specifically includes the following steps (1) to (4).

[0087] Step (1): input the training data corresponding to each sample dynamic obstacle into the route generation sub-model to obtain the predicted sample route information of each sample dynamic obstacle.

[0088] Step (2): input the training data and predicted sample routing information corresponding to each sample dynamic obstacle into the trajectory generation sub-model to obtain multiple predicted sample trajectories of the sample dynamic obstacle, wherein each predicted sample trajectory includes the sample predicted position of the sample dynamic obstacle at the second sample number of future sample moments after the target sample sampling moment, and the sample predicted positions of the second sample number of future sample moments form a single sample predicted position sequence.

[0089] Step (3): respectively calculating the difference values ​​between the plurality of sample prediction position sequences corresponding to the plurality of prediction sample trajectories and the target position sequence (ie, the label data for training), and determining the minimum difference value among the difference values.

[0090] Specifically, the difference value can be calculated by a preset loss function, for example, a mean square error loss function, a cross entropy loss function, an L1 loss function, etc.

[0091] Step (4): using the sum of multiple minimum difference values ​​corresponding to the multiple sample dynamic obstacles as the loss value after the current round of iterative training (ie, the function value of the loss function).

[0092] During the iterative training process, the parameters of the route generation sub-model and the trajectory generation sub-model are iteratively optimized to minimize the loss value until a preset number of iterative training rounds or a convergence value of the loss value is reached, and the iteration is stopped to obtain the pre-trained trajectory prediction model.

[0093] In an embodiment of the present invention, in each round of iterative training, the actual position of each sample dynamic obstacle at the sampling moment of the second sample number of samples is used as the target position sequence, and the minimum difference value between the multiple sample predicted position sequences and the target position sequence predicted by the trajectory generation sub-model in the current round is used as the loss value of the current round to iteratively optimize the parameters of the model. In this way, on the one hand, the trajectory predicted by the model can be constrained by the actual driving trajectory of the sample dynamic obstacle, ensuring the authenticity and feasibility of the trajectory predicted by the model, and avoiding the problem of predicting too random data and failing to form a reasonable and feasible trajectory. On the other hand, compared to determining the loss value by comprehensively considering multiple difference values ​​between the multiple sample predicted position sequences and the target position sequence, this training scheme only uses the minimum difference value among the difference values ​​as the loss value, which can achieve the diversity of the model-predicted trajectory.

[0094] Continue to refer to Figure 1 In the specific implementation of step S15, based on the predicted trajectories of the key obstacles in the initial scene, a generated test scene corresponding to the initial scene is constructed.

[0095] Further, in Figure 1In the illustrated embodiment, for each key obstacle in the initial scene, the step of determining the initial input data (i.e., step S13) and the step of performing the trajectory prediction operation based on the initial input data (i.e., step S14) may be performed multiple times until a preset termination condition is met (e.g., a preset number of executions is reached), and then the step of constructing and generating the test scene (i.e., step S15) is performed, so that multiple predicted trajectories can be obtained for a single key obstacle. For each key obstacle, during each execution of step S13, the target sampling moment selected from the multiple sampling moments may be the same or different.

[0096] Furthermore, the predicted trajectories corresponding to different key obstacles in each initial scene can be randomly combined after trajectory screening, so that multiple possible generated test scenarios can be constructed for a single initial scene.

[0097] For example, there are three key obstacles in an initial scene, namely key obstacle A, key obstacle B, and key obstacle C. Key obstacle A corresponds to predicted trajectories a1, a2, a3, and a4, key obstacle B corresponds to predicted trajectories b1, b2, and b3, and key obstacle C corresponds to predicted trajectories c1 and c2. There are 4×3×2=24 combinations of predicted trajectories corresponding to these three key obstacles, that is, the initial scene has the opportunity to obtain dozens or even more generated test scenes.

[0098] Furthermore, after obtaining the generated test scenario, the generated test scenario can be used to perform a simulation test on the algorithm to be tested of the autonomous driving vehicle; then, the simulation test results are evaluated using preset simulation test evaluation indicators; and the generated test scenarios that do not meet expectations are eliminated based on the evaluation results.

[0099] Furthermore, after obtaining a plurality of generated test scenarios, a risk scenario can be selected therefrom, and the risk scenario can be used to perform simulation testing on the algorithm to be tested of the autonomous driving vehicle. Without limitation, the risk scenario can be, for example, a generated test scenario in which the distance between the actual position of the autonomous driving vehicle and the predicted position of at least one key obstacle at at least one moment is less than a preset distance threshold. Alternatively, other appropriate methods can be used to select risk scenarios to increase the risk of interaction between dynamic obstacles and autonomous driving vehicles during simulation testing, and to obtain more effective and reliable simulation testing results.

[0100] Reference Figure 5 , Figure 5 1 is a schematic diagram of a scene generation device for autonomous driving simulation testing in an embodiment of the present invention. The device may include:

[0101] An initial scene determination module 51 is used to determine at least one initial scene, each initial scene including actual postures and actual speeds of the autonomous driving vehicle and at least one dynamic obstacle at multiple sampling moments, wherein the posture includes position and orientation;

[0102] A key obstacle determination module 52, configured to determine, for each initial scene, at least one key obstacle from the dynamic obstacles included in the initial scene;

[0103] an initial input data determination module 53, configured to select, for each key obstacle in the initial scene, a single target sampling moment from the multiple sampling moments, and then use the actual position and actual speed of the key obstacle, the autonomous driving vehicle, and the remaining key obstacles in the initial scene at the target sampling moment and a first number of sampling moments before the target sampling moment as initial input data;

[0104] A trajectory prediction module 54, configured to perform a trajectory prediction operation based on the initial input data to obtain a predicted trajectory of the key obstacle after the target sampling time;

[0105] The scenario construction module 55 is used to construct a generated test scenario corresponding to the initial scenario based on the predicted trajectories of the key obstacles in the initial scenario.

[0106] For the principle, specific implementation and beneficial effects of the scene generation device for autonomous driving simulation test, please refer to the previous article and Figures 1 to 4 The relevant description of the scene generation method for autonomous driving simulation testing shown in any embodiment will not be repeated here.

[0107] The embodiment of the present invention further provides a storage medium, such as a computer-readable storage medium, on which a computer program is stored, and the computer program is executed by a processor. Figures 1 to 4 The steps of the scene generation method for autonomous driving simulation test shown in any embodiment. The computer-readable storage medium may include a non-volatile memory or a non-transitory memory, and may also include an optical disk, a mechanical hard disk, a solid-state hard disk, etc.

[0108] Specifically, in the embodiment of the present invention, the processor may be a central processing unit (CPU), and the processor may also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0109] It should also be understood that the memory in the embodiments of the present application may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example but not limitation, many forms of random access memory (RAM) are available, such as static RAM (SRAM), dynamic random access memory (DRAM), synchronous DRAM (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link DRAM (SLDRAM) and direct rambus RAM (DR RAM).

[0110] The embodiment of the present invention further provides a terminal, including a memory and a processor, wherein the memory stores a computer program that can be run on the processor, and the processor executes the above-mentioned Figures 1 to 4Any of the embodiments shows the steps of a scenario generation method for autonomous driving simulation testing.

[0111] The embodiment of the present invention further provides a computer program product, including a computer program, wherein the computer program is executed by a processor Figures 1 to 4 Any of the embodiments shows the steps of a scenario generation method for autonomous driving simulation testing.

[0112] It should be understood that the term "and / or" in this article is only a description of the association relationship of the associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article indicates that the associated objects before and after are in an "or" relationship.

[0113] The "plurality" appearing in the embodiments of the present application refers to two or more.

[0114] The first, second, etc. descriptions appearing in the embodiments of the present application are only used for illustration and distinction of the description objects. There is no order, nor do they indicate any special limitation on the number of devices in the embodiments of the present application, and cannot constitute any limitation on the embodiments of the present application.

[0115] It should be pointed out that the serial numbers of the steps in this embodiment do not limit the execution order of the steps.

[0116] Although the present invention is disclosed as above, the present invention is not limited thereto. Any person skilled in the art can make various changes and modifications without departing from the spirit and scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the scope defined by the claims.

Claims

1. A scenario generation method for autonomous driving simulation testing, characterized in that: include: Determine at least one initial scene, each initial scene including actual poses and actual velocities of the autonomous driving vehicle and at least one dynamic obstacle at multiple sampling moments, wherein the poses include position and orientation; For each initial scene, determining at least one key obstacle from the dynamic obstacles included in the initial scene; For each key obstacle in the initial scene, select a single target sampling moment from the multiple sampling moments, and then use the actual position and actual speed of the key obstacle, the autonomous driving vehicle, and the remaining key obstacles in the initial scene at the target sampling moment and a first number of sampling moments before the target sampling moment as initial input data; Performing a trajectory prediction operation based on the initial input data to obtain a predicted trajectory of the key obstacle after the target sampling time; Based on the predicted trajectories of the key obstacles in the initial scene, a generated test scene corresponding to the initial scene is constructed.

2. The method according to claim 1, characterized in that The trajectory prediction operation is to iteratively perform multiple rounds of trajectory prediction, including: In a first round of trajectory prediction, trajectory prediction is performed based on the initial input data to obtain a plurality of predicted trajectory segments of the key obstacle in the first round, wherein each predicted trajectory segment includes a predicted position and predicted velocity of the key obstacle at a second number of future time points after the target sampling time point; In each subsequent round of trajectory prediction, a single target trajectory segment is selected from the multiple predicted trajectory segments obtained in the previous round, and a single target moment is selected from each future moment corresponding to the target trajectory segment. Then, the predicted position and predicted speed of the key obstacle at the target moment and a first number of moments before the target moment, and the actual position and actual speed of the autonomous driving vehicle and the remaining key obstacles in the initial scene at the target moment and a first number of moments before the target moment are used as input data for the current round, and trajectory prediction is performed based on the input data to obtain multiple predicted trajectory segments for the key obstacle in the current round. Until the preset iterative trajectory prediction termination condition is met, the actual position and actual velocity of the key obstacle at the target sampling time and the first number of sampling times before it, as well as the target trajectory segments selected in each round, are used to form the predicted trajectory of the key obstacle after the target sampling time.

3. The method according to claim 2, characterized in that Performing trajectory prediction based on the input data includes: Iteratively performing one or more rounds of routing selection based on the input data to obtain routing information of the key obstacle; Trajectory prediction is performed based on the input data and the routing information to obtain the multiple predicted trajectory segments of the key obstacle.

4. The method according to claim 3, characterized in that The input data further includes the positions of multiple lane segments in the actual driving scene of the autonomous driving vehicle and the topological relationship between the lane segments; Iteratively performing multiple rounds of routing based on the input data, including: In a first round of routing selection, a lane segment closest to the predicted position of the key obstacle at the target time is determined from among the multiple lane segments as the target lane segment obtained in the first round; in each subsequent round of routing selection, a single lane segment is selected from among the lane segments that are sequentially located after the target lane segment obtained in the previous round based on the order of the lane segments in the topological relationship as the target lane segment obtained in the current round; Until the preset iterative routing termination condition is met, the target lane segments obtained in each round are used to form a lane segment sequence as the routing information of the key obstacle.

5. The method according to claim 3 or 4, characterized in that In each round of trajectory prediction, trajectory prediction is performed based on the input data using a pre-trained trajectory prediction model, wherein the trajectory prediction model includes a route generation sub-model and a trajectory generation sub-model; The routing generation sub-model is used to iteratively perform one or more rounds of routing selection based on the input data to obtain routing information of the key obstacle; The trajectory generation sub-model is used to perform trajectory prediction based on the input data and the routing information to obtain the multiple predicted trajectory segments of the key obstacle.

6. The method according to claim 5, characterized in that The pre-trained trajectory prediction model is trained in the following way: Determining a trajectory prediction model to be trained, wherein the trajectory prediction model to be trained includes a route generation sub-model and a trajectory generation sub-model; Acquire a sample scene, wherein the sample scene includes actual positions and actual velocities of a sample autonomous driving vehicle and at least one sample dynamic obstacle at multiple sample sampling moments; For each sample dynamic obstacle in the sample scene, a single target sample sampling moment is selected from the multiple sample sampling moments, and the actual position and actual speed of the sample dynamic obstacle, the sample autonomous driving vehicle, and the remaining sample dynamic obstacles in the sample scene at the target sample sampling moment and at the first sample number of sample sampling moments before the target sample sampling moment are used as the training data corresponding to the sample dynamic obstacle; A training data set is constructed using training data corresponding to multiple sample dynamic obstacles, and the actual position of each sample dynamic obstacle at a second sample number of sample sampling moments after the target sample sampling moment is used as a target position sequence. The trajectory prediction model to be trained is iteratively trained to obtain the pre-trained trajectory prediction model.

7. The method according to claim 6, characterized in that Each round of iterative training includes: Inputting the training data corresponding to each sample dynamic obstacle into the route generation sub-model to obtain predicted sample route information of each sample dynamic obstacle; Inputting the training data and predicted sample routing information corresponding to each sample dynamic obstacle into the trajectory generation sub-model to obtain multiple predicted sample trajectories of the sample dynamic obstacle, wherein each predicted sample trajectory includes a sample predicted position of the sample dynamic obstacle at a second sample number of future sample time points after the target sample sampling time point, and the sample predicted positions at the second sample number of future sample time points form a single sample predicted position sequence; Calculating the difference values ​​between the plurality of sample prediction position sequences corresponding to the plurality of prediction sample trajectories and the target position sequence, and determining the minimum difference value among the difference values; The sum of the multiple minimum difference values ​​corresponding to the multiple sample dynamic obstacles is used as the loss value after the current round of iterative training.

8. The method according to claim 1, characterized in that For each initial scene, at least one key obstacle is determined from the dynamic obstacles contained in the initial scene, including: Select one or more first sampling moments from the multiple sampling moments; For each dynamic obstacle in the initial scene, calculating the distance between the actual position of the autonomous driving vehicle and the actual position of the dynamic obstacle at each first sampling moment; The distances at the first sampling moments are averaged to obtain a first average distance corresponding to the dynamic obstacle; and each dynamic obstacle whose first average distance is less than the first preset safety distance is selected from the initial scene as the key obstacle.

9. The method according to claim 1, characterized in that The determining of at least one initial scenario includes: Acquire multiple candidate initial scenes, each candidate initial scene including actual positions and actual velocities of the autonomous driving vehicle and at least one dynamic obstacle at multiple sampling moments; Determining the number of types of dynamic obstacles contained in each candidate initial scene, and selecting a candidate initial scene with a number of types greater than a first preset number as a preliminary screening initial scene; For each dynamic obstacle in the pre-screened initial scene, selecting one or more second sampling moments from the multiple sampling moments, calculating the distance between the actual position of the autonomous driving vehicle and the actual position of the dynamic obstacle at each second sampling moment, and then averaging the distances at each second sampling moment to obtain a second average distance corresponding to the dynamic obstacle; Count the number of dynamic obstacles whose second average distance is less than the second preset safety distance in each initial screening scene, and record it as the number of interactive targets; Selecting preliminary screening initial scenarios in which the number of interactive targets is greater than a second preset number to obtain the at least one initial scenario.

10. A scene generation device for autonomous driving simulation testing, characterized in that: include: An initial scene determination module is configured to determine at least one initial scene, each initial scene including the actual pose and actual speed of the autonomous driving vehicle and at least one dynamic obstacle at multiple sampling moments, wherein the pose includes position and orientation; a key obstacle determination module, configured to determine, for each initial scene, at least one key obstacle from among the dynamic obstacles contained in the initial scene; an initial input data determination module configured to, for each key obstacle in the initial scene, select a single target sampling moment from the multiple sampling moments, and then use the actual position and actual velocity of the key obstacle, the autonomous driving vehicle, and the remaining key obstacles in the initial scene at the target sampling moment and a first number of sampling moments before the target sampling moment as initial input data; a trajectory prediction module, configured to perform a trajectory prediction operation based on the initial input data to obtain a predicted trajectory of the key obstacle after the target sampling time; The scenario construction module is used to construct a generated test scenario corresponding to the initial scenario based on the predicted trajectory of each key obstacle in the initial scenario.

11. A storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the scenario generation method for autonomous driving simulation testing according to any one of claims 1 to 9 are executed.

12. A terminal comprising a memory and a processor, wherein the memory stores a computer program that can be run on the processor, characterized in that: When the processor runs the computer program, the processor performs the steps of the scenario generation method for autonomous driving simulation testing as described in any one of claims 1 to 9.

13. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the scenario generation method for autonomous driving simulation testing according to any one of claims 1 to 9 are executed.

Images