Scene generation method and apparatus, and electronic device

Abstract

The present disclosure provides a scene generation method and device and electronic equipment, relates to the technical field of artificial intelligence, and in particular to the technical field of automatic driving. The specific implementation scheme is: determining a first scene, wherein in the first scene, a target vehicle moves according to a first trajectory, and an obstacle moves according to a second trajectory; dividing the second trajectory into a plurality of discrete path segments; determining displacement offset amounts respectively offset from the plurality of path segments in displacement, to obtain a total sum of the displacement offset amounts offset from the second trajectory; determining velocity offset amounts respectively offset from the plurality of path segments in velocity, to obtain a total sum of the velocity offset amounts offset from the second trajectory; obtaining a third trajectory based on the second trajectory, the total sum of the displacement offset amounts, and the total sum of the velocity offset amounts; and generating a second scene based on the first trajectory and the third trajectory. Through the present disclosure, a large number of accurate test scenes can be efficiently generated on the basis of cost control.

Description

Technical Field

[0001] This disclosure relates to the field of artificial intelligence technology, and more particularly to a scene generation method, apparatus, and electronic device in the field of autonomous driving. Background Technology

[0002] In autonomous driving technology, safety assessment of autonomous vehicles is a crucial step. This assessment requires testing and evaluating autonomous vehicles across a variety of possible safety scenarios. The diversity of these scenarios accurately reflects real-world driving conditions, enabling a more precise evaluation of the safety of autonomous vehicles.

[0003] However, in related technologies, the scenarios collected for testing and evaluation are obtained from real-world everyday scenarios, but in a large number of real-world road conditions, scenarios where safety issues (e.g., collisions) occur are extremely rare. Summary of the Invention

[0004] This disclosure provides a scene generation method, apparatus, electronic device, non-transitory computer-readable storage medium storing computer instructions, and computer program product.

[0005] According to one aspect of this disclosure, a scene generation method is provided, the method comprising: determining a first scene, wherein a target vehicle moves along a first trajectory and an obstacle moves along a second trajectory; dividing the second trajectory into discrete multiple path segments; determining displacement offsets from the multiple path segments respectively, obtaining a sum of displacement offsets from the second trajectory; determining velocity offsets from the multiple path segments respectively, obtaining a sum of velocity offsets from the second trajectory; obtaining a third trajectory based on the second trajectory, the sum of displacement offsets, and the sum of velocity offsets; and generating a second scene based on the first trajectory and the third trajectory.

[0006] According to another aspect of this disclosure, a scene generation apparatus is provided, comprising: a first determining module for determining a first scene, wherein a target vehicle moves along a first trajectory and obstacles move along a second trajectory in the first scene; a dividing module for dividing the second trajectory into a plurality of discrete path segments; a second determining module for determining displacement offsets from the plurality of path segments respectively, thereby obtaining a sum of displacement offsets from the second trajectory; a third determining module for determining velocity offsets from the plurality of path segments respectively, thereby obtaining a sum of velocity offsets from the second trajectory; a processing module for obtaining a third trajectory based on the second trajectory, the sum of displacement offsets, and the sum of velocity offsets; and a generation module for generating a second scene based on the first trajectory and the third trajectory.

[0007] According to another aspect of this disclosure, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the method described in any of the preceding claims.

[0008] According to another aspect of this disclosure, a non-transitory computer-readable storage medium is provided storing computer instructions, wherein the computer instructions are configured to cause the computer to perform the method according to any one of the preceding claims.

[0009] According to another aspect of this disclosure, a computer program product is provided, including a computer program that, when executed by a processor, implements the method according to any of the preceding claims.

[0010] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0011] The accompanying drawings are provided to better understand this solution and do not constitute a limitation of this disclosure. Wherein:

[0012] Figure 1 This is a flowchart of a scene generation method provided according to an embodiment of the present disclosure;

[0013] Figure 2 This is a schematic diagram of the structure of an autonomous driving system provided according to an embodiment of the present disclosure;

[0014] Figure 3 This is a schematic diagram of a vehicle steering model provided according to an embodiment of the present disclosure;

[0015] Figure 4 This is a structural block diagram of a scene generation apparatus provided according to an embodiment of the present disclosure;

[0016] Figure 5 A schematic block diagram of an example electronic device 500 that can be used to implement embodiments of the present disclosure is shown. Detailed Implementation

[0017] The exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding, and should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0018] In related technologies, when testing and evaluating autonomous vehicles, most driving systems are still trained and evaluated on natural scenes collected from daily life or heuristically generated collision scenarios.

[0019] However, generally speaking, a large number of vehicles result in extremely low collision rates, indicating that there are few safety-critical scenarios in the collected real-world data. Therefore, accumulating scenarios requires a large number of vehicles and extensive testing and operation in numerous real-world environments, which is very costly. Thus, methods for artificially generating scenarios are crucial for risk assessment and cost reduction. To develop realistic simulators for autonomous vehicle testing, simulating various scenarios that might occur near vehicles in the real world is necessary. However, rule-based artificial scenario editing in related technologies also has some shortcomings: due to rule constraints, the trajectories of obstacles are predefined and cannot realistically reflect the movement states of obstacles in the real environment. Therefore, in related technologies, the generation of scenarios (e.g., for autonomous driving testing) suffers from high costs and low quality.

[0020] In view of the above problems, according to embodiments of this disclosure, this disclosure provides a scene generation method. Figure 1 This is a flowchart of a scene generation method provided according to embodiments of this disclosure, such as... Figure 1 As shown, the process includes the following steps:

[0021] Step S102: Determine the first scene, wherein the target vehicle moves along a first trajectory and the obstacle moves along a second trajectory in the first scene;

[0022] As an optional embodiment, the method disclosed herein can be applied to any autonomous driving testing scenario, such as to a terminal or server requiring autonomous driving testing. For example, when applied to a terminal used to perform autonomous driving testing, the terminal may have autonomous driving testing software installed, and the autonomous driving test is implemented through this software. When applied to a terminal, it can easily and simply handle timely autonomous driving testing. Alternatively, when applied to a server performing autonomous driving testing, the server may have an autonomous driving testing platform deployed, and autonomous driving testing is implemented by calling various data (such as detailed map data, detailed positioning data, detailed perception data, etc.). Therefore, when applied to a server, since the data called can be more comprehensive and accurate, detailed autonomous driving testing can be achieved comprehensively and accurately.

[0023] It should be noted that the types of terminals mentioned above can be various, such as mobile terminals or fixed computer devices. Mobile terminals can include mobile phones, iPads, laptops, etc. Similarly, the types of servers mentioned above can also be various, such as local servers or virtual cloud servers. Servers, based on computing power, can be single computer devices or computer clusters integrating multiple computer devices.

[0024] As an optional embodiment, the first scenario described above can be a basic scenario generated based on real road data. This scenario can be some safety-critical scenarios. For example, when considering collision accidents in autonomous driving, in this first scenario, the target vehicle may collide with an obstacle, or the target vehicle may not collide with an obstacle in the first scenario, but there is a risk of collision. That is, in this first scenario, the target vehicle may actually collide with an obstacle, or there may be a potential collision risk.

[0025] As an optional embodiment, the target vehicle can be an autonomous vehicle, also known as the master vehicle, and the obstacle can be an obstacle vehicle, or other people or objects that can collide with the target vehicle or pose a risk of collision.

[0026] It should be noted that in the first scenario described above, when the target vehicle moves along the first trajectory and the obstacle moves along the second trajectory, a safety event occurs between the target vehicle and the obstacle (i.e., some key scenarios of concern), such as a collision. The occurrence of a collision is merely one example of this disclosure; other key safety scenarios are also part of this disclosure and will not be listed here.

[0027] Furthermore, the aforementioned first and second trajectories can be determined based on a predetermined coordinate system, such as a two-dimensional coordinate system in a plane. After determining this coordinate system, in addition to representing the first and second trajectories in the first scenario within this coordinate system, the environment in which the target vehicle and obstacles are located can also be represented, such as interfering vehicles involved in safety incidents involving the target vehicle and obstacles, traffic facilities, road elements (lane lines, intersections, etc.), etc.

[0028] Step S104: Divide the second trajectory into multiple discrete path segments;

[0029] As an optional embodiment, when dividing the second trajectory into discrete path segments, the curve corresponding to the second trajectory from the starting point to the ending point is divided into multiple segments, each segment corresponding to a control point (or reference point). This control point or reference point can be a unit for comparison with the subsequent second trajectory. The overall offset relative to the second trajectory is obtained based on the accumulation of these control points or reference points.

[0030] Step S106: Determine the displacement offsets of the multiple path segments in terms of displacement, and obtain the total displacement offsets of the second trajectory.

[0031] As an optional embodiment, using path segments as comparison objects, the displacement offset from the corresponding path segment is determined for each segment. Then, the displacement offsets corresponding to multiple path segments are accumulated, and the sum of these offsets is obtained by directly summing the individual offsets to obtain the total displacement offset from the second trajectory. It should be noted that the path segment is also a length; to obtain the displacement offset corresponding to this length, the position points along this length can be summed or integrated. Furthermore, the displacement of position points in a path segment can be represented in various ways. For example, it can be represented as a function of time or as a function of path length. The choice can be flexible and based on the calculation needs. In this optional embodiment, the displacement of the corresponding position point in the path segment is represented as a function of path length. Therefore, the displacement of the corresponding path segment is obtained by integrating the path length of the included position points.

[0032] It should be noted that the displacement amounts relative to multiple path segments mentioned above can be determined based on predetermined constraints. These constraints can be of various types, such as direct displacement constraints or curvature constraints.

[0033] Step S108: Determine the velocity offset of each of the multiple path segments in terms of velocity, and obtain the sum of the velocity offsets of the second trajectory.

[0034] As an optional embodiment, using path segments as comparison objects, the velocity offset from the corresponding path segment is determined for each segment. Then, the velocity offsets corresponding to multiple path segments are accumulated, and the sum of these velocity offsets is obtained by directly summing the individual velocity offsets to obtain the total velocity offset from the second trajectory. Similarly, a path segment is a length; to obtain the velocity offset corresponding to this length, the position points along this length can be summed or integrated. Furthermore, the velocity of a path segment can be represented in various ways; for example, it can be represented as a function of time or as a function of the path length. The specific choice can be flexibly made according to the calculation needs. In this optional embodiment, the velocity of the corresponding position point in the path segment is represented as a function of the path length. Therefore, the velocity of the corresponding path segment is obtained based on the starting and ending velocities of the path segment as a function of the path length, and the acceleration of the path segment.

[0035] Furthermore, the aforementioned velocity offset from multiple path segments can also be determined based on predetermined constraints. These constraints can be of various types, such as direct velocity constraints or acceleration constraints.

[0036] Step S110: Based on the second trajectory, the sum of displacement offsets, and the sum of velocity offsets, the third trajectory is obtained;

[0037] As an optional embodiment, when obtaining the third trajectory based on the second trajectory, the sum of displacement offsets, and the sum of velocity offsets, the third trajectory can be obtained by directly offsetting the sum of displacement offsets and the sum of velocity offsets from the second trajectory. It should be noted that the above-mentioned offsetting of the sum of displacement offsets refers to the offset of the entire second trajectory to obtain the displacement on the third trajectory. This can be achieved by offsetting the velocity offsets of each path segment formed by the sum of displacement offsets. By directly summing the displacement and velocity offsets, the new trajectory, i.e., the third trajectory, relative to the second trajectory can be obtained quickly and directly.

[0038] In test scenarios, displacement and velocity offsets receive different levels of attention, or their impact on the test differs. Therefore, to reflect these different effects, different weights can be assigned to displacement and velocity offsets. Different trajectories can be generated based on these weights, which not only more accurately reflects real-world scenarios but also allows for the creation of more diverse new scenarios. For example, when obtaining a third trajectory based on the second trajectory, the sum of displacement and velocity offsets, a first weight corresponding to the sum of displacement offsets and a second weight corresponding to the sum of velocity offsets can be determined. Similarly, the third trajectory can be obtained based on the second trajectory, the sum of displacement offsets, the first weight, the sum of velocity offsets, and the second weight. By assigning weights to displacement and velocity offsets, the impact of displacement and velocity on the generation of new trajectories is characterized. This approach considers both the intuitive impact of displacement and the effective influence of velocity, resulting in more realistic generated scenarios.

[0039] It should be noted that the allocation of the first and second weights mentioned above can be flexibly determined based on the needs of the scenario. For example, when displacement is of greater concern, the first weight can be set higher, while when velocity is of relatively higher concern, the second weight can be set higher. Therefore, setting different weights for displacement and velocity offsets allows for different considerations of the objects of concern based on specific scenarios, enabling differentiated attention in different situations.

[0040] As an optional embodiment, when obtaining a third trajectory based on the second trajectory, the sum of displacement offsets, the first weight, the sum of velocity offsets, and the second weight, different allocations of the first and second weights can yield multiple different third trajectories, i.e., generating new trajectories deviating from the second trajectory by different magnitudes. For example, by repeatedly adjusting the values ​​of the first and second weights, corresponding values ​​for each adjustment can be obtained; based on the second trajectory, the sum of displacement offsets, the sum of velocity offsets, and the corresponding values ​​of the first and second weights for each adjustment, the corresponding third trajectories can be determined. Due to the diversity of the allocation of the first and second weights, a greater number of third trajectories are obtained, effectively improving the efficiency of expanding new trajectories.

[0041] It should be noted that the above scheme for obtaining the third trajectory based on the second trajectory, the sum of displacement offsets, and the sum of velocity offsets only considers the second trajectory itself to obtain the new trajectory. In the process of generating the scene, in addition to considering the original second trajectory itself, to make the generated trajectory blend better with the environment or more closely resemble the real environment, some environment-related factors can also be considered. For example, other people and objects besides the target vehicle and obstacles can be considered. For instance, pedestrians, vehicles, or traffic facilities that interfere with the target vehicle and obstacles can be considered.

[0042] Based on the above considerations of the real-world environment, in this optional embodiment, the lane lines along which the obstacle travels are considered. Because under normal circumstances, vehicle movement is based on traffic rules, following lane lines better reflects real-world driving behavior. Therefore, when obtaining the third trajectory based on the second trajectory, the sum of displacement offsets, and the sum of velocity offsets, the target lane centerline in the first scenario can be determined first; then, based on the target lane centerline, the second trajectory, the sum of displacement offsets, and the sum of velocity offsets, the third trajectory is obtained. Using this processing method, when generating a new trajectory, in addition to referencing the second trajectory of the obstacle, lane lines from basic traffic rules can be used to limit it, ensuring that the newly generated trajectory not only conforms to the real second trajectory but also satisfies normal driving behavior. This constraint that the newly generated trajectory conforms to the second trajectory is in line with driving regulations and is reasonable.

[0043] As an optional embodiment, when obtaining the third trajectory based on the target lane centerline, the second trajectory, the sum of displacement offsets, and the sum of velocity offsets, an initial trajectory can be determined first. This initial trajectory is offset from the second trajectory by the sum of displacement offsets and by the sum of velocity offsets. Then, the lateral offset between the initial trajectory and the target lane centerline is determined. If the lateral offset is less than a lateral offset threshold, the initial trajectory is determined as the third trajectory. This lateral offset threshold can be a pre-defined magnitude or direction of displacement offset.

[0044] For example, the width of the lane corresponding to the lane centerline can be used as the threshold for this lateral offset. When the lateral offset is less than the threshold, it means that the initial trajectory is still within the lane corresponding to the target lane centerline, i.e., the driving behavior is within the lane and conforms to normal driving behavior. However, when the lateral offset between the determined initial trajectory and the target lane centerline exceeds the width of that lane, it can be considered that the obstacle is seriously deviating from the lane when traveling according to the initial trajectory, i.e., it does not conform to vehicle driving behavior. The generated initial trajectory needs to be corrected to conform to normal vehicle driving behavior.

[0045] Therefore, after determining the lateral offset between the initial trajectory and the centerline of the target lane, if the lateral offset is greater than or equal to a lateral offset threshold, the initial trajectory is adjusted to obtain a target trajectory with a lateral offset less than the lateral offset threshold from the centerline of the target lane; this target trajectory is then designated as the third trajectory. Through this process, the third trajectory, extended from the second trajectory, is extended under dual reference lines, ensuring that the extended third trajectory more closely reflects real-world driving behavior.

[0046] Step S112: Generate a second scene based on the first trajectory and the third trajectory.

[0047] As an optional implementation, various methods can be used to generate a second scene based on the first and third trajectories. For example, the second scene can be constructed by keeping the first trajectory of the target vehicle unchanged, replacing the second trajectory of the obstacles with an extended third trajectory, and keeping other environmental aspects unchanged. Another example is by swapping the roles of the target vehicle and the obstacles, that is, by exchanging the driving data of the target vehicle and the obstacles, thereby expanding the scene. For instance, the target vehicle can move along the third trajectory, and the obstacles can move along the first trajectory, thus constructing a new scene. The second scene obtained through the above two methods can take the following forms: in the second scene, the target vehicle moves along the first trajectory, and the obstacles move along the third trajectory; or, in the second scene, the target vehicle moves along the third trajectory, and the obstacles move along the first trajectory. In expanding the test scene, a combination of the above two forms is also possible.

[0048] It should be noted that when using the role-swapping method to expand new scenarios, a new scenario can be constructed not only based on the newly generated trajectories of the obstacles and the original trajectory of the target car, but also directly based on the original trajectories of the obstacles and the original trajectory of the target car. For example, in the first scenario mentioned above, the target car moves along the first trajectory, and the obstacle car moves along the second trajectory. The new second scenario could simply be that the target car moves along the second trajectory, and the obstacle car moves along the first trajectory.

[0049] Expanding new scenarios by using the aforementioned role-swapping method allows the target vehicle to be driven from a new perspective. Furthermore, expanding new scenarios by directly swapping roles only requires simple data transformations and does not require excessive calculations or data operations, thus achieving higher efficiency while ensuring realism and accuracy.

[0050] Through the above steps, since the first scene is a collected real-world scene, the second trajectory of the obstacles within it is a real trajectory. Offsetting based on this real second trajectory ensures that the resulting third trajectory is generated based on the real trajectory, accurately approximating reality. Furthermore, since the first trajectory is also real, when generating the second scene based on the real first trajectory and the accurately approximating third trajectory, the second scene is also more realistic. Additionally, considering the relatively independent displacement and velocity offsets—that is, the reference displacement and velocity are decoupled—this decoupling simplifies the calculation process and effectively improves scene generation efficiency. This approach avoids the need to collect large amounts of already scarce real-world scene data, thus preventing high costs. Furthermore, generating the second scene based on the real first trajectory and the accurately approximating third trajectory allows for the efficient generation of a large number of accurate scenes based on different offsets. Based on these two considerations, it effectively achieves the goal of reducing scene generation costs while ensuring scene accuracy, resulting in more realistic test scenes that effectively meet testing requirements.

[0051] The following examples illustrate this disclosure in the context of autonomous driving scenarios.

[0052] Figure 2 This is a structural schematic diagram of an autonomous driving system provided according to an embodiment of the present disclosure, such as... Figure 2 As shown, the autonomous driving system mainly includes the following components: a high-precision map module, a positioning module, a perception module, a global navigation module, a prediction module, a planning module, and a control module. The high-precision map module provides high-precision map services; the positioning module provides high-precision (centimeter-level) positioning services; the perception module uses cameras, LiDAR, millimeter-wave radar, ultrasonic radar, and other devices combined with advanced obstacle detection algorithms to provide comprehensive environmental perception services for the autonomous vehicle; the prediction module uses perception data obtained from the upstream perception module as input, extracts historical motion parameters of obstacles, and combines Kalman filtering, neural networks, and other methods to infer the future motion trajectory of obstacles for use by the downstream planning and control modules; the global navigation module, based on the vehicle's initial and target positions and the road network topology, uses a global path search algorithm to obtain the optimal global navigation path that meets performance evaluation indicators; the planning module provides services for obstacle avoidance, lane changing decisions, path planning, and speed planning; and the control module performs longitudinal and lateral tracking control based on the driving trajectory provided by the decision-making and planning module.

[0053] Based on the above autonomous driving scenarios, in order to provide safety assessment tests for autonomous driving behavior, it is necessary to simulate a large number of safety scenarios.

[0054] The simulation methods used in related technologies, such as those based on human-imposed constraints, cannot reflect the actual movement of obstacles in real-world scenarios, resulting in low-quality scenarios that are unusable for testing.

[0055] In view of this, and in conjunction with autonomous driving scenarios, this disclosure provides an optional implementation method.

[0056] This optional implementation mainly includes the following processes:

[0057] S1, Data Collection (Testing and Operational Data Collection). It should be noted that during the data collection process, data can be collected from autonomous vehicles traveling on real test roads, from other devices with data collection capabilities traveling on real test roads, or from data collection devices permanently installed on real test roads.

[0058] S2 generates a basic scene based on real road data (corresponding to the first scene mentioned above), and the scene is expressed as follows:

[0059] (A) Map name and version

[0060] (B) Original scene data (obtained based on real scene data collection):

[0061] (1) Scenery: Map elements (lanes, intersections, etc.), traffic facilities (traffic lights), temporary structures (cones, fences, etc.).

[0062] (2) Environment: temperature, weather, light, etc.

[0063] (3) Traffic participants: Assume there are N traffic participants A = (A1, A2, ..., A...). n The original trajectory data of each obstacle is G = (G1, G2, ..., G...). n Within a given time period D (M trajectory points) in the scene data, the trajectory of each obstacle consists of multiple trajectory points, G. i =(G i1 G i2 ,…,G im If the obstacle is static, the trajectory points will coincide at multiple moments within D.

[0064] S3, based on a given base scene S generated in S2, transforms the obstacle trajectories to expand into more simulation scenes, explained in two cases:

[0065] The first scenario involves generating a scenario based on a planning algorithm.

[0066] In this first scenario: the ADC (master vehicle) remains unchanged, meaning the initial and final positions and the navigation path remain unchanged.

[0067] (1) Determine the list of obstacles to be transformed. s (Obstacles that overlap with the path or obstacles that may overlap, corresponding to the obstacles mentioned above that have caused or pose a risk of collision);

[0068] (2) Traverse O s For each of the obstacles O i From the original trajectory G i Obtain the initial trajectory point P in the middle si and the ending trajectory point P ei , with P si Starting from P ei With the destination as the starting point, load the map, using other obstacles and the ADC (main vehicle) as obstacles, and generate multiple candidate trajectories using a similar standard method to ADC: G i →G′ i =(G′ i1 ,G′ i2 ,G′ i3 ,…,G′ ik ).

[0069] (3) Traverse G′ i And keep the trajectory of other obstacles unchanged to form a new scene Si'.

[0070] (4) Complete the expansion of the new scene: S→S′=(S′1,S′2,S′3,…,S′) n (), where each participant includes multiple trajectories.

[0071] The second scenario involves generating a new scene based on the changing roles of the main vehicle and the obstacle vehicle:

[0072] In this second scenario: the concept of ADC (driver assist vehicle) is weakened, and the ADC is treated as a traffic participant, with the initial and final positions of the obstacles serving as the initial and final positions of the ADC to be planned.

[0073] (1) Determine the list of obstacles to be transformed. s (The path contains overlapping obstacles).

[0074] (2) Traverse O s For each obstacle O i From the original trajectory G i Obtain the initial trajectory point P in the middle si and the ending trajectory point P ei .

[0075] (3) Using the initial trajectory point Psi and the ending trajectory point P ei As the initial and final positions of the ADC (Master Vehicle), the original trajectory G is used. i As the reference trajectory for the ADC (Master Vehicle), a new scene Si' is generated.

[0076] (4) Complete the new scene expansion: S→S′=(S′1,S′2,S′3,…,S′) n ).

[0077] In the process of generating candidate trajectories based on the initial and final positions, various methods can be used. One implementation method is given below.

[0078] For real-world road test scenarios, we focus on two types: the first type, where the main vehicle and the obstacle vehicle actually collide; and the second type, where the main vehicle and the obstacle vehicle do not actually collide, but there is a potential risk of collision.

[0079] (1) For the two situations mentioned above, determine the obstacle number that collides with the main vehicle or poses a risk of collision with the main vehicle.

[0080] (2) For the obstacles identified above, a new scenario is generated through a decision planning algorithm.

[0081] The following example uses an obstacle vehicle as an example to illustrate how a new scenario can be generated through a decision-planning algorithm.

[0082] Using the lane centerline as a reference (reference line Ref1) and the original trajectory of the obstacle vehicle as the original trajectory reference (second reference line Ref2), the trajectory is planned (this method ensures that the planned trajectory is as similar as possible to the original trajectory, thus more closely resembling reality). Based on reference lines Ref1 and Ref2, the offset of reference line Ref2 relative to reference line Ref1 at each longitudinal position point can be calculated. Since the second reference line is used to determine the offset relative to the lane center line, the newly generated trajectory, when offset relative to the second reference line, also uses the lane center line as the reference. This ensures that the newly generated trajectory references not only the second reference line corresponding to the obstacle vehicle but also the lane center line, achieving two close approximations to the actual driving, thus guaranteeing the accuracy of the scenario.

[0083] Figure 3 This is a schematic diagram of a vehicle steering model provided according to an embodiment of this disclosure, such as... Figure 3 As shown below: Each point on the new trajectory is represented by x, y, θ, k, v, a, t. These correspond to coordinates, heading, curvature, velocity, acceleration, and time, respectively. Based on the above model, the state equation of the obstacle vehicle is as follows:

[0084]

[0085] By replacing time with path length for transformation, we obtain the following differential formula.

[0086]

[0087] Integrating yields the x, y, θ, and k values ​​corresponding to s.

[0088]

[0089]

[0090]

[0091] k(s)=u(s)

[0092] If the curvature k is expressed as a polynomial in s, then x, y, θ, k can be represented by a series of parameters such as a, b, c, d, etc.

[0093] k(s) = a + bs + cs 2 +ds 3 +…

[0094]

[0095]

[0096]

[0097] because

[0098]

[0099] so

[0100]

[0101] When σ = 0

[0102]

[0103]

[0104]

[0105] Where υ(s) represents the velocity at s, v0 represents the initial velocity, and σ represents the acceleration, which can be a fixed value.

[0106] The starting point is determined by the initial position, heading angle, initial curvature (default is 0), and initial velocity of the obstacle vehicle in the original scene.

[0107] The endpoint is determined by the vehicle's termination position, heading angle, termination curvature (default is 0), and termination speed in the original scene.

[0108] The starting point and ending point information are determined by the starting point and ending point data of the original trajectory of the obstacle vehicle.

[0109] The deviation between the new trajectory and the original trajectory of the obstacle vehicle is represented as the total cost.

[0110] Total cost C total =W1C1+W2C2

[0111] Where W1 and W2 represent weighting coefficients, and C1 and C2 represent the costs of lateral displacement and velocity deviation from the second reference line, respectively. The new trajectory can be discretized according to a given step size to obtain N path segments. For the path length s in each segment, x(s), y(s), θ(s), k(s), υ(s), and t(s) can be calculated using the above formulas. The position and velocity offsets of each state point relative to the original trajectory are also calculated, and finally, they are summed.

[0112]

[0113]

[0114] The subscript "raw" indicates a path segment in the original trajectory.

[0115] By controlling the coefficients of W1 and W2, new trajectories that deviate from the original trajectory by different magnitudes can be generated, and these new trajectories can be used as new scene data.

[0116] Based on the above optional implementation methods, the following beneficial effects can be achieved:

[0117] Simulation scenarios are generated based on real data, and new simulation scenarios are created by transforming and expanding these scenarios, which greatly increases the number of scenarios and coverage.

[0118] Keeping the original map, the initial position of the main vehicle, the target position, and the navigation trajectory unchanged, a new trajectory for the obstacle vehicle is regenerated through a planning-based approach. This new trajectory serves as input for the new scenario, more closely resembling the real-world scenario. It can be used to verify the rationality and reliability of the strategy by considering the diverse movement of obstacle vehicles in real-world scenarios.

[0119] Instead of limiting the initial position, ending position, and navigation trajectory of the main vehicle, the initial and ending positions of the obstacle vehicle are transformed into the initial and target positions of the main vehicle (ADC), and the original trajectory of the obstacle vehicle is used as the reference trajectory of the main vehicle (ADC) to form new scenarios. These scenarios can be considered as original mirror scenarios. From the perspective of the obstacle vehicle, we can see the autonomous driving capability of the main vehicle in the current scenario.

[0120] In this embodiment of the disclosure, a scene generation apparatus is provided. Figure 4 This is a structural block diagram of the scene generation apparatus provided according to the embodiments of this disclosure, such as... Figure 4 As shown, the device includes: a first determining module 41, a dividing module 42, a second determining module 43, a third determining module 44, a processing module 45, and a generating module 46. The device will be described below.

[0121] A first determining module 41 is used to determine a first scene, wherein a target vehicle moves along a first trajectory and obstacles move along a second trajectory in the first scene; a segmentation module 42, connected to the first determining module 41, is used to divide the second trajectory into multiple discrete path segments; a second determining module 43, connected to the segmentation module 42, is used to determine the displacement offsets of each of the multiple path segments in terms of displacement, and obtain the sum of displacement offsets from the second trajectory; a third determining module 44, connected to the second determining module 43, is used to determine the velocity offsets of each of the multiple path segments in terms of velocity, and obtain the sum of velocity offsets from the second trajectory; a processing module 45, connected to the third determining module 44, is used to obtain a third trajectory based on the second trajectory, the sum of displacement offsets, and the sum of velocity offsets; and a generation module 46, connected to the processing module 45, is used to generate a second scene based on the first trajectory and the third trajectory.

[0122] As an optional embodiment, the processing module 45 includes: a first determining unit and a first processing unit, wherein the first determining unit is used to determine a first weight corresponding to the sum of displacement offsets and a second weight corresponding to the sum of velocity offsets; the first processing unit is connected to the first determining unit and is used to obtain a third trajectory based on the second trajectory, the sum of displacement offsets, the first weight, the sum of velocity offsets, and the second weight.

[0123] As an optional embodiment, the first processing unit is configured to obtain the first weight value and the second weight value corresponding to the multiple adjustments by adjusting the weight value of the first weight and the weight value of the second weight multiple times; and to determine the third trajectory corresponding to the multiple adjustments based on the second trajectory, the sum of displacement offset, the sum of velocity offset, and the first weight value and the second weight value corresponding to the multiple adjustments.

[0124] As an optional embodiment, the processing module 45 includes: a second determining unit and a second processing unit, wherein the second determining unit is used to determine the center line of the target lane in the first scenario; and the second processing unit is used to obtain a third trajectory based on the center line of the target lane, the second trajectory, the sum of displacement offsets, and the sum of speed offsets.

[0125] As an optional embodiment, the second processing unit includes a first determining subunit, a second determining subunit, and a third determining subunit. The first determining subunit is used to determine an initial trajectory, wherein the initial trajectory is the sum of the displacement offsets of the second trajectory and the sum of the velocity offsets of the second trajectory. The second determining subunit, connected to the first determining subunit, is used to determine the lateral offset between the initial trajectory and the centerline of the target lane. The third determining subunit, connected to the second determining subunit, is used to determine the initial trajectory as the second trajectory if the lateral offset is less than a lateral offset threshold.

[0126] As an optional embodiment, the second processing unit further includes an adjustment subunit and a fourth determination subunit, wherein the adjustment subunit is connected to the second determination subunit and is used to adjust the initial trajectory when the lateral offset is greater than or equal to a lateral offset threshold, so as to obtain a target trajectory with a lateral offset less than the lateral offset threshold between it and the center line of the target lane; the fourth determination subunit is connected to the adjustment subunit and is used to determine the target trajectory as the second trajectory.

[0127] As an optional embodiment, in the second scenario, the target vehicle moves along a first trajectory and the obstacle moves along a third trajectory; or, in the second scenario, the target vehicle moves along a third trajectory and the obstacle moves along a first trajectory.

[0128] As an optional embodiment, the target vehicle collides with the obstacle in the first scenario, or the target vehicle does not collide with the obstacle in the first scenario, but there is a risk of collision.

[0129] The acquisition, storage, and application of user personal information involved in the technical solution disclosed herein comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0130] According to embodiments of this disclosure, this disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0131] The electronic device includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform any of the above-mentioned scene generation methods.

[0132] The aforementioned readable storage medium may be a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to cause the computer to execute any of the above-mentioned scene generation methods.

[0133] The aforementioned computer program product includes a computer program, which, when executed by a processor, implements any of the above-mentioned scene generation methods.

[0134] Figure 5 A schematic block diagram of an example electronic device 500 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0135] like Figure 5 As shown, the electronic device 500 includes a computing unit 501, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 502 or a computer program loaded from a storage unit 508 into a random access memory (RAM) 503. The RAM 503 may also store various programs and data required for the operation of the electronic device 500. The computing unit 501, ROM 502, and RAM 503 are interconnected via a bus 504. An input / output (I / O) interface 505 is also connected to the bus 504.

[0136] Multiple components in electronic device 500 are connected to I / O interface 505, including: input unit 506, such as keyboard, mouse, etc.; output unit 507, such as various types of monitors, speakers, etc.; storage unit 508, such as disk, optical disk, etc.; and communication unit 509, such as network card, modem, wireless transceiver, etc. Communication unit 509 allows electronic device 500 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0137] The computing unit 501 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 501 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 501 performs the various methods and processes described above, such as the scene generation method. For example, in some embodiments, the scene generation method may be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 508. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 500 via ROM 502 and / or communication unit 509. When the computer program is loaded into RAM 503 and executed by the computing unit 501, one or more steps of the scene generation method described above may be performed. Alternatively, in other embodiments, the computing unit 501 may be configured to perform the scene generation method by any other suitable means (e.g., by means of firmware).

[0138] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0139] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0140] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0141] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0142] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with embodiments of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0143] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact via communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other. Servers can be cloud servers, servers in distributed systems, or servers incorporating blockchain technology.

[0144] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.

[0145] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. A method for generating a scenario, comprising: determining a first scenario in which a target vehicle moves according to a first trajectory and an obstacle moves according to a second trajectory; dividing the second trajectory into a plurality of discrete path segments; determining displacement offsets respectively offset from the plurality of path segments in displacement, to obtain a sum of displacement offsets offset from the second trajectory, wherein the displacement offsets are obtained by integrating displacements of position points on the path segments; determining velocity offsets respectively offset from the plurality of path segments in velocity, to obtain a sum of velocity offsets offset from the second trajectory, wherein the velocity offsets are obtained by integrating velocities of the position points on the path segments; determining a target lane centerline in the first scenario; determining an initial trajectory offset from the second trajectory in displacement by the sum of displacement offsets and offset from the second trajectory in velocity by the sum of velocity offsets; determining a lateral offset between the initial trajectory and the target lane centerline; determining the initial trajectory as a third trajectory when the lateral offset is less than a lateral offset threshold; generating a second scenario in which the target vehicle moves according to the first trajectory and the obstacle moves according to the third trajectory, or in which the target vehicle moves according to the third trajectory and the obstacle moves according to the first trajectory, based on the first trajectory and the third trajectory.

2. The method of claim 1, wherein, The third trajectory is determined based on the second trajectory, the sum of displacement offsets, and the sum of velocity offsets, comprising: determining a first weight corresponding to the sum of displacement offsets and a second weight corresponding to the sum of velocity offsets; determining the third trajectory based on the second trajectory, the sum of displacement offsets, the first weight, the sum of velocity offsets, and the second weight.

3. The method of claim 2, wherein, The third trajectory is determined based on the second trajectory, the sum of displacement offsets, the first weight, the sum of velocity offsets, and the second weight, comprising: obtaining a first weight value and a second weight value corresponding to a plurality of adjustments of the first weight and the second weight respectively by adjusting the weight value of the first weight and the weight value of the second weight multiple times; determining a third trajectory corresponding to the plurality of adjustments respectively based on the second trajectory, the sum of displacement offsets, the sum of velocity offsets, and the first weight value and the second weight value corresponding to the plurality of adjustments respectively.

4. The method of claim 1, wherein, The method further comprises: adjusting the initial trajectory to obtain a target trajectory with a lateral offset between the target trajectory and the target lane centerline less than the lateral offset threshold when the lateral offset is greater than or equal to the lateral offset threshold; determining the target trajectory as the third trajectory.

5. The method of any one of claims 1 to 4, wherein, The target vehicle collides with the obstacle in the first scenario, or the target vehicle does not collide with the obstacle in the first scenario, but there is a risk of collision. 6.A device for generating a scenario, comprising: The first determining module is configured to determine a first scene in which a target vehicle moves according to a first trajectory and an obstacle moves according to a second trajectory. The dividing module is configured to divide the second trajectory into a plurality of discrete path segments. The second determining module is configured to determine displacement offset amounts respectively offset from the plurality of path segments in displacement, to obtain a total displacement offset amount offset from the second trajectory, wherein the displacement offset amount is obtained by integrating displacement of a position point on the path segment. The third determining module is configured to determine velocity offset amounts respectively offset from the plurality of path segments in velocity, to obtain a total velocity offset amount offset from the second trajectory, wherein the velocity offset amount is obtained by integrating velocity of a position point on the path segment. The processing module includes a second determining unit and a second processing unit, wherein the second determining unit is configured to determine a target lane center line in the first scene. The second processing unit includes a first determining sub-unit, a second determining sub-unit, and a third determining sub-unit, wherein the first determining sub-unit is configured to determine an initial trajectory, wherein the initial trajectory is offset from the second trajectory by the total displacement offset amount in displacement and by the total velocity offset amount in velocity; the second determining sub-unit is configured to determine a lateral offset amount between the initial trajectory and the target lane center line; and the third determining sub-unit is configured to determine the initial trajectory as a third trajectory when the lateral offset amount is less than a lateral offset amount threshold. The generating module is configured to generate a second scene based on the first trajectory and the third trajectory, wherein in the second scene, the target vehicle moves according to the first trajectory and the obstacle moves according to the third trajectory, or in the second scene, the target vehicle moves according to the third trajectory and the obstacle moves according to the first trajectory.

7. An electronic device comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1 to 5.

8. A non-transitory computer readable storage medium having stored thereon computer instructions, wherein, The computer instructions are used to enable the computer to perform the method of any one of claims 1 to 5.

9. A computer program product comprising a computer program which, when executed by a processor, implements the method of any one of claims 1 to 5.

Images