Scene generation method and related device

Abstract

A scene generation method and a related device are applied to the field of intelligent driving. According to the embodiment of the invention, the motion information of the obstacle can be obtained according to the scene data of the first driving scene, the motion information of the first obstacle in the first driving scene is adjusted through controllable modification, and the motion information of the second obstacle in the first driving scene is adjusted through controllable modification. If yes, the motion information of the second obstacle is jointly optimized and updated through the motion information of the first obstacle and the original motion information of the second obstacle, and a target driving scene can be generated based on the updated motion information of the first obstacle and the updated motion information of the second obstacle. In a word, according to the scheme of the invention, part of the obstacles in the first driving scene can be controllably adjusted, and part of the obstacles are jointly optimized for adaptive adjustment, so that under the condition that the authenticity of the generated scene is ensured, controllable modification of the scene is realized, and the interactivity between the obstacles in the scene is improved.

Description

Technical Field

[0001] The present application relates to the field of intelligent driving technology, and in particular to a scene generation method and related devices. Background Art

[0002] To ensure safety, vehicles need to be tested before they are put on the market. Among them, simulation test scenarios have the advantages of high coverage, safe test process and high test efficiency, and are widely used in testing intelligent driving terminals. At present, various manufacturers have launched a large number of simulators for self-use or commercial use.

[0003] The ability to build test scenarios is an important factor in measuring the quality of simulators. When building test scenarios, some simulators generate driving scenarios by manually editing objects in the scene, while some simulators collect real road test data and convert it into driving scenarios. However, the scenes edited by the former may be difficult to appear in real situations and lack authenticity. While the scenes are formed by converting road test data, the authenticity of the scenes can be guaranteed, but a lot of time is required to collect real road test scenes. This way of generating driving scenes lacks controllability of the scenes. When new interactive scenes need to be designed, new scenes need to be collected again to obtain new scenes. For example, when a new action (such as braking, changing lanes) needs to be added to the front vehicle to test the intelligent driving function of the vehicle itself, it is often necessary to re-drive the actor car to collect its corresponding road test data to form a new scene.

[0004] In summary, how to improve the controllability of the scene without significantly reducing the realism of the scene is a hot issue that technical personnel in this field are studying. Summary of the invention

[0005] The embodiments of the present application provide a scene generation method and related devices, which can achieve controllable modification of the scene and improve the interactivity between obstacles in the scene while ensuring the authenticity of the generated scene.

[0006] In a first aspect, an embodiment of the present application provides a scene generation method, the method comprising:

[0007] Acquire scene data of a first driving scene, where the scene data is used to indicate initial motion information of obstacles in the first driving scene, where the obstacles in the first driving scene include a first obstacle and a second obstacle;

[0008] Adjusting initial motion information of the first obstacle to generate first motion information of the first obstacle;

[0009] Obtaining second motion information of the second obstacle according to the initial motion information of the second obstacle and the first motion information of the first obstacle;

[0010] A target driving scene is generated according to the first motion information of the first obstacle and the second motion information of the second obstacle.

[0011] The embodiment of the present application obtains the motion information of multiple obstacles according to the scene data of the first driving scene, and one or more new scenes can be constructed by updating the motion information of the obstacles. However, different updating methods can be used for different obstacles. Take the case where the obstacles at least include the first obstacle and the second obstacle. For the first obstacle, the embodiment of the present application adjusts its motion information through controllable modification, that is, the motion information of the first obstacle can be manually modified, automatically modified, or even modified based on the artificial intelligence (AI) model. The motion information of the first obstacle is adjusted. For the second obstacle, the embodiment of the present application updates the motion information of the second obstacle through the initial motion information of the second obstacle and the information of other objects in the scene. This updating process can also be called joint optimization. Among them, the information of other objects in the first driving scene can include the updated motion information of the first obstacle, and can also include the motion information of the vehicle, etc.

[0012] On the one hand, the original trajectory of the second obstacle is taken into account in the process of updating the motion information of the second obstacle, so that the updated motion information of the second obstacle (i.e., the second motion information) is more realistic. On the other hand, the influence of information changes of other objects in the scene is taken into account in the process of updating the motion information of the second obstacle, so that there is a connection between the second obstacle in the scene and other objects in the scene, thereby making the scene update process more interactive. In short, this solution can realize controllable modification of the scene and improve the interactivity between obstacles in the scene while ensuring the authenticity of the generated scene.

[0013] In addition, the number of first motion information can be multiple, that is, the motion information of the first obstacle can be adjusted to generate multiple first motion information, and the multiple first motion information is used to describe multiple possible motion processes of the first obstacle. Correspondingly, the number of second motion information can also be multiple groups. For example, when generating a target driving scene, the scene construction device can obtain a corresponding set of second motion operation information of the second obstacle for each first motion information of the first obstacle, so that multiple new driving scenes can be quickly constructed (that is, the target driving scene can also be multiple), so that the efficiency of scene construction can be significantly improved, and the multiple driving scenes can meet the user's needs for scene authenticity, controllability and interactivity.

[0014] In combination with the first aspect, in a possible implementation of the first aspect, the scene data further includes initial motion information of the vehicle. For example, the initial motion information of the vehicle further includes an initial motion trajectory of the vehicle. Optionally, the initial motion information of the vehicle can be collected by sensors of the vehicle, such as one or more of sensors such as a wheel speed meter, an inertial measurement unit, and a positioning system.

[0015] In combination with the first aspect, in a possible implementation manner of the first aspect, obtaining second motion information of the second obstacle according to initial motion information of the second obstacle and first motion information of the first obstacle includes:

[0016] The second motion information of the second obstacle is obtained according to the initial motion information of the vehicle, the initial motion information of the second obstacle and the first motion information of the first obstacle.

[0017] In this implementation, the motion information of the second obstacle can also be optimized in combination with the motion information of the vehicle, further improving the interactivity and authenticity of the scene process.

[0018] In combination with the first aspect, in another possible implementation of the first aspect, the degree of interaction between the first obstacle and the ego vehicle is not lower than the degree of interaction between the second obstacle and the ego vehicle. The degree of interaction refers to the degree of overlap in time and space between the movement trajectories of obstacles and obstacles or obstacles and the ego vehicle. Exemplarily, if there is a trajectory interaction point between the obstacle and the ego vehicle at a certain moment or time period, when the obstacle or the ego vehicle is at the trajectory interaction point, the closer the position of the obstacle is to the position of the ego vehicle, the higher the degree of interaction between the obstacle and the ego vehicle; if there is no trajectory interaction point between the obstacle and the ego vehicle, the shorter the distance between the position of the obstacle and the position of the ego vehicle when they are closest, the higher the degree of interaction.

[0019] It is understandable that for obstacles that have strong interactions with the vehicle, users have higher design requirements for their actions. Therefore, the first obstacle that has a high degree of interaction with the vehicle independently and controllably adjusts its motion information through controllable modification, while the second obstacle updates its motion information through joint optimization, combining the updated motion information of the first obstacle. This can better meet the requirements for the flexibility of the action design of strongly interactive obstacles and improve the user experience.

[0020] In combination with the first aspect, in another possible implementation of the first aspect, the initial motion information of the first obstacle is adjusted to generate the first motion information of the first obstacle, including: adjusting the initial motion information of the first obstacle according to the user's input to generate the first motion information of the first obstacle. In this implementation, the first obstacle can be an obstacle selected by the user to be adjusted, that is, this solution supports the user to make controllable modifications to the specified obstacle, improves the flexibility of scene updates, and further improves the user experience.

[0021] In combination with the first aspect, in another possible implementation of the first aspect, the initial motion information of the first obstacle is adjusted to generate the first motion information of the first obstacle, including: adjusting the initial motion information of the first obstacle through an artificial intelligence model to generate the first motion information of the first obstacle. In this implementation, the first obstacle can be an obstacle automatically selected by the artificial intelligence model and the motion information of the obstacle is automatically adjusted by the artificial intelligence model, which can not only improve the efficiency of updating the motion information and obtaining the target scene, but also improve the flexibility and intelligence of scene construction, which helps to improve the user experience.

[0022] In combination with the first aspect, in another possible implementation of the first aspect, the second obstacle is an obstacle that interacts with the updated first obstacle. In this case, the updated motion information of the first obstacle can be used when updating the motion information of the second obstacle, so as to more accurately and reasonably update the motion information of the second obstacle.

[0023] In combination with the first aspect, in another possible implementation of the first aspect, the second obstacle is an obstacle that interacts with the ego vehicle in addition to the first obstacle. In this case, the motion information of the ego vehicle can be used when updating the motion information of the second obstacle, so as to more accurately and reasonably update the motion information of the second obstacle.

[0024] In combination with the first aspect, in yet another possible implementation manner of the first aspect, the method further includes:

[0025] Determining the category of the obstacle according to the interaction degree between the obstacle and the vehicle and the interaction degree between the obstacles in the first driving scene, wherein the category of the obstacle includes a first category of obstacles and a second category of obstacles;

[0026] The interaction degree between the first type of obstacles and the ego vehicle is higher than that between the second type of obstacles and the ego vehicle.

[0027] In this way, since users have a greater need for controllable modification of obstacles with stronger interactivity, the above implementation can realize controllable modification of the first type of obstacles and joint optimization of the second type of obstacles, which can meet the demand for controllable modification of obstacles. Especially in the case of modifying obstacles through user operation or AI model, distinguishing between the first type of obstacles and the second type of obstacles can help users or AI models select obstacles that need to be modified and obstacles that need to be jointly optimized, which can improve the user experience.

[0028] Optionally, the first obstacle belongs to a first category of obstacles, and the second obstacle includes at least one obstacle in the second category of obstacles and / or includes at least one obstacle in the first category of obstacles other than the first obstacle. In some embodiments, the first category of obstacles may be referred to as strong interaction obstacles, and the second category of obstacles may be referred to as weak interaction obstacles.

[0029] In combination with the first aspect, in yet another possible implementation manner of the first aspect, the obstacles in the first driving scene include first-category obstacles and second-category obstacles, wherein:

[0030] The first type of obstacle is an obstacle whose interaction with the vehicle satisfies the first condition.

[0031] The second category of obstacles includes obstacles whose interaction degree with the ego vehicle satisfies the second condition and / or obstacles that interact with the third obstacle, and the third obstacle is at least one obstacle in the first category of obstacles, wherein the first condition is different from the second condition. Further, the first obstacle belongs to the first category of obstacles, and the second obstacle includes at least one obstacle in the second category of obstacles and / or includes at least one obstacle in the first category of obstacles other than the first obstacle.

[0032] In combination with the first aspect, in another possible implementation of the first aspect, the first condition includes: a time interval between the obstacle and the ego-vehicle passing through the trajectory interaction point falls within a first time range;

[0033] The second condition includes: the time interval between the obstacle and the ego-vehicle passing through the trajectory interaction point falls within a second time range, and the lower bound of the second time range is greater than the upper bound of the first time range;

[0034] There is interaction between the obstacle and the third obstacle when they meet the following conditions:

[0035] There is a trajectory interaction point between the obstacle and the third obstacle, and the time interval between the two passing the trajectory interaction point falls within the third time range.

[0036] The above implementation provides the characteristics of a first type of obstacle and a second type of obstacle. It can be seen that the first type of obstacle and the vehicle are close in time when passing through the trajectory interaction point, while the second type of obstacle and the vehicle are far apart in time when passing through the trajectory point. Since the closer the time interval, the greater the impact of the obstacle on the behavior of the vehicle, the closer the time interval, the higher the degree of interaction between the obstacle and the vehicle. In this way, the degree of interaction between the vehicle and the obstacle can be quantified, different types of obstacles can be distinguished, and it is helpful to select obstacles that need to be modified and obstacles that need to be jointly optimized, which can improve the user experience.

[0037] In combination with the first aspect, in another possible implementation of the first aspect, the first condition includes: there is a trajectory interaction point between the motion trajectory of the obstacle and the motion trajectory of the self-vehicle, and the first time interval between the two passing the trajectory interaction point is the first N time intervals of the time interval sequence, N is an integer greater than 0, and the first N time intervals of the time interval sequence are less than or equal to the time intervals other than the first N time intervals in the time interval sequence;

[0038] The second condition includes: there is a trajectory interaction point between the obstacle's trajectory and the vehicle's trajectory, and the first time interval between the two passing the trajectory interaction point falls within the first N+1 to K of the time interval sequence, where K is an integer and K>N;

[0039] Among them, the time interval sequence includes M time intervals arranged in order of time length, each of the M time intervals is the interval between the moment when an ego-vehicle interaction obstacle passes through the first trajectory interaction point with the ego-vehicle and the moment when the ego-vehicle passes through the first trajectory interaction point, the ego-vehicle interaction obstacle is an obstacle that has a trajectory interaction point with the ego-vehicle in the first driving scene, M is an integer and M≥K.

[0040] The above implementation provides another feature of the first and second obstacles. The first and second interactive obstacles are determined by sorting the degree of interaction. Relatively strong and relatively weak obstacles in any scenario can be obtained without being restricted by a fixed time range. The compatibility and flexibility of the obstacle classification scheme can be improved, which helps to select obstacles that need to be modified and obstacles that need to be jointly optimized, and can improve the user experience.

[0041] In combination with the first aspect, in another possible implementation of the first aspect, the initial motion information of the first obstacle includes multiple behavior parameters of the first obstacle, and the multiple behavior parameters are used to describe the behavior of the first obstacle. Adjusting the initial motion information of the first obstacle to generate first motion information of the first obstacle includes: adjusting the values ​​or thresholds of the multiple behavior parameters of the first obstacle to generate the first motion information of the first obstacle.

[0042] It should be understood that the behavior here is a behavior in a broad sense, including the behavior and state of an entity at a certain moment, or including actions performed over a period of time. This implementation supports the updating of the value of the behavior parameter or the threshold to flexibly update the movement process of the first obstacle, supports determining the behavior parameters at a certain moment or a certain movement stage, can improve the precision of obstacle information editing, and can obtain a scene that better meets user needs.

[0043] In combination with the first aspect, in another possible implementation manner of the first aspect, the initial motion information of the first obstacle includes a first action set and a first trigger set of the first obstacle, the first action set is used to describe a first action of the first obstacle, the first action set includes a value of a first behavior parameter of the first action, and the first trigger set is used to describe a condition for executing the first action;

[0044] The initial motion information of the first obstacle is adjusted to generate first motion information of the first obstacle, including:

[0045] The first action set and / or the first trigger set of the first obstacle are adjusted to generate the first motion information of the first obstacle. In the above implementation, the behavior parameters of the first obstacle can be integrated into an action set and a trigger set, so that the behavior parameter set within a certain stage can be modified to improve the user experience.

[0046] In combination with the first aspect, in another possible implementation manner of the first aspect, the initial motion information of the first obstacle includes an initial motion trajectory of the first obstacle;

[0047] The initial motion information of the first obstacle is adjusted to generate first motion information of the first obstacle, including:

[0048] The initial motion trajectory of the first obstacle is adjusted to obtain a first motion trajectory, and the first motion information includes the first motion trajectory.

[0049] The initial motion trajectory may include multiple trajectory points, which may indicate a first position change of the first obstacle within the first time period. Similarly, the first motion trajectory may also include multiple trajectory points, which may indicate that the first motion trajectory is used to indicate a second position change of the first obstacle within the first time period.

[0050] In the above implementation, the motion information of the obstacle can be updated by editing the motion trajectory, especially when only the trajectory needs to be adjusted to generate a new scene, which can significantly improve the efficiency of updating the motion information and enhance the user experience.

[0051] In combination with the first aspect, in another possible implementation manner of the first aspect, the initial motion trajectory of the first obstacle includes multiple trajectory points, and the initial motion trajectory of the first obstacle is adjusted to obtain the first motion trajectory, including:

[0052] receiving an update operation of a part or all of the plurality of track points input by a user;

[0053] According to the updating operation, a first motion trajectory is obtained.

[0054] This implementation supports user input operations to update the motion information of the first obstacle, improves the flexibility of obstacle updates, and improves user experience.

[0055] In combination with the first aspect, in another possible implementation of the first aspect, the scene data further includes an initial motion trajectory of the vehicle. According to the update instruction of the initial motion trajectory of the first obstacle, the first motion trajectory is obtained, including:

[0056] Determine a trajectory interaction point between an initial motion trajectory of the first obstacle and an initial motion trajectory of the ego vehicle;

[0057] According to the trajectory interaction points, derived trajectory points are obtained, and the derived trajectory points are located on the initial motion trajectory of the vehicle and are different from the positions of the trajectory interaction points;

[0058] According to the derived trajectory points, a first motion trajectory is obtained.

[0059] This implementation method can quickly obtain a new first motion trajectory by deriving the trajectory interaction points, which can significantly improve the efficiency of updating motion information, improve the user experience, and improve the generalization of the generated scene.

[0060] In combination with the first aspect, in another possible implementation of the first aspect, obtaining second motion information of the second obstacle according to the initial motion information of the second obstacle and the first motion information of the first obstacle includes:

[0061] The second motion information of the second obstacle is determined based on the initial motion information of the second obstacle, the cost of the second motion information of the second obstacle, and the first motion information of the first obstacle.

[0062] In the above implementation, the cost may be evaluated when the motion information of the second obstacle is updated, so that the motion information with a relatively small adjustment cost may be selected as the second motion information of the second obstacle as much as possible, thereby improving the rationality of the generated scene.

[0063] Optionally, the cost of the second motion information includes at least one of a authenticity cost, a smoothness cost, an obstacle avoidance cost, and a scene design cost.

[0064] Specifically, the second motion information includes a second motion trajectory of the second obstacle, and the second motion trajectory includes multiple trajectory points. The authenticity cost is used to indicate the difference between the second motion information of the second obstacle and the initial motion information of the second obstacle, and the authenticity cost is related to the coordinates, orientation, curvature and speed of the multiple trajectory points. The smoothness cost is used to indicate the smoothness of the position change of multiple trajectory points, and the smoothness cost is related to the curvature and acceleration of the second motion trajectory. The obstacle avoidance cost is used to indicate the obstacle avoidance amplitude of the second obstacle. The scenario design cost is a predefined scenario cost design. Exemplarily, the scenario design cost is used to flexibly adjust the cost of making decisions for the vehicle in different test scenarios. For example, corresponding scenario design costs can be predefined for difficult scenarios, special weather scenarios, special terrain scenarios, etc.

[0065] In combination with the first aspect, in another possible implementation manner of the first aspect, the cost J of the second motion information of the second obstacle is i It is expressed by the following formula:

[0066] J i =W X (X i -X ref ) 2 +W k k i 2 +W u u i 2 +W a a i 2 +W s slack i a +W f f hard 2

[0067] Among them, X i The second motion information of the second obstacle i is used to represent the second motion information of the second obstacle i. The second motion information of the second obstacle i may include multiple trajectory points. For example, if the number of trajectory points is represented as m, the second motion information may be represented by the position, direction, curvature, and speed of the m trajectory points. m is the number of trajectory points, and [x, y, phi, k, v] are the horizontal coordinate, vertical coordinate, orientation angle, curvature and velocity of the trajectory points respectively.

[0068] W X ,W k ,W u ,W a ,W s ,Wf Used to represent coefficients. X (X i -X ref ) 2 For authenticity cost, X ref is the initial motion information of the second obstacle i, which may also include the position, orientation, curvature and speed of multiple trajectory points, which will not be described one by one here. k k i 2 +W u u i 2 +W a a i 2 is the smoothness cost, k i represents the curvature of the second obstacle i, a i is the acceleration of the second obstacle i, u i is the derivative of the curvature of the second obstacle i (or the rate of change of curvature), W s slack i 2 is the obstacle avoidance cost, W f f hard 2 is the cost of scenario design. Here, the difficult example cost is the design cost under the difficult example scenario.

[0069] In combination with the first aspect, in another possible implementation manner of the first aspect, the second motion information satisfies one or more of the following constraints: a kinematic constraint, a collision constraint, or a driving regulation constraint, wherein:

[0070] The kinematic constraint is used to indicate the value range of the following behavior parameters in the second motion information: one or more of the velocity, acceleration, curvature of the motion trajectory, rate of change of curvature of the motion trajectory, etc.;

[0071] The collision constraint is used to constrain the second obstacle to have no collision with the vehicle and the obstacles in the first driving scene.

[0072] The driving specification constraint is used to constrain the behavior of the first obstacle to meet a predefined driving specification.

[0073] In combination with the first aspect, in another possible implementation manner of the first aspect, the scene data of the first driving scene further includes initial motion information of a fourth obstacle, and the fourth obstacle has no interaction with the vehicle and the first obstacle. Generating a target driving scene according to the first motion information of the first obstacle and the second motion information of the second obstacle includes:

[0074] A target driving scene is generated according to the first motion information of the first obstacle, the second motion information of the second obstacle, and the initial motion information of the fourth obstacle.

[0075] In a second aspect, an embodiment of the present application provides a scene generation device, the scene generation device comprising a data acquisition unit and a processing unit. The scene generation device is used to implement any method described in the first aspect.

[0076] In a third aspect, an embodiment of the present application provides a computing device, which includes a memory and a processor, wherein the memory is used to store program instructions; when the processor executes the program instructions in the memory, the computing device implements any method described in the first aspect.

[0077] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, in which instructions are stored. When the instructions are executed by at least one processor, the method described in any one of the first aspects is implemented.

[0078] In a fifth aspect, the present application provides a computer program product, which includes computer instructions. When the instructions are executed by at least one processor, the method described in any one of the first aspects is implemented.

[0079] Optionally, the computer program product may be a software installation package. The device or apparatus may download the computer program product and execute the computer program product on the device or apparatus, thereby implementing the aforementioned method.

[0080] The beneficial effects of the technical solutions provided in the second to fifth aspects of the present application can refer to the beneficial effects of the technical solution of the first aspect, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0081] The following is a brief introduction to the drawings required for describing the embodiments.

[0082] Figure 1 It is a scenario schematic diagram of a drive test data collection system;

[0083] Figure 2 It is a schematic diagram of the architecture of a scene generation system provided in an embodiment of the present application;

[0084] Figure 3 It is a flowchart of a scene generation method provided in an embodiment of the present application;

[0085] Figure 4 is a schematic diagram of a driving scenario provided in an embodiment of the present application;

[0086] Figure 5 is a schematic diagram of an obstacle category provided in an embodiment of the present application;

[0087] Figure 6 is a schematic diagram of two trajectory interaction points provided in an embodiment of the present application;

[0088] Figure 7 is a schematic diagram of two motion processes of a self-vehicle and a first obstacle provided in an embodiment of the present application;

[0089] Figure 8 is a schematic diagram of an interactive stage of entering a scene provided by an embodiment of the present application;

[0090] Fig. 9 is a schematic diagram of a trajectory change provided by an embodiment of the present application;

[0091] Fig.10 This is a schematic diagram of a scenario for updating the motion information of an obstacle provided by an embodiment of the present application;

[0092] Fig.11A This is another schematic diagram of a scenario for updating the motion information of an obstacle provided in an embodiment of the present application;

[0093] Fig. 11B This is another schematic diagram of a scenario for updating the motion information of an obstacle provided in an embodiment of the present application;

[0094] Fig.12 is a schematic diagram of a target driving scenario provided in an embodiment of the present application;

[0095] Fig.13 is a structural schematic diagram of a scene generation device provided in an embodiment of the present application;

[0096] Fig.14 It is a structural diagram of a computing device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0097] Before introducing the embodiments of the present application, the terms that may be used in the embodiments of the present application are first introduced.

[0098] 1. Obstacles refer to entities that may delay or hinder the movement of the terminal, such as objects, terrain, or facilities. Objects may include living objects or inanimate objects. In addition, the position of obstacles may be fixed or mobile.

[0099] 2. Intelligent driving, or autonomous driving, refers to the use of data collected by a perception system (such as multiple sensors) to identify or detect objects, form results through calculation and analysis, and control the terminal or assist the human driver or other decision-making body to control the computer system of the terminal. The intelligent driving algorithm refers to an algorithm that can process data to form an output result, which can be used directly to control the vehicle and / or to assist other decision-making structures (such as human drivers) to control the vehicle. Exemplarily, the output results include but are not limited to one or more of obstacle recognition results, path planning, behavioral decisions (such as following, braking, steering, etc.) and motion planning (such as acceleration, deceleration, etc.). In some schemes, the intelligent driving algorithm is integrated into the intelligent driving system, and the meanings of the two can be interchangeable.

[0100] 3. The self-vehicle refers to the vehicle that receives observation during the test, evaluation, or demonstration. Optionally, the self-vehicle in this article can be a test vehicle (or a test vehicle), which can be any vehicle with a perception system function. In some cases, intelligent driving algorithms or perception algorithms can be tested through vehicle testing. For example, in the case of testing intelligent driving algorithms, the behaviors and actions made by the self-vehicle can be regarded as the behaviors and actions made by the intelligent driving algorithm. In other words, the self-vehicle in this article can be regarded as the vehicle itself, or can be regarded as an intelligent driving algorithm.

[0101] The above terms may be optionally applied in the following embodiments.

[0102] The following first introduces the collection scenario of the scene data used in this application. Figure 1 The diagram is a scene diagram of a road test data collection system, including a vehicle 10 and a computing device 20. The vehicle 10 may include a perception system 101, which includes but is not limited to one or more of a camera, a millimeter wave radar, a laser radar, an ultrasonic radar, etc., as well as a position sensor, a posture sensor, etc. The vehicle 10 may also be provided with an intelligent driving algorithm.

[0103] When the vehicle 10 is driving, the perception system 101 can collect scene data during the driving process. Such scene information can include detection data of obstacles in the scene, such as Figure 1 As shown, the obstacles here refer to objects in the driving scene. For example, the obstacles may include inanimate objects, such as other vehicles, or roadblocks, etc. For another example, the obstacles may include living objects, such as pedestrians, or animals, etc.

[0104] The scene data collected during driving can be provided to the computing device 20. The computing device 20 is a device with data processing capabilities, which can generate scenes through scene data. The generated scene can be regarded as a simulation of the driving environment of the vehicle, which can be used to test algorithms, devices, etc., or used as model training data. For example, the generated scene can be used to test intelligent driving algorithms. For another example, the generated scene can be used to test detection devices, such as laser radars, millimeter wave radars, etc. For another example, the generated scene can be used to train models such as obstacle recognition models.

[0105] Optionally, the computing device may be a physical device, such as one or more of a host, a laptop, a handheld terminal, a computer room device, a server, etc. Alternatively, the computing device may also be a virtual device, such as a virtual machine, a container, a cloud, etc. The aforementioned servers include but are not limited to general-purpose computers, cloud servers, or blade servers, etc. When the functions of the computing device are implemented by a server, the number of servers it contains may be one or more (such as a server cluster). A virtual machine is a virtualized computing module. The cloud is a software platform that uses application virtualization technology, which enables one or more software and applications to be developed and run in an independent virtualized environment. Optionally, the cloud can be deployed on a public cloud, a private cloud, or a hybrid cloud, etc.

[0106] In some embodiments, the computing device may include a processor, where the processor includes but is not limited to a central processing unit, a graphics processor, an artificial intelligence processor, a microprocessor, or a programmable logic gate array, etc.

[0107] Figure 2 It is a schematic diagram of the architecture of a scene generation system provided in an embodiment of the present application, wherein the scene generation system includes a perception system 101 and a scene generation device 201.

[0108] The perception system 101 is used to perceive the environmental information around the autonomous driving vehicle in real-time road testing (such as the position and speed information of surrounding vehicles, pedestrians, bicycles, etc., traffic light information, roadside fences, etc.). Figure 1 Related introduction in .

[0109] The scene generation device 201 is a device with data processing capabilities, which can implement one or more operations of editing, updating (such as modification, generalization, etc.), and playing the scene. In some solutions, the above operations can be integrated into a simulator, so the scene generation device 201 of this solution can be a simulator, such as an autonomous driving simulator.

[0110] As a possible implementation, the scene generation device 201 may include one or more of a scene editing module, a scene generalization module, and a scene playback module. The module here is used to describe the function of the scene generation device, and is not used to limit the structure of the scene generation module. Among them, the scene editing module is used to update the obstacles in the scene, for example, to edit the motion information (such as motion trajectory, position, speed, acceleration, direction, action, etc.) of the obstacles in the scene. The scene generalization module is used to realize the batch generation of the trajectory or behavior parameters of the obstacles (for example, based on a generalization of a trajectory point to obtain multiple derived trajectory points), and the scene generalization module can realize the generalization modification of a single scene data to obtain a batch of new scenes. Among them, the behavior parameter indicates a parameter that can describe the behavior (behavior) or state of the obstacle, such as one or more parameters of speed, acceleration, curvature, displacement, motion mode (such as uniform motion, accelerated motion until accelerating to the target speed, decelerated motion until decelerating to the target speed, etc.), and further may include parameters derived from the aforementioned one or more parameters, such as the rate of change of curvature, etc. The scene playback module is mainly used to simulate and reproduce existing driving scenes. For example, according to the movement timeline of obstacles in the scene, it plays the images of objects in the scene changing over time.

[0111] Optionally, the scene generation system may further include a memory 202, which may store data from the perception system 101, such as structured storage of information perceived by the vehicle-mounted sensor. Further, the memory 202 may be provided in the scene generation device 201. In some embodiments, the scene generation device 201 and the memory 202 may be implemented in the computing device 20.

[0112] The following is an introduction to the method of the embodiment of the present application. Figure 3 , Figure 3 is a flow chart of a scene generation method provided in an embodiment of the present application. Optionally, the method is applied to Figure 1 The computing device 20 shown, or applied to Figure 2 The scene generation device 201 shown in FIG. 201 is shown in FIG. 201. For the convenience of description, the following is an example of an execution subject being a scene generation device.

[0113] like Figure 3 The scene generation method shown may include one or more steps from step S301 to step S304. It should be understood that for the convenience of description, the description is made in the order of S301 to S304, and it is not intended to limit the execution to the above order. The embodiment of the present application does not limit the execution order, execution time, execution number, etc. of the above one or more steps. S301 to S304 are as follows:

[0114] Step S301: Acquire scene data of a first driving scene.

[0115] Among them, the driving scene, or simply the scene, usually refers to a sequence of scenes, which can be used to describe the interaction between the self-vehicle and other objects when performing a driving task. The scene refers to a snapshot of an entity, which may include the expression of the entity itself and the connection between entities, etc. The entity includes but is not limited to one or more of the self-vehicle, static scene, dynamic environment, actor and observer. For example, a driving scene may include one or more scenes such as the self-vehicle driving through a certain road, the self-vehicle passing an intersection and interacting with other vehicles and pedestrians, overtaking, turning, or changing lanes. It should be understood that the aforementioned meaning explanations and distances are explanations made to facilitate the understanding of scenes and scenarios, and are not intended to limit this solution. In addition, the first driving scene is a distinguishing description made to facilitate the description of the driving scene. Optionally, the first driving scene can be a scene that the self-vehicle actually experiences during a road test or a simulated scene obtained by processing a scene that is actually experienced. At this time, the scene data of the first driving scene can be real road test data, and its collection process can be found in. Figure 1 Alternatively, the first driving scene may also be a simulation scene obtained through manual or automatic editing.

[0116] The first driving scene contains obstacles, which are entities in the first driving scene that may affect the travel of the vehicle. It should be noted that the obstacles may not actually affect the travel of the vehicle, but they have the possibility or ability to affect the travel of the vehicle. In this application, the number of obstacles in the first driving scene can be multiple, for example Figure 1 The driving scene shown contains obstacles such as the vehicle ahead, roadblocks, and pedestrians.

[0117] The scene data of the first driving scene may indicate the motion information of the entity in the first driving scene, where the entity includes one or more of the aforementioned obstacles, the vehicle, etc. For the sake of distinction, the motion information indicated in the scene data is referred to as initial motion information. Among them, the motion information, or action information, includes information used to describe the behavior and / or action of the entity. The motion information referred to here is considered to have attributes on a time scale. For example, the behavior can be a behavior in a broad sense, including the behavior (such as acceleration, lane change, etc.) or state (such as the current vehicle speed, the current position of the vehicle) at a specific time point. Alternatively, the behavior (or action) may also include the process of the behavior within a period of time (such as accelerating until the vehicle speed reaches the target speed). In some scenes, the behavior and action can be replaced with each other. In short, the motion information can describe the movement process of the entity in the scene and / or the state of any moment in the movement process.

[0118] Exemplarily, the initial motion information may include one or more of the following parameters: trajectory, trajectory point, position, orientation, action, behavior parameters, etc. Among them, the behavior parameter refers to the parameter that describes the behavior, action or state of the object, for example, it may include one or more of speed, acceleration, curvature, displacement, motion mode (for example, uniform motion, accelerated motion until accelerating to the target speed, decelerated motion until decelerating to the target speed, etc.). In some schemes, one or more of the aforementioned trajectory, trajectory point, position, orientation, or action, etc. may also be included in the behavior parameter. It should be noted that the motion information may also include the changes of the above parameters over time. For example, the trajectory can be represented by trajectory points (or position points) at multiple moments, or the trajectory can be represented by trajectory points, speed, speed direction, etc. at multiple moments.

[0119] Figure 4 The figure shows a schematic diagram of a driving scene provided by an embodiment of the present application, wherein the driving scene includes multiple obstacles, which are represented as obstacles o1-o9. The scene data of the driving scene may indicate the driving trajectory (or position at a certain moment), driving direction and speed of the obstacle, or the scene data of the driving scene may be processed (e.g., logsim processing) to obtain the driving trajectory (or position at a certain moment), driving direction and speed of the obstacle.

[0120] In this solution, for multiple obstacles in the first driving scene, the corresponding processing methods may be different. For example, taking the first driving scene including the first obstacle and the second obstacle as an example, the first obstacle and the second obstacle may be processed in different ways to construct the target driving scene. It is understandable that the first obstacle and the second obstacle described in this article are obstacles located in the same driving scene.

[0121] As a possible implementation, there are differences in properties or features between the first obstacle and the second obstacle. The first obstacle and the second obstacle are introduced as an example below:

[0122] Case 1: The first obstacle and the second obstacle have different interaction degrees with the vehicle. For example, the interaction degree between the first obstacle and the vehicle is not lower than the interaction degree between the second obstacle and the vehicle.

[0123] The degree of interaction can indicate the degree of influence of obstacles on the driving of the vehicle over a period of time. Figure 4 The movement change of obstacle o8 can have a great impact on the driving of the ego vehicle. For example, if the obstacle o8 is fast, the ego vehicle needs to slow down in time to avoid collision; if the obstacle o8 stops at a distance from the ego vehicle, the ego vehicle can maintain a certain speed to pass through the intersection.

[0124] In some possible cases, obstacles in the first driving scene can be divided into different categories based on the degree of interaction. Exemplarily, the category of obstacles can be determined according to the degree of interaction between obstacles and the vehicle in the first driving scene and the degree of interaction between obstacles. For example, the category of obstacles includes first-class obstacles and second-class obstacles, and the degree of interaction between the first-class obstacles and the vehicle is higher than the degree of interaction between the second-class obstacles and the vehicle. Further, according to the scene data of the first driving scene, the degree of interaction between entities in the first driving scene can be determined, wherein the entities can include obstacles and the vehicle, obstacles and obstacles, etc. Optionally, the determination of the degree of interaction can be automatically identified by an interaction degree calculation algorithm, for example, the degree of interaction between entities can be determined according to a rule-based method (such as determining the order of reaching the trajectory interaction point) and a learning-based method.

[0125] The following are several examples of rule-based methods for determining the degree of interaction. In some schemes, the degree of interaction between an obstacle and the ego vehicle can be indicated by the degree to which the obstacle's motion trajectory overlaps with the ego vehicle's motion trajectory in time and space. For example, if the position of the obstacle is closer to the position of the ego vehicle at a certain moment or time period of the ego vehicle's travel, the degree of interaction between the obstacle and the ego vehicle is higher. For another example, when there is a trajectory interaction point between the ego vehicle and the obstacle, the closer the time when the two pass through the trajectory interaction point, the higher the degree of interaction between the obstacle and the ego vehicle. Combined with Figure 4 , the ego vehicle is a straight-moving vehicle, and there is a spatial interaction point between its trajectory and that of obstacle o6, that is, both will pass through the intersection, but obstacle o6 has already passed the intersection before the ego vehicle reaches the intersection, that is, although the trajectories of the two overlap, the time interval between the positions where the trajectories overlap is relatively long, so obstacle o6 can be regarded as having no interaction with the ego vehicle. Similarly, the degree of interaction between obstacle o2 and the ego vehicle is stronger than that between obstacle o6. However, there is an interaction point between the trajectory of obstacle o1 and the ego vehicle, and the time interval between the two passing the interaction point is relatively short, so the degree of interaction between obstacle o1 and the ego vehicle is relatively high. The learning-based method can be a method for determining the degree of interaction obtained by training the model through training data, reinforcement learning, deep learning, or neural networks.

[0126] In some possible situations, the first obstacle belongs to the first category of obstacles, the second obstacle is an obstacle that interacts with the first obstacle, and / or the second obstacle is an obstacle that interacts with the ego-vehicle in addition to the first obstacle.

[0127] In some possible cases, the first obstacle belongs to the first category of obstacles, and the second obstacle includes obstacles in the second category of obstacles and / or includes obstacles in the first category of obstacles except the first obstacle. Further, the first category of obstacles can be called strong interaction obstacles, and the second category of obstacles can be called weak interaction obstacles. Figure 5 Shown is a schematic diagram of an obstacle category provided by an embodiment of the present application. The first driving scene may include strongly interactive obstacles o1, o5, o8, weakly interactive obstacles o2, o3, o4 and o7, and non-interactive obstacles o6 and o9. It should be understood that the strength here can be absolute, for example, some schemes support the use of quantitative standards to evaluate the strength of the interaction between obstacles and the vehicle. Alternatively, strength can also be relative. For example, in some schemes, the present application supports sorting the interaction procedures between the obstacles in the first driving scene and the vehicle, taking the first N with a higher degree of interaction as strongly interactive obstacles, and taking the first N+1 to M as weakly interactive obstacles, where N and M are both positive integers, and M>N.

[0128] In some possible cases, the obstacles in the first driving scene include obstacles of the first category and obstacles of the second category, wherein: the obstacles of the first category are obstacles whose degree of interaction with the ego vehicle satisfies a first condition. The obstacles of the second category include obstacles whose degree of interaction with the ego vehicle satisfies a second condition and / or obstacles that interact with a third obstacle, wherein the third obstacle is at least one obstacle in the first category, and the first condition is different from the second condition.

[0129] The following is a detailed introduction to the first condition, the second condition, and the conditions under which the interaction between two obstacles is indicated:

[0130] Design 1, the first condition includes that the time interval between the obstacle and the ego vehicle passing through the trajectory interaction point falls within the first time range. The trajectory interaction point here can be a point where the trajectories overlap, or a position where the distance between the two trajectories is less than or less than the distance threshold. See Figure 6 is a schematic diagram of two trajectory interaction points provided in an embodiment of the present application, such as Figure 6 In part (a), the obstacle o1 and the trajectory of the vehicle overlap, and the overlapping position is the interaction point 601. Figure 6In part (b), the trajectory of obstacle o5 does not overlap with the trajectory of the ego vehicle, and the shortest distance between the two trajectories is d0. If d0 is less than a preset distance threshold, the points on the two trajectories corresponding to the distance are trajectory interaction points. In other words, position 602 and position 603 are located on the trajectories of the ego vehicle and obstacle o5, respectively, and the distance between position 602 and position 603 is less than a preset distance threshold, then position 602 and position 603 are respectively used as trajectory interaction points of the ego vehicle and obstacle o5 with each other. The distance threshold can be preset or customized, such as 5 meters (m), 3m, 1m, etc.

[0131] Exemplarily, the first duration range may be less than or equal to the first duration range, for example, less than 5 seconds (s), less than or equal to 7s, etc. Figure 6 In part (a), taking the first duration range of less than or equal to 5s as an example, if the time when the vehicle passes through the trajectory interaction point 601 is t1, and the time when the obstacle o1 passes through the trajectory interaction point 601 is t2, if the time interval between t2 and t1 is less than or equal to 5s, then the obstacle o1 belongs to the first type of obstacle. Furthermore, the first duration range may also have a lower bound. For example, the first duration range may be a range that is greater than or greater than or equal to the second duration, and less than or less than or equal to the first duration. For example, the first duration range may be (0.5s, 5s]. In some cases, a duration range of less than or equal to 0.5s may be used to determine other types of obstacles.

[0132] The second condition includes that the time interval between the obstacle and the vehicle passing through the trajectory interaction point falls within the second duration range, and the lower limit of the second duration range is greater than the upper limit of the first duration range. For example, the second duration range can be a duration range greater than or equal to or greater than the third duration, where the third duration is greater than or equal to the first duration. Exemplarily, the first duration range is less than 5s, and the second duration range is greater than or equal to 5s. Furthermore, the second condition may also include an upper limit, for example, the second duration range may be (5s, 12s].

[0133] When the obstacle and the third obstacle meet the following conditions, there is an interaction between the two: there is a trajectory interaction point between the obstacle and the third obstacle, and the time interval between the two passing the trajectory interaction point falls within the third duration range. For example, the third duration range is a range less than or equal to the fourth duration, such as less than 3s, 5s, etc. For another example, the third duration range may also include a lower bound, that is, greater than or greater than the fifth duration, for example, the fourth duration range is [0.5, 3s].

[0134] Design 2, the first condition includes that there is a trajectory interaction point between the movement trajectory of the obstacle and the movement trajectory of the vehicle, and the first time interval between the two passing the trajectory interaction point is the first N of the time interval sequence, where N is an integer greater than 0.

[0135] The time interval sequence includes M time intervals arranged in order of duration, each of the M time intervals is the interval between the moment when an ego vehicle interaction obstacle passes through a first trajectory interaction point with the ego vehicle and the moment when the ego vehicle passes through the first trajectory interaction point, and the ego vehicle interaction obstacle is an obstacle that has a trajectory interaction point with the ego vehicle in the first driving scene. Please refer to Table 1, which is an interaction relationship between an obstacle and the ego vehicle in a first driving scene provided by an embodiment of the present application, wherein the first driving scene is Figure 4 In the driving scenario, obstacles o3 and o9 do not have trajectory interaction points with the ego vehicle, and can be regarded as having no interaction with the ego vehicle. The remaining obstacles, namely obstacles o1, o2, o4, o5, o6, o7, and o8, all have trajectory interaction points with the ego vehicle, so the above obstacles belong to ego vehicle interaction obstacles. The time intervals between the moments when the above ego vehicle interaction obstacles pass through their respective trajectory interaction points with the ego vehicle and the moments when the ego vehicle passes through the corresponding trajectory interaction points are shown in Table 1. The time intervals can be sorted from small to large in length to form the following sequence (for the convenience of description, the obstacles corresponding to the time intervals are written in brackets): 1.50s (o1), 2.80s (o5), 3s (o8), 5.01s (o2), 5.12s (o4), 5.15s (o7), 12.50s (o6), at this time M = 7. Among them, the obstacles ranked in the top 3 (i.e., N=3) in terms of time interval are first-class obstacles, i.e., obstacles o1, o5, and o8 are first-class obstacles. It is not difficult to see that the first N time intervals in the time interval sequence are less than (or less than or equal to) the time intervals other than the first N time intervals in the time interval sequence. It should be understood that the aforementioned order is only an example. If the order is sorted from large to small, the first obstacle is the obstacle ranked in the last N positions and is a first-class obstacle.

[0136] Table 1

[0137]

[0138]

[0139] Furthermore, the second condition includes that the obstacle's motion trajectory and the vehicle's motion trajectory have a trajectory interaction point and the first time interval between the two passing the trajectory interaction point falls within the first N+1 to K of the time interval sequence, K is an integer and K>N, and M≥K or M>K. Continuing with Table 1, the second category of obstacles can include the first 4 to 6 obstacles, that is, N=3 and K=6, that is, the second category of obstacles can be obstacles o2, o4, and o7.

[0140] Furthermore, the second category of obstacles also includes obstacles that interact with the first category of obstacles, wherein the third obstacle is at least one obstacle in the first category of obstacles. The obstacles that interact here may exemplarily include obstacles that meet at least one of the following two conditions: there is a trajectory interaction point with the third obstacle and the time interval between the trajectory interaction point falls within the third time range, for example, the third time range is 0-3s; or multiple obstacles may be sorted by the degree of interaction between the obstacles and the third obstacle, so that one or more obstacles are selected as obstacles that interact with the third obstacle in descending order of the degree of interaction.

[0141] Exemplary, combined Figure 4 Taking case 1 as an example, the third obstacle may be obstacle o1 (o1 is a first-category obstacle), there is a trajectory interaction point between obstacle o1 and obstacle o3, and the time interval between the two passing through the trajectory interaction point is less than the fourth time length (such as 5s), then there is an interaction between obstacle o1 and obstacle o3, and therefore obstacle o3 belongs to the second-category obstacle.

[0142] It should be understood that the above two designs are only for the convenience of describing the classification of obstacles, so the two designs for distinguishing the types of obstacles are introduced as examples. The above two designs can also be combined. One combination example is as follows: from M time intervals, obstacles corresponding to the first N time intervals that fall within the first time range are taken as the first type of obstacles. If the number of time intervals that fall within the first time range is less than N, only obstacles whose corresponding time intervals fall within the first time range are taken as the first type of obstacles. Of course, there can be other combined designs in the specific implementation process, which will not be exemplified here one by one.

[0143] In some possible cases, the obstacles in the first driving scene may also include a third type of obstacle, which may also be called a background obstacle. The third type of obstacle has no interaction with the ego vehicle and no interaction with the first type of obstacle. Exemplarily, the third type of obstacle has no track interaction point with the ego vehicle and no track interaction point with the first type of obstacle. In combination with the aforementioned design 1, if obstacle o6 has a track interaction point with the ego vehicle, but the time after the track interaction is greater than the upper limit of the second duration range (i.e., 12s), then obstacle o6 belongs to the third type of obstacle; or, in combination with the aforementioned design 2, if obstacle o6 has a track interaction point with the ego vehicle, but the time interval after the track interaction does not belong to the first K positions, then obstacle o6 belongs to the third type of obstacle. Obstacle o9 has no track interaction point with the ego vehicle and no track interaction point with the first type of obstacle, and is a third type of obstacle. In some possible cases, the third type of obstacle may also be other obstacles in the first driving scene except the ego vehicle, the first type of obstacle, and the second type of obstacle.

[0144] Optionally, the motion trajectory used in the aforementioned determination of the interaction type may be a motion trajectory within a predefined time period. For example, the vehicle may continue to travel, and the first driving scene may be a scene within a local time period within the duration of the travel.

[0145] Case 2: The first obstacle and the second obstacle have different interest levels. The interest level is determined by the user or the algorithm. For example, the first obstacle is an obstacle selected by the user. Since it is selected (e.g., selected or edited) by the user, the first obstacle has a higher interest level than other obstacles. For another example, the first obstacle and the second obstacle are both obstacles selected by the user, but the motion trajectory of the first obstacle can be freely edited, while the trajectory of the second obstacle does not need to be edited by the user. In this case, the first obstacle has a higher interest level than the second obstacle.

[0146] As a possible implementation, the first obstacle may be selected by a user or an algorithm (e.g., a rule-based obstacle selection algorithm or a machine learning-based obstacle selection algorithm). For example, the user selects a rabbit obstacle, also called an actor car, from multiple obstacles in the first driving scene, and the user can control the rabbit obstacle so that it performs a designed interactive action with the vehicle. At this time, the scene generation device may receive a selection operation or an editing operation of a certain obstacle or several obstacles by the user, and the selected obstacle is the first obstacle.

[0147] Optionally, in this case, the second obstacle may be specified by the user, or automatically determined. Exemplarily, the user may also select some obstacles from multiple obstacles as the second obstacle. Exemplarily, the second obstacle may be determined in the following manner: when the trajectory of the specified rabbit obstacle changes, an obstacle that may conflict with its trajectory is used as the second obstacle, or an obstacle other than the first obstacle that may conflict with the trajectory of the ego vehicle is used as the second obstacle. That is, the second obstacle is an obstacle that interacts with the first obstacle, or the second obstacle is an obstacle that interacts with the ego vehicle other than the first obstacle.

[0148] Further, in case 2, the first driving scene may include background obstacles in addition to the first obstacle and the second obstacle, and the background obstacles are obstacles that do not interact with the first obstacle and the vehicle. In some implementations, the background obstacles may be other obstacles in the first driving scene in addition to the vehicle, the first obstacle, and the second obstacle.

[0149] In case 3, the first obstacle may be any obstacle in the first driving scene, and the second obstacle may be an obstacle that interacts with the first obstacle, or the second obstacle may be an obstacle that interacts with the vehicle other than the first obstacle. Optionally, the first driving scene may also include background obstacles other than the vehicle, the first obstacle, and the second obstacle.

[0150] It should be understood that the above three situations are exemplary explanations for facilitating understanding of obstacles and are not strict limitations on the solution. The above two situations may also be combined, for example, the first obstacle may be a strong interaction obstacle specified by the user, and the second obstacle may be a strong interaction obstacle and / or a weak interaction obstacle that conflicts with the trajectory of the first obstacle.

[0151] It should be understood that the first obstacle may be one or more obstacles, and the second obstacle may also be one or more obstacles. In addition, in combination with the above description of the driving scene data, when the first driving scene includes the first obstacle and the second obstacle, the scene data of the first driving scene may include the initial motion information of the first obstacle and the initial motion information of the second obstacle.

[0152] As a possible implementation, the first driving scene may be obtained based on data collected from the vehicle, and the scene data of the first driving scene may also include motion information of the vehicle. Optionally, the initial motion information of the vehicle may be collected by sensors of the vehicle, such as one or more of a wheel speed meter, an inertial measurement unit, a positioning system, a radar sensor, an image sensor, and the like.

[0153] Optionally, the scene data of the driving scene acquired in step S301 may be based on sensor data (e.g. Figure 1 and Figure 2 For example, the sensor data is processed by logsim to obtain the motion information of the obstacle and the vehicle in the first driving scene.

[0154] Step S302: adjusting the initial motion information of the first obstacle to generate first motion information of the first obstacle.

[0155] Specifically, similar to the initial motion information, the first motion information may include one or more of the following parameters: one or more of the motion trajectory, trajectory points, position, orientation, action, behavior parameters, etc. of the first obstacle.

[0156] The following is an exemplary introduction to the implementation method of adjusting the initial motion information of the first obstacle to obtain the first motion information:

[0157] Implementation method 1: Adjust the values ​​or thresholds of the multiple behavior parameters of the first obstacle to generate the first motion information of the first obstacle. For the description of the behavior parameters, refer to the above, and the first motion information includes the updated values ​​of the multiple behavior parameters and / or the updated thresholds of the behavior parameters.

[0158] Taking the update of parameter values ​​as an example, referring to Table 2, the initial motion information may include values ​​of parameters such as absolute speed (e.g., target speed), relative speed (e.g., relative to the vehicle speed), and relative distance. The update indication may include new values ​​of some or all of the parameters in the initial motion information (i.e., updated values). The scene generation device may obtain the first motion information according to the update indication, and the first motion information includes new values ​​of some or all of the parameters in the aforementioned initial motion information. Of course, when the value of a certain parameter is not updated, the value of the parameter in the first motion information is the same as the value in the initial motion information. For example, the updated value of the obstacle model in Table 2 is the same as the value in the initial motion information.

[0159] Table 2

[0160] Initial motion information Value Updated value Target speed v1 v2 Distance to the vehicle d1 d2 Relative to the speed of the vehicle vr1 Vr2 Obstacle Model Car following model Car following model …… …… ……

[0161] See also Figure 7 , Figure 7 is a schematic diagram of two motion processes of a self-vehicle and a first obstacle provided in an embodiment of the present application, see Figure 7 In part (a), the motion information of the first obstacle may include the behavior parameters of the first obstacle at time t4, such as speed v=40 kilometers per hour (km / h). The behavior parameters may be adjusted, see Figure 7 In part (b), after adjustment, the speed of the first obstacle at time t4 is 35 km / h. For another example, the motion information of the first obstacle may include a threshold value of a behavior parameter of the first obstacle for changing lanes, see Figure 7 In the first driving scenario, when the relative distance between the first obstacle and the vehicle reaches d1, the first obstacle performs a lane change operation. Figure 7 In part (b), the threshold of the relative distance can be updated to d2, and when the relative distance between the first obstacle and the vehicle is d2, the first obstacle performs a lane change operation. Further, the obstacle model can also be changed, for example, the lane change model after time t4 can be updated to an S-turn model.

[0162] As a possible implementation, the initial motion information of the first obstacle includes a first action set and a first trigger set, and the adjustment of the initial motion information of the first obstacle includes adjusting the behavior parameters in the first action set and / or adjusting the values ​​of the behavior parameters in the first trigger set. Among them, the first action set is used to describe the first action of the first obstacle, including the value of the first behavior parameter of the first action. The first trigger set is used to describe the conditions for performing the first action. Exemplarily, the first action may include one or more of changing lanes, overtaking, following a vehicle, turning, braking, accelerating, decelerating, reversing, entering the next action stage, etc. It should be understood that the parameters of the first action set may be different from the parameters in the first trigger set. Please refer to Table 3, which is information on behavior parameters that may be included in an action set provided in an embodiment of the present application, wherein a behavior label can be used to represent a behavior parameter, and its name and meaning can be found in the description of the first column of Table 3.

[0163] Table 3 Action set

[0164]

[0165] Table 4 is a behavior parameter information that may be included in a trigger set provided in an embodiment of the present application. The conditional label is used to represent the behavior parameter in the trigger set. The name and meaning of the conditional label can be found in the description of the first column of Table 4.

[0166] Table 4 Trigger Set

[0167]

[0168] Accordingly, the scene generation device adjusts the first action set and / or the first trigger set of the first obstacle to generate the first motion information of the first obstacle. Figure 7 , the first action set can be the action set of the first obstacle at time t4, and the first action set can include absolute speed (i.e. objTargetSpeed ​​in Table 3), relative speed (i.e. objectRelSpeed ​​in Table 3), and running to target speed with acceleration (objTargetSpeed ​​and objectAcc in Table 3). Further, the lane change mode of the first obstacle can be selected, and the interaction model satisfied by the first obstacle can be set. After updating the value of the first action set, as Figure 7 As shown in part (b), the motion process of the first obstacle changes accordingly. The trigger set is similar, and examples are not given here one by one.

[0169] In one case, the first action set and the first trigger set may be the action set and the trigger set corresponding to any moment or any stage of the first obstacle. In this case, the solution can support editing the state and behavior of the obstacle at any moment or any stage of the first obstacle's movement, thereby improving the flexibility of scene generation.

[0170] As another possible situation, there is an interaction between the first obstacle and the ego vehicle, and the interaction between the first obstacle and the ego vehicle can be decomposed into at least one interaction stage. The action of the first obstacle in each interaction stage can be described by an action set, and the triggering method of the interaction stage can be described by a trigger set. The aforementioned first action set and first trigger set can be an action set and a trigger set in a certain interaction stage or in several interaction nodes. The division of the interaction stage can be related to the type of interaction, and the interaction type can include one or more of the following interaction types: cut-in, cross, merge, lane-change, etc. See Figure 8 , Figure 8 It is a schematic diagram of the interactive stages of a cut-in scene provided by an embodiment of the present application. The cut-in scene may include a car chasing stage, a speed regulating stage, and a cut-in execution stage. In the car chasing stage, the social car (regarded as the first obstacle) may be designed to accelerate so that the relative distance between it and the self-car reaches the chase distance, triggering entry into the next stage; in the speed regulating stage, the social car may be designed to adjust its speed (e.g., accelerate or decelerate) so that the relative distance between it and the self-car reaches the overtake distance, triggering entry into the next stage; in the cut-in stage, the social car may change lanes and insert itself in front of the self-car. It is not difficult to see that each stage may be represented by a corresponding action set (e.g., speed, acceleration, motion model, etc.) and a trigger set (e.g., relative distance, relative speed, etc.). That is, the first action set and the first trigger set may be the corresponding action set and trigger set in any one or several of the aforementioned stages.

[0171] In this case, the scene generation device can identify the interaction category between the first obstacle and the self-vehicle based on the initial motion information of the first obstacle and the initial motion information of the self-vehicle, and determine at least one interaction stage based on the interaction category corresponding to the first obstacle. Further, the scene generation device adjusts the action set and trigger set parameters under the first interaction stage to obtain the first motion information of the first obstacle. The first interaction stage may be part or all of at least one interaction stage. In addition, when identifying the interaction category, the scene generation device can identify the interaction category between the first obstacle and the self-vehicle based on the scene data of the first driving scene.

[0172] As a possible implementation, the identification of the interaction type can be a rule-based method or a learning-based method. Exemplarily, Table 5 lists the interaction types and the characteristics of the interaction types. The scene generation device can be based on the motion information between the first obstacle and the ego vehicle. When the motion information between the two indicates that the motion relationship between the two satisfies a certain feature, the interaction type is the interaction type corresponding to the feature. For example, if it is determined through the initial motion information of the obstacle and the initial motion information of the ego vehicle that the obstacle cuts in from the lane parallel to the ego vehicle to the front of the ego vehicle, it indicates that the interaction type between the obstacle and the ego vehicle is a cut-in type.

[0173] Table 5

[0174]

[0175] Furthermore, each interaction type may include one or more interaction stages. For example, merge may include two stages: speed regulation and import, and lane-change may include one stage. It should be noted that Table 5 is an exemplary interaction type and its characteristics made to facilitate understanding of the identification of interaction types. In the specific implementation process, the identification of interaction types can also be achieved based on other rules. In addition, the number and name of the interaction stages corresponding to the interaction type may also have other designs, for example, more or fewer names may be included. This application does not strictly limit the interaction types, their characteristics, and the included interaction stages.

[0176] As a possible implementation, the above adjustment may be obtained based on an operation input by a user. For example, the user manually inputs the updated value or value of the behavior parameter. Accordingly, the scene generation device receives the value or threshold of the above behavior parameter input by the user.

[0177] As another possible implementation, the above adjustment may be obtained through the output of an algorithm or an AI model. For example, the scene generation device may receive the values ​​of multiple sets of behavior parameters (such as action sets, trigger sets, etc.) output by the algorithm or AI model. At this time, a first motion information of the first obstacle may be obtained according to the values ​​of each set of behavior parameters. Among them, the values ​​of the multiple sets of behavior parameters output by the model are different from the values ​​of the original behavior parameters, so that the motion process of the first obstacle changes. For example, in the original driving scene (i.e., the first driving scene), the first obstacle turns left at a speed of 30 yards, and the AI ​​model can adjust the speed when turning left to 25, 35, 40, 45, and 50 yards, thereby obtaining at least 4 updated motion information, so that in the at least 4 new scenes obtained, the first obstacle has more possible turning radii when turning left. In short, the generalized modification of the behavior parameters of the first obstacle can be achieved through algorithms or AI, so that the motion information of multiple obstacles can be quickly generalized from one driving scene, so that multiple derivative scenes can be quickly obtained later. Exemplary, combined with Figure 2 , the generalization process can be implemented by the scene generalization module in the scene generation device 201.

[0178] Implementation method 2: The initial motion trajectory of the first obstacle is adjusted to obtain a first motion trajectory, and the first motion information includes the first motion trajectory.

[0179] See also Fig. 9 , Fig. 9 It is a schematic diagram of a trajectory change provided in an embodiment of the present application, wherein the solid line with an arrow is the initial motion trajectory of the obstacle, and the dotted line with an arrow is the second motion trajectory (i.e., the updated motion trajectory). Taking obstacle o5 as an example, its updated motion trajectory has a larger turning radius.

[0180] In some scenarios, the motion trajectory includes multiple trajectory points, and the position of at least one trajectory point among the multiple trajectory points may be updated to form a new trajectory.

[0181] As a possible implementation, the updating of the track points may be indicated by an operation input by a user. Exemplarily, the scene construction device may receive an updating operation input by a user for some or all of the track points, and obtain the first motion track according to the updating operation. Fig.10 , Fig.10: is a scene schematic diagram of updating the motion information of an obstacle provided by an embodiment of the present application, wherein the arrow with a solid line is the initial motion trajectory of the obstacle o5, which may include multiple trajectory points, such as trajectory point 1001, etc. The scene generation device may output an operation interface, and the operation interface may include the initial motion trajectory of the obstacle o6 and the trajectory points on the trajectory. The user may perform operations on the operation interface, such as one or more of information input operations, click operations, drag operations, and long press operations, and the information input operations may include inputting the horizontal coordinate, vertical coordinate, or distance from the starting point of the trajectory point 1001, and the drag operation may include dragging the trajectory point 1001 to another position. Accordingly, the scene generation device receives the user's operation instruction, and the operation instruction includes the target position to which the trajectory point 1001 is dragged, so that the updated trajectory of the obstacle o5 can be obtained according to the target position to which the trajectory point 1001 is dragged. Of course, the scene generation device may record the user's position adjustment of one or more trajectory points, and generate the updated trajectory of the obstacle o5 when the user confirms that the new trajectory is obtained (for example, when the user indicates to save or the user submits the modification).

[0182] As another possible implementation, the update of the trajectory point can be indicated by the output of the algorithm or AI model. Exemplarily, the scene generation device can calculate the key trajectory points in the trajectory points of the initial trajectory of the first obstacle, generate one or more derived interaction points based on the key trajectory points (this process can also be called a generalization process), and thus obtain one or more motion trajectories of the first obstacle based on the one or more derived interaction points. Among them, the key trajectory point is, for example, the interaction point between the first obstacle and the motion trajectory of the vehicle, or the turning point, deceleration point, etc. of the first obstacle, and the determination of the key point is not strictly limited here. Taking the key trajectory point as the trajectory interaction point as an example, the scene construction device can determine the trajectory interaction point between the initial motion trajectory of the first obstacle and the initial motion trajectory of the vehicle, and obtain the derived trajectory point according to the trajectory interaction point, wherein the derived trajectory point is a trajectory point located on the initial motion trajectory of the vehicle and different from the trajectory interaction point, and the first motion trajectory is obtained based on the trajectory derivative point. For example, after obtaining the derived trajectory point, the first obstacle trajectory interaction point and part or all of the trajectory points other than the trajectory interaction point are smoothly connected to obtain the first motion trajectory.

[0183] Furthermore, before generating the motion trajectory based on the derived trajectory points, the derived trajectory points may also undergo conditional screening. In other words, only the derived trajectory points that meet the preset conditions will be used to generate the motion trajectory of the first obstacle, otherwise the derived trajectory points may be discarded or updated. For example, the scene generation device can determine whether the derived trajectory points exceed the road boundary, whether the motion trajectory obtained based on the derived trajectory points collides with other obstacles (or the vehicle), etc. In a possible implementation, when the derived trajectory points do not exceed the road boundary and the motion trajectory obtained based on the derived trajectory points does not collide with other obstacles (or the vehicle), the first motion trajectory of the first obstacle is obtained based on the derived trajectory points.

[0184] In some solutions, the derived interaction points derived from the same key trajectory point, or the derived interaction points and the initial key trajectory point, meet certain distribution rules. Taking the key trajectory point as the trajectory interaction point between the first obstacle and the ego vehicle as an example, a distribution rule can be as follows: along the motion trajectory of the ego vehicle, the distance between any two interaction points is equal. The interaction points here include derived interaction points and basic interaction points. See Fig.11A and Fig. 11B , Fig.11A and Fig. 11B is another scenario diagram of updating the motion information of an obstacle provided by an embodiment of the present application. Among them, the key trajectory points are as follows Fig.11A and Fig. 11B The hollow squares are shown in Figure 2, and the derived trajectory points are derived based on the key trajectory points as shown in Figure 2. Fig.11A and Fig. 11B As shown in the solid circle, it can be seen that there is a certain distance between the derived trajectory points and the trajectory key points, and between the derived trajectory points.

[0185] Optionally, there may be multiple first obstacles. For example, the trajectory of the multiple obstacles may be updated or the behavior parameters may be updated. For another example, the scene generation device may automatically derive the motion trajectory of each obstacle in the multiple obstacles, or update the behavior parameters of each obstacle in the multiple obstacles.

[0186] Step S303: obtaining second motion information of the second obstacle according to the initial motion information of the second obstacle and the first motion information of the first obstacle.

[0187] Specifically, after the motion information of the first obstacle is updated, the motion information of the second obstacle can be updated according to the updated motion information of the first obstacle (i.e., the second motion information). When the motion information of the second obstacle is updated, it can be updated based on the initial motion information and combined with the first motion information of the first obstacle, so that the motion trajectory of the second obstacle can be as close to its original trajectory as possible without colliding with the second obstacle, thereby improving authenticity.

[0188] As a possible implementation, the trajectory update of the second obstacle can be achieved through an AI model. For example, the scene generation device inputs the initial motion information of the second obstacle and the first motion information of the first obstacle into the AI ​​model to obtain the second motion information of the second obstacle. Specifically, the AI ​​model is trained through the motion information of obstacles in multiple driving scenes, and can achieve joint optimization of multiple obstacles. Furthermore, the motion information of the obstacle output by the AI ​​model can satisfy one or more of the following constraints: kinematic constraints, collision constraints, or driving specification constraints, etc. (introduced below). Furthermore, the AI ​​model can also update the motion information of the second obstacle at the lowest cost. For a description of the cost, please refer to the description below.

[0189] As a possible implementation, for the trajectory update of the second obstacle, the optimal motion information may be obtained by calculating the updated trajectory cost.

[0190] Specifically, the scene generation device may determine the second motion information of the second obstacle based on the initial motion information of the second obstacle, the cost of the second motion information of the second obstacle, and the first motion information of the first obstacle. When calculating the cost, the cost of the second motion information can be expressed in an algebraic form, so that the scene generation device can obtain a set of algebraic solutions by a numerical solution method, thereby obtaining the motion trajectory of the second obstacle.

[0191] Among them, the cost of the second motion information includes at least one of authenticity cost, smoothness cost, obstacle avoidance cost and scene design cost. Specifically, the second motion information includes the second motion trajectory of the second obstacle, and the second motion trajectory includes multiple trajectory points. The authenticity cost is used to indicate the difference between the second motion information of the second obstacle and the initial motion information of the second obstacle, and the authenticity cost is related to the coordinates, orientation, curvature and speed of the multiple trajectory points. The smoothness cost is used to indicate the smoothness of the position change of the multiple trajectory points, and the smoothness cost is related to the curvature and acceleration of the second motion trajectory. The obstacle avoidance cost is used to indicate the obstacle avoidance amplitude of the second obstacle. The scene design cost is a predefined scene cost design. Exemplarily, the scene design cost is used to flexibly adjust the cost of making decisions in different test scenarios of the vehicle. For example, for difficult scenes, special weather scenes (such as rainy scenes, blizzard scenes, foggy scenes, etc.), special terrain scenes (such as winding mountain road scenes, desert area scenes, high temperature scenes, frozen soil scenes, etc.), etc., the corresponding scene design costs can be predefined.

[0192] Optionally, when the second obstacle includes multiple obstacles, the cost of the second motion information of the second obstacle is the sum of the costs of the second motion information of the multiple obstacles.

[0193] Exemplarily, when the second obstacle includes one obstacle (the obstacle i is taken as an example below), the cost J of the second motion information of the second obstacle is i It can be calculated by the following formula:

[0194] J i =W X (X i -X ref ) 2 +W k k i 2 +W u u i 2 +W a a i 2 +W s slack i 2 +W f f hard 2

[0195] Among them, X iThe second motion information of the second obstacle i is used to represent the second motion information of the second obstacle i. The second motion information of the second obstacle i may include multiple trajectory points. For example, if the number of trajectory points is represented as m, the second motion information may be represented by the position, direction, curvature, and speed of the m trajectory points. m is the number of trajectory points, and [x, y, phi, k, v] are the horizontal coordinate, vertical coordinate, orientation angle, curvature and velocity of the trajectory points respectively.

[0196] W X ,W k ,W u ,W a ,W s ,W f Used to represent coefficients. X (X i -X ref ) 2 For authenticity cost, X ref is the initial motion information of the second obstacle i, which may also include the position, orientation, curvature and speed of multiple trajectory points, which will not be described one by one here. k k i 2 +W u u i 2 +W a a i 2 is the smoothness cost, k i represents the curvature of the second obstacle i, a i is the acceleration of the second obstacle i, u i is the derivative of the curvature of the second obstacle i (or the rate of change of curvature), W s slack i 2 is the obstacle avoidance cost, W f f hard 2 The hard case cost is the design cost under the hard case scenario, that is, W f f hard 2 It belongs to the cost of scene design.

[0197] As another example, when the second obstacle includes n (n is an integer > 1) obstacles, the cost of the second motion information of the second obstacle may satisfy the following formula:

[0198]

[0199] Among them, J iRepresents the cost of the second motion information of the i-th obstacle among n (n is an integer and n>0) obstacles included in the second obstacle, i=1, 2, 3 . . . and i≤n.

[0200] In some possible cases, the second motion information also satisfies one or more of the following constraints: kinematic constraints, collision constraints, or driving specification constraints, etc. The kinematic constraints are used to indicate the value ranges of the following behavior parameters in the second motion information: curvature, speed, acceleration, rate of change of curvature, etc. The collision constraints are used to constrain the second obstacle to have no collision with other objects, where the other objects may include the vehicle, one or more obstacles in the first driving scene (such as the first obstacle), etc. The driving specification constraints are used to constrain the behavior of the first obstacle to meet predefined driving specifications.

[0201] The following is an exemplary introduction to each of the above constraints:

[0202] (1) Kinematic constraints are used to constrain the range of curvature, velocity, acceleration, curvature derivative, etc., so as to ensure that the driving trajectory is more reasonable and conforms to the kinematic constraints. For example, the curvature k, velocity v, acceleration a, curvature derivative u, etc. of the second obstacle fall within the following ranges:

[0203] k≥kappaLowBound,k≤kappaUpperBound

[0204] v≥vLowBound,v≤vUpperBound

[0205] u≥uLowBound,u≤uUpperBound

[0206] a≥aLowBound,a≤aUpperBound

[0207] Among them, kappaLowBound and kappaUpperBound are used to indicate the threshold of curvature, vLowBound and vUpperBound are used to indicate the threshold of velocity. The rest of the parameters are similar and will not be described here one by one.

[0208] (2) The motion information of the second obstacle needs to conform to the kinematic model. For example, the displacement x, displacement y, time t, curvature k, velocity v, acceleration a, curvature change rate u, θ, etc. of the second obstacle must satisfy the following formula:

[0209] x · (t) = v(t)cos(θ(t))

[0210] y · (t) = v(t)sin(θ(t))

[0211] θ · (t) = v(t)κ(t)

[0212] κ · (t) = u(t)

[0213] v · (t) = a(t)

[0214] (3) Collision constraint: to ensure that the second obstacle does not collide with other obstacles.

[0215] The following introduces the constraints for achieving non-collision between the second obstacles, between the second obstacle and the first obstacle, and between the second obstacle and the ego vehicle. Taking the second obstacle as an example, the positions of the two obstacles are represented by (x1, y1) and (x2, y2), the position of the ego vehicle is represented by (egox, egoy), and the position of the first obstacle is represented by (keyObjx, keyObjy).

[0216] Exemplarily, the non-collision between the second obstacles can be expressed by the following formula:

[0217] (x1-x2)*(x1-x2)+(y1-y2)*(y1-y2)≥slack

[0218] slack≥slackLower,slack≤slackUpper

[0219] Wherein, slackLower represents the lower threshold value when collision is identified, and slackUpper represents the upper threshold value when collision is identified. The aforementioned threshold values ​​may be predefined or predesigned.

[0220] Exemplarily, the non-collision between the second obstacle and the vehicle can be expressed by the following formula:

[0221] (x1-egox)*(x1-egox)+(y1-egoy)*(y1-egoy)≥ slackEgo

[0222] slackEgo≥slackEgoLower,slackEgo≤slackEgoUpper

[0223] Exemplarily, the non-collision between the second obstacle and the first obstacle can be expressed by the following formula:

[0224] (x1-keyObjx)*(x1-keyObjx)+(y1-keyObjy)*(y1-keyObjy)≥ slackKeyObj

[0225] slackKeyObj≥slackKeyObjLower,slackKeyObj≤slackKeyObjUpper

[0226] (4) Driving standard constraint, which is used to constrain the driving trajectory of the second obstacle to comply with the road constraint, such as not compacting the road edge. Exemplarily, a driving standard constraint of the second obstacle (x1, y1) can be expressed by the following formula:

[0227] -slackNormal≤(x1-xNormal)*sin(thetaNormal)-(y1-yNormal)*cos(thetaNormal)≤slackNoraml.

[0228] Among them, slackNormal, xNormal, yNormal, etc. are used to represent parameters related to road specifications.

[0229] As a possible implementation manner, the scene generation device may solve the aforementioned cost design, or the aforementioned one or more constraints, through a solver, to obtain the second motion information of the second obstacle.

[0230] Step S304: Generate a target driving scene according to the first motion information of the first obstacle and the second motion information of the second obstacle.

[0231] Optionally, the target driving scenario can be used to test an intelligent driving algorithm, or the target driving scenario can be used as model training data, for example, for training an obstacle recognition model, or for training an autonomous driving decision model.

[0232] In one possible scenario, the target driving scene can be played in a scene generation device. When playing the target driving scene, in the target driving scene, the first obstacle moves through the motion process indicated by the first motion information. The motion process here includes the motion trajectory and the behavior parameters during the motion process. Correspondingly, the second obstacle moves through the motion process indicated by the second motion information. Figure 2 , the scene playing function can be realized by the scene playing module in the scene generating device 201.

[0233] For example, see Fig.12 , Fig.12 is a schematic diagram of a target driving scenario provided by an embodiment of the present application, wherein the arrow represents the initial motion trajectory, and the dotted line with an arrow represents the motion trajectory obtained after adjustment.

[0234] Optionally, the scene data of the first driving scene further includes initial motion information of a fourth obstacle. The scene generation device may generate a target driving scene according to the first motion information of the first obstacle, the second motion information of the second obstacle, and the initial motion information of the fourth obstacle.

[0235] Further optionally, the fourth obstacle has no interaction with the ego vehicle and the first obstacle. For example, the fourth obstacle may belong to the third category of obstacles (ie, background obstacles).

[0236] The method of the embodiment of the present application is described above, and the device of the embodiment of the present application is provided below.

[0237] It is understandable that the multiple devices provided in the embodiments of the present application, such as scene generation devices, computing devices, etc., in order to implement the functions in the above-mentioned method embodiments, include hardware structures, software units, or combinations of hardware structures and software structures corresponding to the execution of each function. It should be easily appreciated by those skilled in the art that, in combination with the various functions described in the embodiments disclosed herein, the device and the modules in the device can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different device implementations to implement the aforementioned method embodiments in different usage scenarios, and different implementations of the device should not be considered to exceed the scope of the embodiments of the present application.

[0238] Several possible arrangements are listed below.

[0239] See also Fig.13 , Fig.13 1 is a schematic diagram of a scene generation device provided in an embodiment of the present application. The scene generation device 130 may include an acquisition unit 1301 and a processing unit 1302. The scene generation device 130 is used to implement the aforementioned scene generation method, for example Figure 3 The scene generation method in the illustrated embodiment.

[0240] In a possible implementation, the acquisition unit 1301 is used to acquire scene data of a first driving scene, where the scene data is used to indicate initial motion information of an obstacle in the first driving scene, where the obstacles in the first driving scene include a first obstacle and a second obstacle;

[0241] The processing unit 1302 is used for:

[0242] Adjusting initial motion information of the first obstacle to generate first motion information of the first obstacle;

[0243] Obtaining second motion information of the second obstacle according to the initial motion information of the second obstacle and the first motion information of the first obstacle;

[0244] A target driving scene is generated according to the first motion information of the first obstacle and the second motion information of the second obstacle.

[0245] In a possible implementation manner, the scene data further includes initial motion information of the vehicle.

[0246] In a possible implementation, the processing unit 1302 is further configured to obtain second motion information of the second obstacle according to the initial motion information of the vehicle, the initial motion information of the second obstacle, and the first motion information of the first obstacle.

[0247] In a possible implementation manner, the degree of interaction between the first obstacle and the ego vehicle is higher than or not lower than the degree of interaction between the second obstacle and the ego vehicle.

[0248] In a possible implementation, the processing unit 1302 is further configured to: adjust the initial motion information of the first obstacle according to the user input to generate the first motion information of the first obstacle.

[0249] In a possible implementation, the processing unit 1302 is further configured to: adjust the initial motion information of the first obstacle through an artificial intelligence model to generate first motion information of the first obstacle.

[0250] In a possible implementation manner, the second obstacle is an obstacle that interacts with the first obstacle.

[0251] In a possible implementation manner, the second obstacle is an obstacle other than the first obstacle that interacts with the vehicle.

[0252] In a possible implementation, the processing unit 1302 is further configured to: determine the category of the obstacle according to the degree of interaction between the obstacle and the vehicle and the degree of interaction between the obstacles in the first driving scene, wherein the category of the obstacle includes a first category of obstacles and a second category of obstacles;

[0253] The interaction degree between the first type of obstacles and the ego vehicle is higher than that between the second type of obstacles and the ego vehicle.

[0254] Optionally, the first obstacle belongs to a first category of obstacles, and the second obstacle includes at least one obstacle in the second category of obstacles and / or includes at least one obstacle in the first category of obstacles other than the first obstacle. In some embodiments, the first category of obstacles may be referred to as strong interaction obstacles, and the second category of obstacles may be referred to as weak interaction obstacles.

[0255] In a possible implementation manner, the obstacles in the first driving scene include first-category obstacles and second-category obstacles. The explanations of the first-category obstacles and the second-category obstacles refer to the above.

[0256] In a possible implementation, the initial motion information of the first obstacle includes multiple behavior parameters of the first obstacle, and the multiple behavior parameters are used to describe the behavior of the first obstacle. The processing unit 1302 is further used to: adjust the values ​​or thresholds of the multiple behavior parameters of the first obstacle to generate the first motion information of the first obstacle.

[0257] In a possible implementation, the initial motion information of the first obstacle includes a first action set and a first trigger set of the first obstacle. The processing unit 1302 is further configured to: adjust the first action set and / or the first trigger set of the first obstacle to generate the first motion information of the first obstacle.

[0258] In a possible implementation manner, the initial motion information of the first obstacle includes an initial motion trajectory of the first obstacle.

[0259] The processing unit 1302 is further configured to adjust the initial motion trajectory of the first obstacle to obtain a first motion trajectory, wherein the first motion information includes the first motion trajectory.

[0260] In a possible implementation, the initial motion trajectory of the first obstacle includes multiple trajectory points. The acquisition unit 1301 is further configured to receive an update operation of some or all of the multiple trajectory points input by a user. The processing unit 1302 is further configured to obtain the first motion trajectory according to the update operation.

[0261] In a possible implementation, the scene data also includes the initial motion trajectory of the vehicle. The processing unit 1302 is also used for:

[0262] Determine a trajectory interaction point between an initial motion trajectory of the first obstacle and an initial motion trajectory of the ego vehicle;

[0263] According to the trajectory interaction points, derived trajectory points are derived, and the positions of the derived trajectory points are located on the initial motion trajectory of the vehicle and are different from the positions of the trajectory interaction points;

[0264] According to the derived trajectory points, a first motion trajectory is obtained.

[0265] In a possible implementation, the processing unit 1302 is further configured to:

[0266] The second motion information of the second obstacle is determined based on the initial motion information of the second obstacle, the cost of the second motion information of the second obstacle, and the first motion information of the first obstacle.

[0267] Optionally, the cost of the second motion information includes one or more of authenticity cost, smoothness cost, obstacle avoidance cost, and scene design cost.

[0268] In a possible implementation manner, the cost J of the second motion information of the second obstacle i It is expressed by the following formula:

[0269] J i =W X (X i -X ref ) 2 +W k k i 2 +W u u i 2 +W a a i 2 +W s slack i 2 +W f f hard 2

[0270] The introduction of the parameters can be found in the above.

[0271] In a possible implementation manner, the second motion information satisfies one or more of the following constraints: kinematic constraints, collision constraints, or driving regulation constraints.

[0272] In a possible implementation, the scene data of the first driving scene further includes initial motion information of a fourth obstacle, and the fourth obstacle has no interaction with the ego vehicle and the first obstacle.

[0273] The processing unit 1302 is further configured to generate a target driving scene according to the first motion information of the first obstacle, the second motion information of the second obstacle, and the initial motion information of the fourth obstacle.

[0274] Fig.14 The figure shows a schematic diagram of the structure of a computing device provided in an embodiment of the present application. The computing device 140 is a device with computing capabilities, where the device can be a physical device, such as a controller, a processor, a server (such as a rack server), a host, etc., or a virtual device, such as a virtual machine, a container, etc.

[0275] like Fig.14As shown, the computing device 140 includes: a processor 1402 and a memory 1401, and optionally includes a bus 1404 and a communication interface 1403. The processor 1402 and the memory 1401 communicate with each other via the bus 1404. It should be understood that the present application does not limit the number of processors and memories in the computing device 140.

[0276] The memory 1401 is used to provide a storage space, and the storage space can optionally store application data, user data, operating system, and computer programs. The memory 1401 may include a volatile memory, such as a random access memory (RAM). The memory 1401 may also include a non-volatile memory, such as a read-only memory (ROM), a flash memory, a hard disk drive (HDD), or a solid state drive (SSD).

[0277] Processor 1402 is a module for performing calculations, and may include any one or more of a controller (e.g., a storage controller), a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP), a digital signal processor (DSP), a coprocessor (assisting the central processor in completing corresponding processing and applications), an application specific integrated circuit (ASIC), a microcontroller unit (MCU), a virtual machine, a container, and the like.

[0278] The communication interface 1403 is used to provide information input or output for at least one processor. And / or, the communication interface 1403 can be used to receive data sent externally and / or send data to the outside. The communication interface 1403 can be a wired link interface including an Ethernet cable, etc., or a wireless link (Wi-Fi, Bluetooth, general wireless transmission and other wireless communication technologies, etc.) interface. Optionally, the communication interface 1403 can also include a transmitter (such as a radio frequency transmitter, an antenna, etc.) coupled to the interface, or a receiver, etc.

[0279] The bus 1404 may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Fig.14 The bus 1404 is represented by only one line, but does not mean that there is only one bus or one type of bus. The bus 1404 may include a path for transmitting information between various components of the computing device 140 (eg, the memory 1401, the processor 1402, and the communication interface 1403).

[0280] In the embodiment of the present application, the memory 1401 stores executable instructions, and the processor 1402 executes the executable instructions to implement the aforementioned scene generation method, for example Figure 3 The scene generation method in the embodiment shown in the figure. That is, the memory 1401 stores instructions for executing the scene generation method.

[0281] The embodiment of the present application further provides a computing device cluster, the computing device cluster comprising at least one computing device 140, each computing device 140 comprising a processor 1402 and a memory 1401;

[0282] The processor 1402 of at least one computing device 140 is used to execute instructions stored in the memory 1401 of at least one computing device 140, so that the computing device cluster implements the aforementioned scene generation method, for example Figure 3 The scene generation method in the illustrated embodiment. Optionally, the memory stores instructions for executing the scene generation method.

[0283] The embodiment of the present application provides a computer-readable storage medium, in which instructions are stored. When the instructions are executed by at least one processor, the aforementioned scene generation method is implemented, for example Figure 3 The scene generation method in the illustrated embodiment.

[0284] The computer-readable storage medium may be any available medium that can be stored by a computing device, or a data storage device such as a data center that contains one or more available media. The computer-readable storage medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a magnetic tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state hard disk).

[0285] The present application provides a computer program product, which includes computer instructions. When the instructions are executed on at least one processor, the aforementioned scene generation method is implemented, for example Figure 3The scene generation method in the illustrated embodiment.

[0286] Optionally, the computer program product may be a software installation package or an image package. When the aforementioned method is required, the computer program product may be downloaded and executed on a computing device.

[0287] In the embodiments of the present application, the words "exemplary" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary" or "for example" in the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific way.

[0288] The "at least one" mentioned in the embodiments of the present application refers to one or more, and "plurality" refers to two or more. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b, or c can be represented by: a, b, c, (a and b), (a and c), (b and c), or (a and b and c), where a, b, c can be single or multiple. "And / or" describes the association relationship of associated objects, indicating that three relationships can exist. For example, A and / or B can be represented by: A exists alone, A and B exist at the same time, and B exists alone, where A and B can be singular or plural. The character " / " generally indicates that the associated objects before and after are in an "or" relationship.

[0289] Furthermore, unless otherwise specified, the ordinal numbers such as "first" and "second" used in the embodiments of the present application are used to distinguish multiple objects, and are not used to limit the order, timing, priority or importance of multiple objects. For example, the first storage device and the second storage device are only for the convenience of description, and do not represent the difference in device structure, deployment order, importance, etc. of the first storage device and the second storage device.

[0290] Those skilled in the art will appreciate that all or part of the steps to implement the above embodiments may be accomplished by hardware, or by programs instructing related hardware to accomplish the steps. The programs may be stored in a computer-readable storage medium, and the above-mentioned storage medium may be a read-only memory, a disk, or an optical disk, etc.

[0291] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the protection scope of the technical solutions of the embodiments of the present invention.

Claims

1. A scene generation method, characterized in that: Methods include: Acquire scene data of a first driving scene, where the scene data of the first driving scene is used to indicate initial motion information of the ego vehicle and obstacles in the first driving scene, where the obstacles in the first driving scene include a first obstacle and a second obstacle; adjusting the initial motion information of the first obstacle to generate first motion information of the first obstacle; Obtaining second motion information of the second obstacle according to the initial motion information of the second obstacle and the first motion information of the first obstacle; A target driving scene is generated according to the first motion information of the first obstacle and the second motion information of the second obstacle.

2. The method according to claim 1, characterized in that The obtaining, according to the initial motion information of the second obstacle and the first motion information of the first obstacle, second motion information of the second obstacle includes: The second motion information of the second obstacle is obtained according to the initial motion information of the vehicle, the initial motion information of the second obstacle and the first motion information of the first obstacle.

3. The method according to claim 1 or 2, characterized in that: The degree of interaction between the first obstacle and the vehicle is not lower than the degree of interaction between the second obstacle and the vehicle.

4. The method according to any one of claims 1 to 3, characterized in that: The second obstacle is an obstacle that interacts with the first obstacle and / or an obstacle that interacts with the vehicle other than the first obstacle.

5. The method according to any one of claims 1 to 4, characterized in that: The adjusting the initial motion information of the first obstacle to generate first motion information of the first obstacle includes: The initial motion information of the first obstacle is adjusted according to user input or through an artificial intelligence model to generate first motion information of the first obstacle.

6. The method according to any one of claims 1 to 5, characterized in that: The obstacles in the first driving scene include first-category obstacles and second-category obstacles, wherein: The first type of obstacle is an obstacle whose degree of interaction with the vehicle satisfies a first condition. The second type of obstacles include obstacles whose interaction degree with the ego vehicle satisfies a second condition and / or obstacles that interact with a third obstacle, the third obstacle is at least one obstacle in the first type of obstacles, and the first condition is different from the second condition; The first obstacle belongs to the first category of obstacles, and the second obstacle includes at least one obstacle in the second category of obstacles and / or includes at least one obstacle in the first category of obstacles other than the first obstacle.

7. The method according to claim 6, characterized in that The first condition includes: the time interval between the obstacle and the ego vehicle passing through the trajectory interaction point falls within a first time range; The second condition includes: the time interval between the obstacle and the vehicle passing through the trajectory interaction point falls within a second time range, and the lower limit of the second time range is greater than the upper limit of the first time range; There is interaction between the obstacle and the third obstacle when the following conditions are met: There is a trajectory interaction point between the obstacle and the third obstacle, and the time interval between the two passing the trajectory interaction point falls within the third time range.

8. The method according to claim 6, characterized in that The first condition includes: there is a trajectory interaction point between the movement trajectory of the obstacle and the movement trajectory of the vehicle, and the first time interval between the two passing the trajectory interaction point is the first N time intervals of the time interval sequence, N is an integer greater than 0, and the first N time intervals of the time interval sequence are less than or equal to the time intervals other than the first N time intervals in the time interval sequence; The second condition includes: there is a trajectory interaction point between the movement trajectory of the obstacle and the movement trajectory of the vehicle, and the first time interval between the two passing the trajectory interaction point falls within the first N+1 to K of the time interval sequence, where K is an integer and K>N; The time interval sequence includes M time intervals arranged in order of duration, each of the M time intervals is the interval between the moment when an ego-vehicle interaction obstacle passes through a first trajectory interaction point with the ego-vehicle and the moment when the ego-vehicle passes through the first trajectory interaction point, the ego-vehicle interaction obstacle is an obstacle that has a trajectory interaction point with the ego-vehicle in the first driving scene, M is an integer and M≥K.

9. The method according to any one of claims 1 to 8, characterized in that: The initial motion information of the first obstacle includes a plurality of behavior parameters of the first obstacle, and the plurality of behavior parameters are used to describe the behavior of the first obstacle; and the adjusting the initial motion information of the first obstacle to generate the first motion information of the first obstacle includes: The values ​​or thresholds of the multiple behavior parameters of the first obstacle are adjusted to generate first motion information of the first obstacle.

10. The method according to any one of claims 1 to 8, characterized in that: The initial motion information of the first obstacle includes a first action set and a first trigger set of the first obstacle, the first action set is used to describe a first action of the first obstacle, the first action set includes a value of a first behavior parameter of the first action, and the first trigger set is used to describe a condition for executing the first action; The adjusting the initial motion information of the first obstacle to generate first motion information of the first obstacle includes: A first action set and / or a first trigger set of the first obstacle is adjusted to generate first motion information of the first obstacle.

11. The method according to any one of claims 1 to 8, characterized in that: The initial motion information of the first obstacle includes an initial motion trajectory of the first obstacle; The adjusting the initial motion information of the first obstacle to generate first motion information of the first obstacle includes: The initial motion trajectory of the first obstacle is adjusted to obtain a first motion trajectory, and the first motion information includes the first motion trajectory.

12. The method according to claim 11, characterized in that The initial motion trajectory of the first obstacle includes a plurality of trajectory points, and the adjusting the initial motion trajectory of the first obstacle to obtain the first motion trajectory includes: receiving an update operation of a part or all of the plurality of track points input by a user; According to the updating operation, the first motion trajectory is obtained.

13. The method according to claim 11, characterized in that The adjusting the initial motion trajectory of the first obstacle to obtain the first motion trajectory includes: Determining a trajectory interaction point between an initial motion trajectory of the first obstacle and an initial motion trajectory of the ego vehicle; According to the trajectory interaction point, a derived trajectory point is obtained, wherein the derived trajectory point is a trajectory point located on the initial motion trajectory of the vehicle and different from the trajectory interaction point; A first motion trajectory is obtained according to the derived trajectory points.

14. The method according to any one of claims 1 to 13, characterized in that: Obtaining second motion information of the second obstacle according to the initial motion information of the second obstacle and the first motion information of the first obstacle includes: The second motion information of the second obstacle is determined based on the initial motion information of the second obstacle, the cost of the second motion information of the second obstacle, and the first motion information of the first obstacle.

15. The method according to claim 14, characterized in that The cost of the second motion information includes at least one of a authenticity cost, a smoothness cost, an obstacle avoidance cost, and a scene design cost, the second motion information includes a second motion trajectory of the second obstacle, and the second motion trajectory includes a plurality of trajectory points. The authenticity cost is used to indicate the difference between the second motion information of the second obstacle and the initial motion information of the second obstacle, and the authenticity cost is related to the coordinates, orientation, curvature and speed of the plurality of trajectory points in the second motion trajectory. The smoothness cost is used to indicate the smoothness of position changes of the plurality of trajectory points in the second motion trajectory, and the smoothness cost is related to the curvature and acceleration of the second motion trajectory; The obstacle avoidance cost is used to indicate the obstacle avoidance range of the second obstacle. The scenario design cost is a predefined scenario cost design.

16. The method according to any one of claims 1 to 15, characterized in that: The second motion information satisfies one or more of the following constraints: kinematic constraints, collision constraints, or driving regulation constraints, wherein: The kinematic constraint is used to indicate the value range of the following behavior parameters in the second motion information: one or more of speed, acceleration, curvature of the motion trajectory, and rate of change of curvature of the motion trajectory; The collision constraint is used to constrain the second obstacle to have no collision with the vehicle and obstacles in the first driving scene. The driving specification constraint is used to constrain the behavior of the first obstacle to meet a predefined driving specification.

17. The method according to any one of claims 1 to 16, characterized in that: The scene data of the first driving scene further includes initial motion information of a fourth obstacle, and the fourth obstacle has no interaction with the vehicle and the first obstacle. The generating a target driving scene according to the first motion information of the first obstacle and the second motion information of the second obstacle includes: A target driving scene is generated according to the first motion information of the first obstacle, the second motion information of the second obstacle, and the initial motion information of the fourth obstacle.

18. A scene generation device, characterized in that: The device comprises an acquisition unit and a processing unit, wherein: The acquisition unit is used to acquire scene data of a first driving scene, where the scene data of the first driving scene is used to indicate initial motion information of the ego vehicle and obstacles in the first driving scene, where the obstacles in the first driving scene include a first obstacle and a second obstacle; The processing unit is used for: adjusting the initial motion information of the first obstacle to generate first motion information of the first obstacle; Obtaining second motion information of the second obstacle according to the initial motion information of the second obstacle and the first motion information of the first obstacle; A target driving scene is generated according to the first motion information of the first obstacle and the second motion information of the second obstacle.

19. The scene generation device according to claim 18, characterized in that: The processing unit is also used for: The second motion information of the second obstacle is obtained according to the initial motion information of the vehicle, the initial motion information of the second obstacle and the first motion information of the first obstacle.

20. The scene generation device according to claim 18 or 19, characterized in that: The degree of interaction between the first obstacle and the vehicle is not lower than the degree of interaction between the second obstacle and the vehicle.

21. The scene generation device according to any one of claims 18 to 20, characterized in that: The second obstacle is an obstacle that interacts with the first obstacle and / or an obstacle that interacts with the vehicle other than the first obstacle.

22. The scene generation device according to any one of claims 18 to 21, characterized in that: The processing unit is also used for: The initial motion information of the first obstacle is adjusted according to user input or through an artificial intelligence model to generate first motion information of the first obstacle.

23. The scene generation device according to any one of claims 18 to 22, characterized in that: The obstacles in the first driving scene include first-category obstacles and second-category obstacles, wherein: The first type of obstacle is an obstacle whose degree of interaction with the vehicle satisfies a first condition. The second type of obstacles include obstacles whose interaction degree with the ego vehicle satisfies a second condition and / or obstacles that interact with a third obstacle, the third obstacle is at least one obstacle in the first type of obstacles, and the first condition is different from the second condition; The first obstacle belongs to the first category of obstacles, and the second obstacle includes obstacles in the second category of obstacles and / or includes obstacles in the first category of obstacles except the first obstacle.

24. The scene generation device according to any one of claims 18 to 23, characterized in that: The initial motion information of the first obstacle includes a plurality of behavior parameters of the first obstacle, where the plurality of behavior parameters are used to describe the behavior of the first obstacle; and the processing unit is further used to: The values ​​or thresholds of the multiple behavior parameters of the first obstacle are adjusted to generate first motion information of the first obstacle.

25. The scene generation device according to any one of claims 18 to 23, characterized in that: The initial motion information of the first obstacle includes a first action set and a first trigger set of the first obstacle, the first action set is used to describe a first action of the first obstacle, the first action set includes a value of a first behavior parameter of the first action, and the first trigger set is used to describe a condition for executing the first action; The processing unit is also used for: A first action set and / or a first trigger set of the first obstacle is adjusted to generate first motion information of the first obstacle.

26. The scene generation device according to any one of claims 18 to 23, characterized in that: The initial motion information of the first obstacle includes an initial motion trajectory of the first obstacle; The processing unit is also used for: The initial motion trajectory of the first obstacle is adjusted to obtain a first motion trajectory, and the first motion information includes the first motion trajectory.

27. The scene generation device according to claim 26, characterized in that: The initial motion trajectory of the first obstacle includes a plurality of trajectory points; The acquisition unit is further used to receive an update operation of a part or all of the plurality of track points input by a user; The processing unit is further configured to obtain the first motion trajectory according to the updating operation.

28. The scene generation device according to claim 27, characterized in that: The processing unit is also used for: Determining a trajectory interaction point between an initial motion trajectory of the first obstacle and an initial motion trajectory of the ego vehicle; According to the trajectory interaction point, a derived trajectory point is obtained, wherein the derived trajectory point is a trajectory point located on the initial motion trajectory of the vehicle and different from the trajectory interaction point; A first motion trajectory is obtained according to the derived trajectory points.

29. The scene generation device according to any one of claims 18 to 28, characterized in that: The processing unit is further used for: The second motion information of the second obstacle is determined based on the initial motion information of the second obstacle, the cost of the second motion information of the second obstacle, and the first motion information of the first obstacle.

30. The scene generation device according to claim 29, characterized in that: The cost of the second motion information includes at least one of a authenticity cost, a smoothness cost, an obstacle avoidance cost, and a scene design cost, the second motion information includes a second motion trajectory of the second obstacle, and the second motion trajectory includes a plurality of trajectory points. The authenticity cost is used to indicate the difference between the second motion information of the second obstacle and the initial motion information of the second obstacle, and the authenticity cost is related to the coordinates, orientation, curvature and speed of the plurality of trajectory points in the second motion trajectory. The smoothness cost is used to indicate the smoothness of position changes of the plurality of trajectory points in the second motion trajectory, and the smoothness cost is related to the curvature and acceleration of the second motion trajectory; The obstacle avoidance cost is used to indicate the obstacle avoidance range of the second obstacle. The scenario design cost is a predefined scenario cost design.

31. The scene generation device according to any one of claims 18 to 30, characterized in that: The second motion information satisfies one or more of the following constraints: kinematic constraints, collision constraints, or driving regulation constraints, wherein: The kinematic constraint is used to indicate the value range of the following behavior parameters in the second motion information: one or more of curvature, velocity, acceleration, and curvature derivative; The collision constraint is used to constrain the second obstacle to have no collision with the vehicle and obstacles in the first driving scene. The driving specification constraint is used to constrain the behavior of the first obstacle to meet a predefined driving specification.

32. The scene generation device according to any one of claims 18 to 31, characterized in that: The scene data of the first driving scene further includes initial motion information of a fourth obstacle, and the fourth obstacle has no interaction with the vehicle and the first obstacle. The processing unit is further used for: A target driving scene is generated according to the first motion information of the first obstacle, the second motion information of the second obstacle, and the initial motion information of the fourth obstacle.

33. A computing device, characterized in that The computing device includes a memory and a processor, wherein the memory is used to store program instructions; When the processor executes the program instructions in the memory, the computing device is caused to perform the method according to any one of claims 1 to 17.

34. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores program instructions, and when the program instructions are executed by a processor, the method according to any one of claims 1 to 17 is implemented.

Images