Vehicle driving scene video generation method, electronic device, and storage medium

Abstract

Embodiments of the present application provide a vehicle driving scene video generation method, an electronic device and a storage medium. The method comprises: obtaining scene simulation data and lane line labeling data; determining to-be-updated trajectory data by using a preset screening condition and the lane line labeling data; performing update processing on the to-be-updated trajectory data to obtain target trajectory data of a candidate vehicle; and performing scene rendering based on the target trajectory data and the scene simulation data to generate a target driving scene video. The present application solves the technical problems of low efficiency of scene generation and insufficient authenticity and effectiveness of an autonomous driving scene in the related art.

Description

Technical Field

[0001] This application relates to the field of vehicle technology, and more specifically, to a method for generating vehicle driving scene videos, an electronic device, and a storage medium. Background Technology

[0002] The rapid development of autonomous driving has spurred the demand for generating complex driving scenarios, especially long-tail scenarios—those that are relatively rare but crucial to the safety of driving systems. In the field of autonomous driving scenario construction, simulation platforms and generative diffusion models each demonstrate their strengths, showcasing advantages in scene richness and realism. However, the driving scenario construction methods in these technologies still have significant limitations. While simulation platforms can accurately simulate the driving environment, their scenes differ from the real world, making it difficult to fully reflect the actual performance of autonomous driving algorithms. Although generative diffusion model-based solutions enhance scene realism, they cannot ensure spatiotemporal consistency, particularly in the accuracy of ground markings such as lane lines and turn signals, further limiting their application value. Existing methods, when generating long-tail scenarios, suffer from two main problems: firstly, the simulation scene's appearance does not match the real environment well, affecting the effectiveness of algorithm testing; secondly, their ability to automatically edit vehicle behavior is insufficient, especially in the precise control and real-time generation of new trajectories. This results in low scene generation efficiency and an inability to cover sufficient long-tail driving situations.

[0003] There is currently no good solution to the above problems. Summary of the Invention

[0004] This application provides a method for generating vehicle driving scene videos, an electronic device, and a storage medium to at least solve the technical problems of insufficient realism and effectiveness of autonomous driving scenes and low scene generation efficiency in related technologies.

[0005] According to one aspect of the embodiments of this application, a method for generating a vehicle driving scene video is provided, comprising: acquiring scene simulation data and lane line annotation data, wherein the scene simulation data is used to represent virtual data generated by simulating at least one object located within a driving area in the current driving scene, and the lane line annotation data is used to represent the geometric attributes of lane lines in the driving road; determining trajectory data to be updated using preset filtering conditions and lane line annotation data, wherein the preset filtering conditions are used to filter out multiple candidate vehicles associated with the target vehicle during the movement of the target vehicle, and the trajectory data to be updated is used to represent the original trajectory data corresponding to the candidate vehicles; updating the trajectory data to be updated to obtain target trajectory data of the candidate vehicles; and performing scene rendering based on the target trajectory data and scene simulation data to generate a target driving scene video, wherein the target driving scene video is used to simulate and verify the intelligent driving system of the target vehicle.

[0006] Optionally, acquiring scene simulation data includes: acquiring an initial driving dataset; decoupling the initial driving dataset to obtain first point cloud data and second point cloud data, wherein the first point cloud data is used to represent dynamic obstacle data in the road, and the second point cloud data is used to represent static background data in the road; performing initialization and reconstruction processing on the first point cloud data in a first coordinate system to obtain vehicle object data, and performing initialization and reconstruction processing on the second point cloud data in a second coordinate system to obtain background object data; and determining scene simulation data based on the vehicle object data and background object data.

[0007] Optionally, determining the trajectory data to be updated using preset filtering conditions and lane line annotation data includes: obtaining lane centerline data based on lane line annotation data; determining the associated lanes of the target vehicle based on the lane centerline data, wherein the associated lanes include the current lane where the target vehicle is located, the left adjacent lane of the current lane, and the right adjacent lane of the current lane; in response to the relevant vehicles in the associated lanes meeting the preset filtering conditions, determining the relevant vehicles as candidate vehicles, wherein the preset filtering conditions indicate that the longitudinal distance between the relevant vehicles and the target vehicle is less than a preset distance threshold; and determining the original trajectory data of the candidate vehicles as the trajectory data to be updated.

[0008] Optionally, updating the trajectory data to be updated to obtain the target trajectory data of the candidate vehicle includes: obtaining the action type to be updated corresponding to the candidate vehicle, wherein the action type to be updated includes at least one of the following: lane change cut-in type, emergency braking type; updating the trajectory data to be updated based on the action type to be updated to obtain intermediate trajectory data; and optimizing the intermediate trajectory data to obtain the target trajectory data.

[0009] Optionally, obtaining the action type to be updated corresponding to the candidate vehicle includes: in response to the candidate vehicle being located in the current lane where the target vehicle is located, determining the action type to be updated to be an emergency braking type; in response to the candidate vehicle not being located in the current lane where the target vehicle is located, determining the action type to be updated to be a lane change / cut-in type.

[0010] Optionally, updating the trajectory data to be updated based on the action type to be updated to obtain intermediate trajectory data includes: in response to the action type to be updated being a lane change cut-in type, determining a target interpolation control point based on the action type to be updated; performing interpolation processing based on the target interpolation control point to obtain first interpolated trajectory data; and using the first interpolated trajectory data to update the trajectory data to be updated to obtain intermediate trajectory data.

[0011] Optionally, updating the trajectory data to be updated based on the action type to be updated to obtain intermediate trajectory data includes: in response to the action type to be updated being an emergency braking type, determining braking attribute data based on the action type to be updated; performing interpolation processing based on the braking attribute data to obtain second interpolated trajectory data; and using the second interpolated trajectory data to update the trajectory data to be updated to obtain intermediate trajectory data.

[0012] Optionally, optimizing the intermediate trajectory data to obtain the target trajectory data includes: fitting the preset road surface height data to obtain local road surface height information; correcting the intermediate trajectory data based on the local road surface height information to obtain a trajectory correction result; and smoothing the trajectory correction result to obtain the target trajectory data.

[0013] According to another aspect of the embodiments of this application, a vehicle driving scene video generation device is also provided, comprising: an acquisition module, configured to acquire scene simulation data and lane line annotation data, wherein the scene simulation data is used to represent virtual data generated by simulating at least one object located within the driving area in the current driving scene, and the lane line annotation data is used to represent the geometric attributes of lane lines in the driving road; a determination module, configured to determine trajectory data to be updated using preset filtering conditions and lane line annotation data, wherein the preset filtering conditions are used to filter out multiple candidate vehicles associated with the target vehicle during the movement of the target vehicle, and the trajectory data to be updated is used to represent the original trajectory data corresponding to the candidate vehicles; an update module, configured to update the trajectory data to be updated to obtain target trajectory data of the candidate vehicles; and a generation module, configured to perform scene rendering based on the target trajectory data and scene simulation data to generate a target driving scene video, wherein the target driving scene video is used to simulate and verify the intelligent driving system of the target vehicle.

[0014] Optionally, the acquisition module is further configured to: acquire an initial driving dataset; decouple the initial driving dataset to obtain a first point cloud data and a second point cloud data, wherein the first point cloud data is used to represent dynamic obstacle data in the road, and the second point cloud data is used to represent static background data in the road; perform initialization and reconstruction processing on the first point cloud data in a first coordinate system to obtain vehicle object data, and perform initialization and reconstruction processing on the second point cloud data in a second coordinate system to obtain background object data; and determine scene simulation data based on the vehicle object data and the background object data.

[0015] Optionally, the determining module is further configured to: obtain lane centerline data based on lane line marking data; determine the associated lanes of the target vehicle based on the lane centerline data, wherein the associated lanes include the current lane where the target vehicle is located, the left adjacent lane of the current lane, and the right adjacent lane of the current lane; in response to the relevant vehicles in the associated lanes meeting preset filtering conditions, determine the relevant vehicles as candidate vehicles, wherein the preset filtering conditions indicate that the longitudinal distance between the relevant vehicles and the target vehicle is less than a preset distance threshold; and determine the original trajectory data of the candidate vehicles as trajectory data to be updated.

[0016] Optionally, the update module is further configured to: obtain the action type to be updated corresponding to the candidate vehicle, wherein the action type to be updated includes at least one of the following: lane change cut-in type, emergency braking type; update the trajectory data to be updated based on the action type to be updated to obtain intermediate trajectory data; and optimize the intermediate trajectory data to obtain target trajectory data.

[0017] Optionally, the update module is also configured to: determine the action type to be updated as emergency braking in response to the candidate vehicle being located in the current lane where the target vehicle is located; and determine the action type to be updated as lane change / cut-in in response to the candidate vehicle not being located in the current lane where the target vehicle is located.

[0018] Optionally, the update module is further configured to: in response to the action type to be updated being a lane change cut-in type, determine a target interpolation control point based on the action type to be updated; perform interpolation processing based on the target interpolation control point to obtain first interpolated trajectory data; and use the first interpolated trajectory data to update the trajectory data to be updated to obtain intermediate trajectory data.

[0019] Optionally, the update module is further configured to: in response to the action type to be updated being an emergency braking type, determine braking attribute data based on the action type to be updated; perform interpolation processing based on the braking attribute data to obtain second interpolated trajectory data; and use the second interpolated trajectory data to update the trajectory data to be updated to obtain intermediate trajectory data.

[0020] Optionally, the update module is also used to: perform fitting processing on preset road surface height data to obtain local road surface height information; perform correction processing on intermediate trajectory data based on local road surface height information to obtain trajectory correction results; and perform smoothing processing on trajectory correction results to obtain target trajectory data.

[0021] According to another aspect of the embodiments of this application, an electronic device is also provided, including: a memory storing an executable program; and a processor for running the program, wherein the program executes the methods in various embodiments of this application when it runs.

[0022] According to another aspect of the embodiments of this application, a computer-readable storage medium is also provided, the computer-readable storage medium including a stored executable program, wherein, when the executable program is running, it controls the device where the computer-readable storage medium is located to perform the methods of various embodiments of this application.

[0023] According to another aspect of the embodiments of this application, a computer program product is also provided, including a computer program that, when executed by a processor, implements the methods of various embodiments of this application.

[0024] According to another aspect of the embodiments of this application, a computer program product is also provided, including a non-volatile computer-readable storage medium storing a computer program that, when executed by a processor, implements the methods in various embodiments of this application.

[0025] According to another aspect of the embodiments of this application, a computer program is also provided, which, when executed by a processor, implements the methods of the various embodiments of this application.

[0026] In this embodiment, scenario simulation data and lane line annotation data are acquired, and then the trajectory data to be updated is determined using preset filtering conditions and lane line annotation data. Subsequently, the trajectory data to be updated is updated to obtain the target trajectory data of candidate vehicles. Finally, scenario rendering is performed based on the target trajectory data and scenario simulation data to generate a target driving scenario video, thereby achieving efficient simulation verification of the intelligent driving system in complex long-tail scenarios. By fusing scenario simulation data and lane line annotation data, a dynamic and accurate virtual driving environment is constructed, effectively filtering and updating candidate vehicle trajectories that may affect the target vehicle's driving, ensuring a high degree of consistency between vehicle behavior in the generated long-tail scenario video and the real world. In the update processing, the trajectories of candidate vehicles are updated to exhibit complex interference patterns that could affect the target vehicle in the simulation scenario, such as sudden lane changes and emergency stops, while maintaining speed and smoothness. This significantly improves the realism and challenge of the scenario, enriching the test scenarios for the intelligent driving system and providing training opportunities for safe driving under extreme conditions, enhancing the robustness and adaptability of the intelligent driving system in the face of unforeseen events. Scene rendering based on target trajectory data and scene simulation data generates target driving scene videos, providing an effective simulation verification method for intelligent driving systems. This solves the technical problems of insufficient realism and effectiveness of autonomous driving scenarios and low scene generation efficiency in related technologies. Attached Figure Description

[0027] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0028] Figure 1 This is a flowchart of a vehicle driving scene video generation method according to an embodiment of this application;

[0029] Figure 2 This is a structural block diagram of a vehicle driving scene video generation device according to an embodiment of this application. Detailed Implementation

[0030] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0031] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0032] According to an embodiment of this application, a method embodiment for generating vehicle driving scene video is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0033] This method embodiment can be executed in an electronic device or similar computing device that includes memory and a processor. Taking a computer terminal as an example, the computer terminal may include one or more processors (processors may include, but are not limited to, central processing units (CPUs), graphics processing units (GPUs), digital signal processing (DSP) chips, microcontroller units (MCUs), field-programmable gate arrays (FPGAs), neural network processors (NPUs), tensor processors (TPUs), artificial intelligence (AI) type processors, etc.) and memory for storing data. Optionally, the computer terminal may also include transmission devices, input / output devices, and display devices for communication functions. Those skilled in the art will understand that the above structural description is merely illustrative and does not limit the structure of the computer terminal. For example, the computer terminal may include more or fewer components than described above, or have a different configuration than described above.

[0034] The memory can be used to store computer programs, such as application software programs and modules, like the computer program corresponding to the vehicle driving scene video generation method in this application embodiment. The processor executes various functional applications and data processing by running the computer program stored in the memory, thereby realizing the aforementioned vehicle driving scene video generation method. The memory may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory may further include memory remotely located relative to the processor, and these remote memories can be connected to the mobile terminal via a network. Examples of the aforementioned networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0035] The transmission device is used to receive or send data via a network. Specific examples of the network described above may include a wireless network provided by the mobile terminal's communication provider. In one example, the transmission device includes a Network Interface Controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the transmission device may be a Radio Frequency (RF) module, used for wireless communication with the Internet.

[0036] Display devices can be, for example, touchscreen liquid crystal displays (LCDs) and touch displays (also referred to as "touchscreens" or "touch displays"). The LCD allows users to interact with the user interface of the mobile terminal. In some embodiments, the mobile terminal has a graphical user interface (GUI), which allows users to interact with the GUI through finger contact and / or gestures on a touch-sensitive surface. Optional human-computer interaction functions include: creating web pages, drawing, word processing, creating electronic documents, playing games, video conferencing, instant messaging, sending and receiving emails, call interfaces, playing digital video, playing digital music, and / or web browsing, etc. Executable instructions for performing the above human-computer interaction functions are configured / stored in one or more processor-executable computer program products or readable storage media.

[0037] This embodiment provides a method for generating vehicle driving scene videos. Figure 1 This is a flowchart of a vehicle driving scene video generation method according to an embodiment of this application, such as... Figure 1 As shown, the process includes the following steps:

[0038] Step S11: Obtain scene simulation data and lane line annotation data. The scene simulation data is used to represent virtual data generated to simulate at least one object located in the driving area in the current driving scenario. The lane line annotation data is used to represent the geometric properties of lane lines in the driving road.

[0039] Step S12: Determine the trajectory data to be updated using preset filtering conditions and lane line marking data. The preset filtering conditions are used to filter out multiple candidate vehicles associated with the target vehicle during the target vehicle's movement. The trajectory data to be updated is used to represent the original trajectory data corresponding to the candidate vehicles.

[0040] Step S13: Update the trajectory data to be updated to obtain the target trajectory data of the candidate vehicles;

[0041] Step S14: Render the scene based on the target trajectory data and scene simulation data to generate a target driving scene video. The target driving scene video is used to simulate and verify the intelligent driving system of the target vehicle.

[0042] The aforementioned scenario simulation data is a type of virtual data used to recreate specific driving scenarios in a computer environment. It includes information about various objects within the driving area, such as vehicles, pedestrians, and static obstacles, as well as their corresponding motion states and environmental conditions. Scenario simulation data can be acquired using a simulation platform capable of simulating various driving environments, including but not limited to traffic flow, weather conditions, and road types. By setting scenario parameters, such as the number of vehicles, pedestrian behavior, and time points, highly customized scenario simulation data can be generated. The scenario simulation data includes the state information of all objects within the driving area, such as position, speed, and acceleration, as well as environmental conditions, such as lighting, weather, and time. Scenario simulation data can also be acquired using 3D reconstruction technology. Techniques such as Street Gaussians can be used to reconstruct the real-world road environment with high fidelity, thereby obtaining scenario simulation data. Specifically, this involves extracting information from images taken from multiple angles or LiDAR point cloud data, reconstructing a 3D model of the environment, and then adding dynamic behaviors by editing vehicle instances to generate scenario simulation data.

[0043] The lane marking data mentioned above records the geometric attributes of lane lines on roads, including but not limited to information such as lane position, length, width, and curvature. This data is crucial for simulating vehicle routes and determining whether vehicles are adhering to lane rules. For existing datasets, lane marking data can be manually generated by professionals using image annotation software to annotate lane lines in road images or videos, including information such as lane position, length, width, and curvature. Lane perception models built using deep learning techniques, such as convolutional neural networks, can automatically identify and annotate lane lines directly from videos or images. Combining data from multiple sensor sources, such as vehicle vision sensors, LiDAR, and radar, and using data fusion algorithms such as Kalman filtering and particle filtering, more accurate lane marking data can be obtained.

[0044] The aforementioned preset screening criteria can be a set of predefined standards used to automatically identify objects that may have a potential impact on the target vehicle during its driving process, especially other vehicles in lanes near the target vehicle, i.e., candidate vehicles.

[0045] The aforementioned trajectory data to be updated refers to the original data of the candidate vehicle traveling along a preset path in the original scenario. The trajectory data to be updated can be modified to form vehicle behavior that includes long-tail events. After the update process, the target trajectory data of the candidate vehicle in the simulated scenario is obtained, aiming to simulate complex driving situations that pose a challenge to the target vehicle.

[0046] Based on preset filtering criteria, candidate vehicles in the scenario simulation data are located and identified. These criteria consider the relative position and speed between the target vehicle and candidate vehicles, ensuring that selected vehicles are likely to cause long-tail events. With the assistance of lane line annotation data, it is possible to more accurately determine whether candidate vehicles are located in lanes near the target vehicle, thereby identifying which vehicles' trajectories need to be updated.

[0047] After identifying candidate vehicles, a heuristic strategy can be used to update the original trajectory data of each vehicle, generating a series of preset disturbance behaviors, such as sudden cut-in and emergency braking. This update process not only modifies the candidate vehicles' driving paths but also ensures that the new trajectories match road conditions, with smooth changes in speed and angular velocity, making the simulation scenario more realistic and reliable, and improving the effectiveness of autonomous driving system simulation verification.

[0048] After obtaining the target trajectory data, a target driving scene video is generated by rendering the scene using 3D Gaussian splashing technology based on the target trajectory data and scene simulation data. In this process, by integrating the updated vehicle trajectory data and complete scene details, a highly realistic virtual driving environment can be created, which is particularly suitable for testing the reaction capability of the target vehicle's intelligent driving system when encountering long-tail events. Therefore, this embodiment effectively generates a target driving scene video containing long-tail events, providing a novel simulation verification method for autonomous driving systems. Especially after updating the trajectory data of candidate vehicles, this embodiment not only enhances the realism and diversity of the scene but also ensures the rationality and safety of vehicle dynamic behavior, thereby greatly improving the simulation training and verification effect of intelligent driving systems in complex traffic conditions and promoting the safety and maturity of autonomous driving technology.

[0049] Based on steps S11 to S14 above, scenario simulation data and lane line annotation data are acquired, and then the trajectory data to be updated is determined using preset filtering conditions and lane line annotation data. Subsequently, the trajectory data to be updated is updated to obtain the target trajectory data of candidate vehicles. Finally, scenario rendering is performed based on the target trajectory data and scenario simulation data to generate a target driving scenario video, thereby achieving efficient simulation verification of the intelligent driving system in complex long-tail scenarios. By fusing scenario simulation data and lane line annotation data, a dynamic and accurate virtual driving environment is constructed, effectively filtering and updating candidate vehicle trajectories that may affect the target vehicle's driving, ensuring a high degree of consistency between vehicle behavior in the generated long-tail scenario video and the real world. In the update processing, the trajectories of candidate vehicles are updated to exhibit complex interference patterns that could affect the target vehicle in the simulation scenario, such as sudden lane changes and emergency stops, while maintaining speed and smoothness. This significantly improves the realism and challenge of the scenario, not only enriching the test scenarios for the intelligent driving system but also providing training opportunities for safe driving under extreme conditions, enhancing the robustness and adaptability of the intelligent driving system in the face of unforeseen events. Scene rendering based on target trajectory data and scene simulation data generates target driving scene videos, providing an effective simulation verification method for intelligent driving systems. This solves the technical problems of insufficient realism and effectiveness of autonomous driving scenarios and low scene generation efficiency in related technologies.

[0050] The method for generating vehicle driving scene videos in the embodiments of this application will be further described below.

[0051] In an optional embodiment, step S11, acquiring scene simulation data includes:

[0052] Step S111: Obtain the initial driving dataset;

[0053] Step S112: Decouple the initial driving dataset to obtain first point cloud data and second point cloud data, wherein the first point cloud data is used to represent dynamic obstacle data in the road and the second point cloud data is used to represent static background data in the road.

[0054] Step S113: Initialize and reconstruct the first point cloud data in the first coordinate system to obtain vehicle object data, and initialize and reconstruct the second point cloud data in the second coordinate system to obtain background object data.

[0055] Step S114: Determine scene simulation data based on vehicle object data and background object data.

[0056] The aforementioned initial driving dataset can be a collection of raw data collected by autonomous vehicles in real-world road environments, including but not limited to information captured by vehicle sensors such as the position, speed, and acceleration of roads, obstacles, other vehicles, and pedestrians.

[0057] After acquiring the initial driving dataset, decoupling the dataset involves separating the point cloud data into dynamic obstacle data and static background data. Dynamic obstacle data refers to objects that change over time, such as moving vehicles and pedestrians, while static background data encompasses unchanging environmental features, such as road surfaces and buildings. The first point cloud data specifically refers to the point cloud information associated with dynamic obstacles in the initial driving dataset, while the second point cloud data corresponds to the static background features. After decoupling, they represent the dynamic and static components of the scene, respectively.

[0058] In this embodiment, the first coordinate system can be the vehicle's own coordinate system, and the second coordinate system can be the world coordinate system. The vehicle object data is initialized and reconstructed in the vehicle's own coordinate system, while the background object data is initialized and reconstructed in the world coordinate system. This ensures that the reconstructed vehicles and background can be correctly positioned and interacted with in the virtual environment. The vehicle object data and background object data represent the 3D models of vehicles and other dynamic obstacles on the road, as well as the static background, providing key elements for subsequent scene simulation.

[0059] The starting point of this application's embodiments is an existing initial driving dataset, which typically originates from the operational records of autonomous vehicles in real-world road environments and contains rich road and environmental information. Subsequently, this application's embodiments implement decoupling processing, classifying and separating point cloud data to distinguish dynamic obstacle data from static background data. This can usually be achieved using machine learning algorithms or traditional computer vision methods, ensuring the accuracy and efficiency of the reconstruction process.

[0060] After decoupling, this embodiment initializes and reconstructs point cloud data in both the vehicle coordinate system and the world coordinate system. This allows for detailed modeling of dynamic obstacles and static backgrounds separately. The vehicle object data, processed in the vehicle coordinate system, ensures accurate reconstruction of the vehicle model, while the background object data, processed in the world coordinate system, guarantees consistency in the scene space. Finally, by combining the vehicle object data and the background object data, a framework for scene simulation data is established. This not only preserves the basic structure and environment of the original scene but also provides greater flexibility and realism in subsequent scene editing and expansion.

[0061] Based on the above optional embodiments, by acquiring an initial driving dataset and then decoupling it to obtain first point cloud data and second point cloud data, the first point cloud data is then initialized and reconstructed in the first coordinate system to obtain vehicle object data, and the second point cloud data is initialized and reconstructed in the second coordinate system to obtain background object data. Finally, scene simulation data is determined based on the vehicle object data and background object data. This effectively separates dynamic and static elements, ensures the accuracy of the reconstructed model, and provides clear and orderly data support for subsequent vehicle behavior editing.

[0062] In an optional embodiment, step S12, determining the trajectory data to be updated using preset filtering conditions and lane line marking data includes:

[0063] Step S121: Obtain lane centerline data based on lane line annotation data;

[0064] Step S122: Determine the associated lane of the target vehicle based on the lane centerline data. The associated lane includes the current lane where the target vehicle is located, the lane adjacent to the left of the current lane, and the lane adjacent to the right of the current lane.

[0065] Step S123: In response to the relevant vehicles in the associated lane meeting the preset screening conditions, the relevant vehicles are identified as candidate vehicles, wherein the preset screening conditions indicate that the longitudinal distance between the relevant vehicles and the target vehicle is less than a preset distance threshold.

[0066] Step S124: The original trajectory data of the candidate vehicles is determined as the trajectory data to be updated.

[0067] The lane centerline data mentioned above can be lane centerline information extracted from lane line labeling data. It is used to identify the center position of each lane and is crucial for determining vehicle driving lines and lane change analysis.

[0068] The aforementioned associated lanes can be the current lane where the target vehicle is located, as well as the lanes immediately to the left and right of the current lane. Associating lanes helps to focus on the dynamics of traffic participants that may affect the target vehicle, thereby filtering out more relevant and challenging scenario events.

[0069] The aforementioned preset filtering conditions can be a set of predefined rules used to identify other vehicles that may potentially affect the target vehicle during its travels, i.e., candidate vehicles. The core consideration of the preset filtering conditions is the longitudinal distance between vehicles, ensuring that the selected candidate vehicles are close enough to the target vehicle to generate effective long-tail events.

[0070] The aforementioned raw trajectory data records information about the actual driving paths of candidate vehicles in the scene, including their position, speed, and direction of travel. This data forms the basis for scene reconstruction and vehicle behavior analysis. Once a relevant vehicle is identified as a candidate vehicle, its raw trajectory data is considered as trajectory data to be updated. This data will then be edited to introduce long-tail events, such as sudden lane changes or emergency braking, thereby generating new trajectories that have simulation and verification value for the target vehicle's intelligent driving system.

[0071] Extracting lane centerline data from lane marking data involves image processing techniques such as edge detection and Hough transform to accurately locate the center position of the lane lines, providing a foundation for subsequent lane analysis. After acquiring the lane centerline data, the associated lanes of the target vehicle are further defined. By calculating the lateral distance between the vehicle and each lane centerline, the lane where the target vehicle is located, as well as its adjacent left and right lanes, are determined, forming a set of lanes of interest. This facilitates subsequent analysis of vehicle behavior in the associated lanes.

[0072] In response to the discovery of relevant vehicles within the associated lane that meet preset screening criteria, these vehicles are marked as candidate vehicles. The preset screening criteria focus on the longitudinal distance between vehicles. When the distance between other vehicles and the target vehicle is less than a preset threshold, it indicates that the relevant vehicles may interact with the target vehicle during subsequent driving, thus becoming potential targets for generating long-tail events. Once candidate vehicles are identified, this embodiment of the application uses their original trajectory data as the trajectory data to be updated. This trajectory data serves as the starting point for editing vehicle behavior; subsequently, new trajectories will be generated based on this data to simulate long-tail scenarios and verify the response strategies of the target vehicle's intelligent driving system.

[0073] Specifically, for each segment of data in the initial driving dataset, its lane line annotation data can be obtained, thereby obtaining the center lines of all lanes [L1, L2, ..., L]. N ], in This indicates the position of a point on the center line of the lane, and K represents the number of points on a center line.

[0074] Furthermore, the lane centerline closest to the target vehicle is designated as the current lane, i.e., current lane L. ego Based on the lateral position of the lane centerline, the nearest lane centerlines on the left and right sides are taken as the left and right lanes, respectively, that is, the left adjacent lane L of the current lane. left and the adjacent lane to the right of the current lane L right Specifically, first, change the center line of the lane [L1, L2, ..., L N To establish the vehicle's coordinate system, calculate the coordinates [p1, p2, ..., p] closest to the target vehicle for each centerline.N Then calculate [p1, p2, ..., p]; N The point closest to the target vehicle is selected as the current lane, denoted as L. ego Finally, according to [p1,p2,...,p N Sort by horizontal coordinates, and take the distance L. ego The center lines of the nearest points to the left and right of the corresponding points are taken as the left and right lanes, denoted as L. left and L right Determine if the relevant vehicle is located at L. ego L left or L right And if the longitudinal distance from the target is less than a set threshold, such as 30 meters, then select the corresponding scene data for editing and updating; otherwise, skip that segment of data.

[0075] Based on the above optional embodiments, lane centerline data is obtained based on lane line annotation data, and then the associated lane of the target vehicle is determined based on the lane centerline data. Subsequently, in response to the relevant vehicles in the associated lanes meeting the preset screening conditions, the relevant vehicles are identified as candidate vehicles. Finally, the original trajectory data of the candidate vehicles is identified as the trajectory data to be updated. This lays a solid foundation for subsequent vehicle trajectory editing, ensuring that the generated target driving scene video can accurately reflect the performance of the autonomous driving system when encountering sudden and rare traffic events, and improving the efficiency and flexibility of scene generation.

[0076] In an optional embodiment, in step S13, the trajectory data to be updated is updated to obtain the target trajectory data of the candidate vehicles, including:

[0077] Step S131: Obtain the action type to be updated corresponding to the candidate vehicle, wherein the action type to be updated includes at least one of the following: lane change cut-in type, emergency braking type;

[0078] Step S132: Update the trajectory data to be updated based on the action type to be updated to obtain intermediate trajectory data;

[0079] Step S133: Optimize the intermediate trajectory data to obtain the target trajectory data.

[0080] The aforementioned candidate vehicles are other vehicles identified as potentially affecting the target vehicle's driving. The candidate vehicles meet certain distance conditions, making them objects for trajectory editing in the simulation.

[0081] Furthermore, a series of simulated actions can be designed for the candidate vehicle to create interference with the target vehicle's driving, thereby verifying the autonomous driving system's ability to respond. The types of actions to be updated mainly include two types: lane change and emergency braking. The lane change can simulate the candidate vehicle suddenly changing lanes and cutting into the target vehicle's lane; the emergency braking can simulate the candidate vehicle suddenly applying emergency braking while driving.

[0082] This application's embodiments first determine the types of actions that candidate vehicles need to simulate, selecting lane changing or emergency braking as the updated actions. This can be based on the analysis of the current scene and the target vehicle's driving state. By selecting the type of action to be updated randomly or with a preset probability, the diversity and unpredictability of the scene can be increased.

[0083] For a selected action type to be updated, embodiments of this application can edit the candidate vehicle's trajectory data to generate intermediate trajectory data. For example, if the lane change / cut-in type is selected, the vehicle trajectory is adjusted so that it cuts into the current lane at a preset speed and angle at a specific time point; if the emergency braking type is selected, the required deceleration and braking distance of the vehicle are calculated to ensure a safe stop in front of the target vehicle. Specifically, this involves Bezier interpolation algorithms or other curve generation techniques to ensure the continuity of the trajectory and the reasonableness of the speed.

[0084] To improve the quality of intermediate trajectory data, embodiments of this application can further optimize the processing to correct potential speed abrupt changes or physical inconsistencies, generating target trajectory data. Optimization typically includes, but is not limited to, smoothing of velocity and angular velocity, adjusting the trajectory to fit the road, and considering actual vehicle dynamics constraints to ensure that the edited vehicle behavior is not only challenging but also realistic and consistent with behavior in real-world driving scenarios.

[0085] Based on the above optional embodiments, by obtaining the action type to be updated corresponding to the candidate vehicle, and then updating the trajectory data to be updated based on the action type to be updated, intermediate trajectory data is obtained. Finally, the intermediate trajectory data is optimized to obtain the target trajectory data. This can simulate complex scenarios such as lane changing and emergency braking under precise control. This not only enhances the diversity of scenarios, but also ensures the physical rationality and behavioral smoothness in the simulation process. It further improves the test accuracy and robustness of the autonomous driving system when facing long-tail events, and significantly improves the efficiency and quality of scenario generation.

[0086] In an optional embodiment, step S131, obtaining the action type to be updated corresponding to the candidate vehicle includes:

[0087] In response to the candidate vehicle being located in the current lane of the target vehicle, the action type to be updated is determined to be emergency braking.

[0088] In response to the fact that the candidate vehicle is not located in the current lane of the target vehicle, the action type to be updated is determined to be a lane change cut-in type.

[0089] Specifically, the core principle of editing the behavior of other vehicles is to interfere with the normal driving of the target vehicle, that is, to control the target point of the candidate vehicle to be located at a relatively close distance in front of the own vehicle. If the candidate vehicle is located at L... left or L right If a candidate vehicle suddenly cuts into the current lane and is close to the target vehicle, the lane can be edited. If the candidate vehicle is located in lane L... ego Ahead, you can edit the prompts for candidate vehicles to brake quickly. Additionally, you can trigger different combinations of behaviors probabilistically, such as a neighboring vehicle suddenly cutting in and braking. Furthermore, you can edit the placement of candidate vehicles statically, or their rotation.

[0090] Based on the above optional embodiments, when the candidate vehicle is in the current lane where the target vehicle is located, the action type to be updated is determined to be emergency braking. Then, when the candidate vehicle is not in the current lane where the target vehicle is located, the action type to be updated is determined to be lane change and cut-in. This not only improves the efficiency and relevance of scenario generation, but also ensures that the simulation verification can more comprehensively and deeply reveal the real performance and potential defects of the autonomous driving system when facing sudden traffic events.

[0091] In an optional embodiment, in step S132, the trajectory data to be updated is updated based on the type of action to be updated, and the intermediate trajectory data obtained includes:

[0092] In response to the action type to be updated being a lane change cut-in type, the target interpolation control point is determined based on the action type to be updated;

[0093] Interpolation processing is performed based on the target interpolation control points to obtain the first interpolated trajectory data;

[0094] The first interpolated trajectory data is used to update the trajectory data to be updated, thus obtaining intermediate trajectory data.

[0095] The aforementioned target interpolation control points can be the control points for Bézier interpolation. After determining the lane change / cut-in type as the action type to be updated, the control points for Bézier interpolation can be calculated based on the vehicle's initial and final poses, as well as preset lane change / cut-in parameters (such as lane change speed and angle). The control points for Bézier interpolation determine the smoothness and rationality of the lane change / cut-in action, and are key to generating realistic lane change / cut-in trajectories.

[0096] The aforementioned first interpolated trajectory data can be continuous trajectory data of a candidate vehicle changing lanes, calculated based on the target interpolated control points through Bezier interpolation processing. This first interpolated trajectory data includes information on the vehicle's position, velocity, and acceleration changes during the lane change process, providing a basis for subsequent trajectory updates. The intermediate trajectory data can be the lane change trajectory data obtained after Bezier interpolation, containing the preliminary trajectory of the candidate vehicle's lane change action.

[0097] Specifically, the starting pose of the candidate vehicle's approach action is first set to the pose of the candidate vehicle in a certain frame when it approaches the target vehicle, and the ending pose is the vehicle's pose plus S meters in the driving direction (assuming it is the x-axis direction); the translation amount of the starting pose is taken as the starting point position p. start The initial pose corresponds to the vehicle's frontal orientation (x-axis direction) as the direction vector v of the starting point. start The translation amount at the end of the pose is used as the end point position p. end The direction of the vehicle's front (x-axis direction) corresponding to the final pose is the direction vector v of the final point. end According to p start v start p end v end Calculate the control points for Bézier interpolation:

[0098] p control_1 =p start +(p end,x -p start,x )*α*v start

[0099] p control_2 =p end -(p end,x -p start,x )*β*v end

[0100] Where α and β are dimensionless constants, specifically 0.3. The first interpolation trajectory data, i.e. trajectory pose, is obtained by Bessel interpolation.

[0101] Based on the above optional embodiments, in response to the action type to be updated being a lane change cut-in type, a target interpolation control point is determined based on the action type to be updated, and then interpolation processing is performed according to the target interpolation control point to obtain first interpolated trajectory data. Finally, the first interpolated trajectory data is used to update the trajectory data to be updated to obtain intermediate trajectory data. This can improve the realism of the scene and the effectiveness of the test. The intermediate trajectory data obtained at the end lays a solid foundation for creating realistic and targeted long-tail event scenes.

[0102] In an optional embodiment, in step S1332, the trajectory data to be updated is updated based on the type of action to be updated, and the intermediate trajectory data obtained includes:

[0103] In response to the action type to be updated being emergency braking, brake attribute data is determined based on the action type to be updated.

[0104] Interpolation processing is performed on braking attribute data to obtain second interpolated trajectory data;

[0105] The second interpolated trajectory data is used to update the trajectory data to be updated, thus obtaining intermediate trajectory data.

[0106] The aforementioned braking attribute data can be a set of parameters related to the type of emergency braking, including but not limited to braking start time, braking end time, speed before braking, and deceleration during braking, which are used to calculate the trajectory changes of candidate vehicles during emergency braking.

[0107] The second interpolated trajectory data mentioned above is continuous trajectory data of the candidate vehicle during emergency braking, generated through interpolation processing. By setting braking attribute data and applying the interpolation algorithm, a smooth transition of the candidate vehicle from normal driving to complete stop can be achieved.

[0108] Specifically, firstly, the start time t of the candidate vehicle's braking is set. start and braking duration Δt brake And calculate the speed v at the start of braking. start Further calculate the braking distance d brake =0.5*v start *Δt brake This allows us to obtain the initial position M of the vehicle's braking action. start and termination pose M end Then, a uniform deceleration proportionality coefficient r = 2x - x is adopted. 2 (x∈[0,1]) interpolation M start and M end This yields the continuous trajectory poses of the candidate vehicle during emergency braking, i.e., the second interpolated trajectory data. For static behaviors, such as static vehicle placement or flipping, the target pose of the candidate vehicle can be directly set.

[0109] Based on the above optional embodiments, in response to the action type to be updated being emergency braking, braking attribute data is determined based on the action type to be updated, and then interpolation processing is performed based on the braking attribute data to obtain second interpolated trajectory data. Finally, the second interpolated trajectory data is used to update the trajectory data to be updated to obtain intermediate trajectory data. This can improve the realism of the scene and the effectiveness of the test. The final intermediate trajectory data lays a solid foundation for creating realistic and targeted long-tail event scenarios.

[0110] In an optional embodiment, in step S133, the intermediate trajectory data is optimized to obtain the target trajectory data, including:

[0111] Step S1331: Fit the preset road surface height data to obtain local road surface height information;

[0112] Step S1332: Correct the intermediate trajectory data based on the local road surface height information to obtain the trajectory correction result;

[0113] Step S1333: Smooth the trajectory correction result to obtain the target trajectory data.

[0114] The aforementioned preset road surface height data can be a dataset of road surface height information acquired through radar point clouds or other sensors and stored after pose fusion processing. This preset road surface height data accurately reflects the actual undulations of the road and is the basis for correcting vehicle trajectories to conform to the road surface.

[0115] The aforementioned local road surface height information can be extracted from preset road surface height data through fitting processing, representing the local road height features at the location corresponding to the candidate vehicle's new trajectory. Obtaining this local road surface height information ensures that every point on the vehicle's new trajectory takes into account changes in road surface height, thereby avoiding significant issues of floating or sinking into the road surface.

[0116] The trajectory correction result obtained by correcting intermediate trajectory data using local road surface height information aims to make the vehicle trajectory closer to actual road conditions and improve the realism and credibility of the scene.

[0117] Specifically, a road surface extraction algorithm can be used to extract single-frame radar road surface points, and a Random Sample Consensus (RANSAC) algorithm can be used to remove non-road noise points. Then, multi-frame radar road surface points are fused by pose to obtain preset road surface height data. Subsequently, road surface points corresponding to the preset road surface height data are extracted based on their positions in the intermediate trajectory data, and a local road surface plane is fitted to obtain local road surface height information. Furthermore, the intermediate trajectory data is corrected based on the local road surface height information to obtain the trajectory correction result.

[0118] Because intermediate trajectory data after Bezier interpolation or height adjustment may contain abrupt changes in vehicle speed and angular velocity, resulting in abrupt and unrealistic behavior of candidate vehicles, this embodiment of the application can use an exponential moving average algorithm to smooth the speed and angular velocity of the trajectory correction result, thereby obtaining the target trajectory data. The speed and angular velocity of the trajectory correction result can be calculated from the translation and rotation components of the speed and angular velocity of the trajectory correction result, respectively.

[0119] Based on the target trajectory data and combined with the scene simulation data, 3D Gaussian splashing frame-by-frame rendering is performed to generate a target driving scene video. The target driving scene video is used to simulate and verify the intelligent driving system of the target vehicle.

[0120] Based on the above optional embodiments, by using preset road surface height data for fitting processing, local road surface height information is obtained. Then, based on the local road surface height information, intermediate trajectory data is corrected to obtain trajectory correction results. Finally, the trajectory correction results are smoothed to obtain target trajectory data. This significantly enhances the precision of scene generation, making long-tail event simulation more consistent with real-world driving scenarios and providing a more accurate and effective testing environment for autonomous driving systems.

[0121] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use and processing of the relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation portals are provided for users to choose to authorize or refuse.

[0122] According to an embodiment of this application, an apparatus embodiment for a vehicle driving scene video generation method is provided. It should be noted that the apparatus can be used to execute the above-described vehicle driving scene video generation method.

[0123] Figure 2 This is a structural block diagram of a vehicle driving scene video generation device according to an embodiment of this application, such as... Figure 2 As shown, the device includes:

[0124] The acquisition module 201 is used to acquire scene simulation data and lane line annotation data. The scene simulation data is used to represent virtual data generated by simulating at least one object located in the driving area in the current driving scenario. The lane line annotation data is used to represent the geometric attributes of lane lines in the driving road. The determination module 202 is used to determine the trajectory data to be updated using preset filtering conditions and lane line annotation data. The preset filtering conditions are used to filter out multiple candidate vehicles associated with the target vehicle during the target vehicle's movement. The trajectory data to be updated is used to represent the original trajectory data corresponding to the candidate vehicles. The update module 203 is used to update the trajectory data to be updated to obtain the target trajectory data of the candidate vehicles. The generation module 204 is used to perform scene rendering based on the target trajectory data and scene simulation data to generate a target driving scene video. The target driving scene video is used to simulate and verify the intelligent driving system of the target vehicle.

[0125] Optionally, the acquisition module 201 is further configured to: acquire an initial driving dataset; decouple the initial driving dataset to obtain a first point cloud data and a second point cloud data, wherein the first point cloud data is used to represent dynamic obstacle data in the road, and the second point cloud data is used to represent static background data in the road; perform initialization and reconstruction processing on the first point cloud data in a first coordinate system to obtain vehicle object data, and perform initialization and reconstruction processing on the second point cloud data in a second coordinate system to obtain background object data; and determine scene simulation data based on the vehicle object data and the background object data.

[0126] Optionally, the determining module 202 is further configured to: obtain lane centerline data based on lane line marking data; determine the associated lanes of the target vehicle based on the lane centerline data, wherein the associated lanes include the current lane where the target vehicle is located, the left adjacent lane of the current lane, and the right adjacent lane of the current lane; in response to the relevant vehicles in the associated lanes meeting preset filtering conditions, determine the relevant vehicles as candidate vehicles, wherein the preset filtering conditions indicate that the longitudinal distance between the relevant vehicles and the target vehicle is less than a preset distance threshold; and determine the original trajectory data of the candidate vehicles as trajectory data to be updated.

[0127] Optionally, the update module 203 is further configured to: obtain the action type to be updated corresponding to the candidate vehicle, wherein the action type to be updated includes at least one of the following: lane change cut-in type, emergency braking type; update the trajectory data to be updated based on the action type to be updated to obtain intermediate trajectory data; optimize the intermediate trajectory data to obtain target trajectory data.

[0128] Optionally, the update module 203 is further configured to: determine the action type to be updated as emergency braking type in response to the candidate vehicle being located in the current lane where the target vehicle is located; and determine the action type to be updated as lane change / cut-in type in response to the candidate vehicle not being located in the current lane where the target vehicle is located.

[0129] Optionally, the update module 203 is further configured to: in response to the action type to be updated being a lane change cut-in type, determine a target interpolation control point based on the action type to be updated; perform interpolation processing based on the target interpolation control point to obtain first interpolation trajectory data; and use the first interpolation trajectory data to update the trajectory data to be updated to obtain intermediate trajectory data.

[0130] Optionally, the update module 203 is further configured to: in response to the action type to be updated being an emergency braking type, determine braking attribute data based on the action type to be updated; perform interpolation processing based on the braking attribute data to obtain second interpolated trajectory data; and use the second interpolated trajectory data to update the trajectory data to be updated to obtain intermediate trajectory data.

[0131] Optionally, the update module 203 is further configured to: perform fitting processing on preset road surface height data to obtain local road surface height information; perform correction processing on intermediate trajectory data based on local road surface height information to obtain trajectory correction results; and perform smoothing processing on trajectory correction results to obtain target trajectory data.

[0132] It should be noted that the above modules can be implemented by software or hardware. For the latter, they can be implemented in the following ways, but are not limited to: all the above modules are located in the same processor; or, the above modules are located in different processors in any combination.

[0133] Embodiments of this application also provide an electronic device, including: a memory storing an executable program; and a processor for running the program, wherein the program executes the methods in various embodiments of this application when it runs.

[0134] Optionally, in this embodiment, the processor can be configured to perform the following steps via a computer program:

[0135] S1, acquire scene simulation data and lane line annotation data, wherein the scene simulation data is used to represent virtual data generated to simulate at least one object located in the driving area in the current driving scenario, and the lane line annotation data is used to represent the geometric properties of lane lines in the driving road;

[0136] S2, determine the trajectory data to be updated using preset filtering conditions and lane line marking data. The preset filtering conditions are used to filter out multiple candidate vehicles associated with the target vehicle during the target vehicle's movement, and the trajectory data to be updated is used to represent the original trajectory data corresponding to the candidate vehicles.

[0137] S3, update the trajectory data to be updated to obtain the target trajectory data of the candidate vehicles;

[0138] S4 performs scene rendering based on target trajectory data and scene simulation data to generate target driving scene video, which is used to simulate and verify the intelligent driving system of the target vehicle.

[0139] Embodiments of this application also provide a computer-readable storage medium including a stored executable program, wherein, when the executable program is running, it controls the device where the computer-readable storage medium is located to perform the methods of various embodiments of this application.

[0140] Optionally, in this embodiment, the storage medium may be configured to store a computer program for performing the following steps:

[0141] S1, acquire scene simulation data and lane line annotation data, wherein the scene simulation data is used to represent virtual data generated to simulate at least one object located in the driving area in the current driving scenario, and the lane line annotation data is used to represent the geometric properties of lane lines in the driving road;

[0142] S2, determine the trajectory data to be updated using preset filtering conditions and lane line marking data. The preset filtering conditions are used to filter out multiple candidate vehicles associated with the target vehicle during the target vehicle's movement, and the trajectory data to be updated is used to represent the original trajectory data corresponding to the candidate vehicles.

[0143] S3, update the trajectory data to be updated to obtain the target trajectory data of the candidate vehicles;

[0144] S4 performs scene rendering based on target trajectory data and scene simulation data to generate target driving scene video, which is used to simulate and verify the intelligent driving system of the target vehicle.

[0145] Embodiments of this application also provide a computer program product, including a computer program that, when executed by a processor, implements the methods of various embodiments of this application.

[0146] Embodiments of this application also provide a computer program product, including a non-volatile computer-readable storage medium for storing a computer program that, when executed by a processor, implements the methods in various embodiments of this application.

[0147] Embodiments of this application also provide a computer program that, when executed by a processor, implements the methods described in the various embodiments of this application.

[0148] In the above embodiments of this application, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0149] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units can be a logical functional division, and in actual implementation, there may be other division methods. For instance, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling, direct coupling, or communication connection may be through some interfaces; the indirect coupling or communication connection between units or modules may be electrical or other forms.

[0150] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0151] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0152] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard drive, magnetic disk, or optical disk.

[0153] The above description is only a preferred embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. A method for generating video of a vehicle driving scene, characterized in that, include: Acquire scene simulation data and lane line annotation data, wherein the scene simulation data is used to represent virtual data generated to simulate at least one object located in the driving area in the current driving scenario, and the lane line annotation data is used to represent the geometric attributes of lane lines in the driving road; Lane centerline data is obtained based on the lane line marking data; the associated lanes of the target vehicle are determined based on the lane centerline data, wherein the associated lanes include the current lane where the target vehicle is located, the left adjacent lane of the current lane, and the right adjacent lane of the current lane; in response to the related vehicles in the associated lanes meeting preset filtering conditions, the related vehicles are determined as candidate vehicles, wherein the preset filtering conditions indicate that the longitudinal distance between the related vehicles and the target vehicle is less than a preset distance threshold; the original trajectory data of the candidate vehicles are determined as trajectory data to be updated. Obtain the action type to be updated corresponding to the candidate vehicle, wherein the action type to be updated includes at least one of the following: lane change cut-in type, emergency braking type; update the trajectory data to be updated based on the action type to be updated to obtain intermediate trajectory data; optimize the intermediate trajectory data to obtain target trajectory data; Based on the target trajectory data and the scene simulation data, scene rendering is performed to generate a target driving scene video, wherein the target driving scene video is used to simulate and verify the intelligent driving system of the target vehicle.

2. The method according to claim 1, characterized in that, Obtaining the scenario simulation data includes: Obtain the initial driving dataset; The initial driving dataset is decoupled to obtain first point cloud data and second point cloud data, wherein the first point cloud data is used to represent dynamic obstacle data in the road and the second point cloud data is used to represent static background data in the road. The first point cloud data is initialized and reconstructed in the first coordinate system to obtain vehicle object data, and the second point cloud data is initialized and reconstructed in the second coordinate system to obtain background object data. The scene simulation data is determined based on the vehicle object data and the background object data.

3. The method according to claim 1, characterized in that, The types of actions to be updated corresponding to the candidate vehicles include: In response to the candidate vehicle being located in the current lane where the target vehicle is located, the action type to be updated is determined to be the emergency braking type; In response to the fact that the candidate vehicle is not located in the current lane where the target vehicle is located, the action type to be updated is determined to be the lane change cut-in type.

4. The method according to claim 1, characterized in that, The intermediate trajectory data is obtained by updating the trajectory data based on the action type to be updated, including: In response to the action type to be updated being the lane change cut-in type, a target interpolation control point is determined based on the action type to be updated; Interpolation processing is performed based on the target interpolation control points to obtain the first interpolation trajectory data; The intermediate trajectory data is obtained by updating the trajectory data to be updated using the first interpolated trajectory data.

5. The method according to claim 1, characterized in that, The intermediate trajectory data is obtained by updating the trajectory data based on the action type to be updated, including: In response to the action type to be updated being the emergency braking type, braking attribute data is determined based on the action type to be updated; Interpolation processing is performed on the braking attribute data to obtain the second interpolated trajectory data; The intermediate trajectory data is obtained by updating the trajectory data to be updated using the second interpolated trajectory data.

6. The method according to claim 1, characterized in that, The intermediate trajectory data is optimized to obtain the target trajectory data, which includes: Local road surface height information is obtained by fitting the preset road surface height data. Based on the local road surface height information, the intermediate trajectory data is corrected to obtain the trajectory correction result; The trajectory correction result is smoothed to obtain the target trajectory data.

7. An electronic device, characterized in that, include: Memory, which stores executable programs; A processor for running the program, wherein the program, when running, performs the method according to any one of claims 1 to 6.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored executable program, wherein, when the executable program is executed, it controls the device on which the storage medium is located to perform the method according to any one of claims 1 to 6.

Images