Method for generating vehicle driving scene video, and electronic device and storage medium

By acquiring and updating scenario simulation data and lane line annotation data, target driving scenario videos are generated, solving the problem of insufficient realism and effectiveness in autonomous driving scenario generation, and achieving efficient simulation verification and complex scenario generation.

WO2026232856A1PCT designated stage Publication Date: 2026-11-12CHERY AUTOMOBILE CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
PCT/CN2025/148269
Authority / Receiving Office
WO · WO
Patent Type
Applications
Current Assignee / Owner
Priority Date
2025-08-27
Filing Date
2025-12-31
Publication Date
2026-11-12

AI Technical Summary

Technical Problem

Existing methods for generating autonomous driving scenarios are insufficient in terms of realism and effectiveness, especially when generating long-tail scenarios, they fail to reflect actual performance and are inefficient in terms of scenario generation.

Method used

By acquiring scene simulation data and lane line annotation data, using preset filtering conditions to determine the trajectory data to be updated, and updating it, a target driving scene video is generated, and scene rendering technology is integrated to improve the realism and consistency of the scene.

Benefits of technology

It significantly improves the realism and effectiveness of autonomous driving scenarios, enriches test scenarios, enhances the robustness and adaptability of intelligent driving systems under extreme conditions, and provides an effective simulation verification method.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN2025148269_12112026_PF_FP_ABST
    Figure CN2025148269_12112026_PF_FP_ABST
Patent Text Reader

Abstract

A method for generating a vehicle driving scene video, and an electronic device and a storage medium. The method comprises: acquiring scene simulation data and lane marking annotation data; using a preset filtering condition and the lane marking annotation data to determine trajectory data to be updated; updating the trajectory data to be updated, so as to obtain target trajectory data of candidate vehicles; and on the basis of the target trajectory data and the scene simulation data, performing scene rendering to generate a target driving scene video.
Need to check novelty before this filing date? Find Prior Art

Description

Methods for generating vehicle driving scene videos, electronic devices and storage media

[0001] Cross-reference

[0002] This disclosure claims priority to Chinese Patent Application No. 202511211024.6, filed with the Chinese Patent Office on August 27, 2025, entitled “Vehicle Driving Scene Video Generation Method, Electronic Device and Storage Medium”, the entire contents of which are incorporated herein by reference. Technical Field

[0003] This disclosure relates to the field of vehicle technology, and more specifically, to a method for generating vehicle driving scene videos, an electronic device, and a storage medium. Background Technology

[0004] The rapid development of autonomous driving has spurred the demand for generating complex driving scenarios, especially long-tail scenarios, which are relatively rare but crucial to the safety of driving systems. In the field of autonomous driving scenario construction, simulation platforms and generative diffusion models each demonstrate their strengths, showcasing advantages in scene richness and realism. However, the driving scenario construction methods in these technologies still have significant limitations. While simulation platforms can accurately simulate the driving environment, their scenes differ from the real world's appearance, making it difficult to fully reflect the actual performance of autonomous driving algorithms. While generative diffusion model-based solutions enhance scene realism, they cannot ensure spatiotemporal consistency, particularly in the accuracy of ground markings such as lane lines and turn signals, further limiting their application value. Existing methods, when generating long-tail scenarios, suffer from two main problems: firstly, the simulation scene's appearance does not match the real environment well, affecting the effectiveness of algorithm testing; secondly, their ability to automatically edit vehicle behavior is insufficient, especially in the precise control and real-time generation of new trajectories. This results in low scene generation efficiency and an inability to cover sufficient long-tail driving scenarios.

[0005] There is currently no good solution to the above problems. Summary of the Invention

[0006] This disclosure provides a method for generating vehicle driving scene videos, an electronic device, and a storage medium to at least address the technical problems of insufficient realism and effectiveness of autonomous driving scenes and low scene generation efficiency in related technologies.

[0007] According to one aspect of the present disclosure, a method for generating a vehicle driving scene video is provided, comprising the following steps: acquiring scene simulation data and lane line annotation data, wherein the scene simulation data is configured to represent virtual data generated to simulate at least one object located within a driving area in the current driving scene, and the lane line annotation data is configured to represent the geometric attributes of lane lines in the driving road; determining trajectory data to be updated using preset filtering conditions and lane line annotation data, wherein the preset filtering conditions are configured to filter out multiple candidate vehicles associated with the target vehicle during the target vehicle's movement, and the trajectory data to be updated is configured to represent the original trajectory data corresponding to the candidate vehicles; updating the trajectory data to be updated to obtain target trajectory data of the candidate vehicles; and performing scene rendering based on the target trajectory data and scene simulation data to generate a target driving scene video, wherein the target driving scene video is configured to simulate and verify the intelligent driving system of the target vehicle.

[0008] Optionally, acquiring scene simulation data includes the following steps: Acquiring an initial driving dataset; decoupling the initial driving dataset to obtain first point cloud data and second point cloud data, wherein the first point cloud data represents dynamic obstacle data in the road, and the second point cloud data represents static background data in the road; performing initialization and reconstruction processing on the first point cloud data in a first coordinate system to obtain vehicle object data, and performing initialization and reconstruction processing on the second point cloud data in a second coordinate system to obtain background object data; determining scene simulation data based on the vehicle object data and background object data.

[0009] Optionally, determining the trajectory data to be updated using preset filtering conditions and lane line annotation data includes the following steps: Obtaining lane centerline data based on lane line annotation data; determining the associated lanes of the target vehicle based on the lane centerline data, wherein the associated lanes include the current lane where the target vehicle is located, the lane adjacent to the left of the current lane, and the lane adjacent to the right of the current lane; in response to relevant vehicles in the associated lanes meeting preset filtering conditions, identifying the relevant vehicles as candidate vehicles, wherein the preset filtering conditions indicate that the longitudinal distance between the relevant vehicles and the target vehicle is less than a preset distance threshold; and determining the original trajectory data of the candidate vehicles as the trajectory data to be updated.

[0010] Optionally, updating the trajectory data to be updated to obtain the target trajectory data of the candidate vehicles includes the following steps: obtaining the action type to be updated corresponding to the candidate vehicles, wherein the action type to be updated includes at least one of the following: lane change / cut-in type, emergency braking type; updating the trajectory data to be updated based on the action type to be updated to obtain intermediate trajectory data; optimizing the intermediate trajectory data to obtain the target trajectory data.

[0011] Optionally, obtaining the action type to be updated corresponding to the candidate vehicle includes the following steps: In response to the candidate vehicle being located in the current lane where the target vehicle is located, the action type to be updated is determined to be an emergency braking type; in response to the candidate vehicle not being located in the current lane where the target vehicle is located, the action type to be updated is determined to be a lane change / cut-in type.

[0012] Optionally, updating the trajectory data to be updated based on the action type to obtain intermediate trajectory data includes the following steps: Responding to the action type to be updated being a lane change / cut-in type, a target interpolation control point is determined based on the action type; interpolation is performed based on the target interpolation control point to obtain first interpolated trajectory data; the first interpolated trajectory data is used to update the trajectory data to be updated to obtain intermediate trajectory data.

[0013] Optionally, updating the trajectory data to be updated based on the action type to obtain intermediate trajectory data includes the following steps: In response to the action type to be updated being an emergency braking type, braking attribute data is determined based on the action type; interpolation is performed on the braking attribute data to obtain second interpolated trajectory data; the second interpolated trajectory data is used to update the trajectory data to be updated to obtain intermediate trajectory data.

[0014] Optionally, optimizing the intermediate trajectory data to obtain the target trajectory data includes the following steps: Fitting the data using preset road surface height data to obtain local road surface height information; correcting the intermediate trajectory data based on the local road surface height information to obtain a trajectory correction result; and smoothing the trajectory correction result to obtain the target trajectory data.

[0015] According to another aspect of the embodiments of this disclosure, a vehicle driving scene video generation apparatus is also provided, comprising the following modules: an acquisition module, configured to acquire scene simulation data and lane line annotation data, wherein the scene simulation data is configured to represent virtual data generated by simulating at least one object located within the driving area in the current driving scene, and the lane line annotation data is configured to represent the geometric attributes of lane lines in the driving road; a determination module, configured to determine trajectory data to be updated using preset filtering conditions and lane line annotation data, wherein the preset filtering conditions are configured to filter out multiple candidate vehicles associated with the target vehicle during the movement of the target vehicle, and the trajectory data to be updated is configured to represent the original trajectory data corresponding to the candidate vehicles; an update module, configured to update the trajectory data to be updated to obtain target trajectory data of the candidate vehicles; and a generation module, configured to perform scene rendering based on the target trajectory data and scene simulation data to generate a target driving scene video, wherein the target driving scene video is configured to simulate and verify the intelligent driving system of the target vehicle.

[0016] Optionally, the acquisition module is further configured to perform the following steps: acquire an initial driving dataset; decouple the initial driving dataset to obtain first point cloud data and second point cloud data, wherein the first point cloud data is set to represent dynamic obstacle data in the road, and the second point cloud data is set to represent static background data in the road; perform initialization and reconstruction processing on the first point cloud data in a first coordinate system to obtain vehicle object data, and perform initialization and reconstruction processing on the second point cloud data in a second coordinate system to obtain background object data, wherein the first coordinate system is the vehicle coordinate system, and the second coordinate system is the world coordinate system; determine scene simulation data based on the vehicle object data and background object data.

[0017] Optionally, the determination module is further configured to perform the following steps: Obtain lane centerline data based on lane line annotation data; determine the associated lanes of the target vehicle based on the lane centerline data, wherein the associated lanes include the current lane where the target vehicle is located, the left adjacent lane of the current lane, and the right adjacent lane of the current lane; in response to relevant vehicles in the associated lanes meeting preset filtering conditions, determine the relevant vehicles as candidate vehicles, wherein the preset filtering conditions indicate that the longitudinal distance between the relevant vehicles and the target vehicle is less than a preset distance threshold; determine the original trajectory data of the candidate vehicles as trajectory data to be updated.

[0018] Optionally, the update module is further configured to perform the following steps: Obtain the action type to be updated corresponding to the candidate vehicle, wherein the action type to be updated includes at least one of the following: lane change cut-in type, emergency braking type; update the trajectory data to be updated based on the action type to be updated to obtain intermediate trajectory data; optimize the intermediate trajectory data to obtain target trajectory data.

[0019] Optionally, the update module is also configured to perform the following steps: In response to the candidate vehicle being located in the current lane where the target vehicle is located, determine that the action type to be updated is an emergency braking type; in response to the candidate vehicle not being located in the current lane where the target vehicle is located, determine that the action type to be updated is a lane change / cut-in type.

[0020] Optionally, the update module is further configured to perform the following steps: In response to the action type to be updated being a lane change cut-in type, a target interpolation control point is determined based on the action type to be updated; interpolation processing is performed based on the target interpolation control point to obtain first interpolated trajectory data; the first interpolated trajectory data is used to update the trajectory data to be updated to obtain intermediate trajectory data.

[0021] Optionally, the update module is further configured to perform the following steps: In response to the action type to be updated being an emergency braking type, determine braking attribute data based on the action type to be updated; perform interpolation processing based on the braking attribute data to obtain second interpolated trajectory data; use the second interpolated trajectory data to update the trajectory data to be updated to obtain intermediate trajectory data.

[0022] Optionally, the update module is also configured to perform the following steps: Fitting the preset road surface height data to obtain local road surface height information; correcting the intermediate trajectory data based on the local road surface height information to obtain a trajectory correction result; and smoothing the trajectory correction result to obtain the target trajectory data.

[0023] According to another aspect of the present disclosure, an electronic device is also provided, including: a memory storing an executable program; and a processor configured to run the program, wherein the program executes the methods in the various embodiments of the present disclosure when it runs.

[0024] According to another aspect of the embodiments of the present disclosure, a computer-readable storage medium is also provided, the computer-readable storage medium including a stored executable program, wherein, when the executable program is executed, it controls the device where the computer-readable storage medium is located to perform the methods of the various embodiments of the present disclosure.

[0025] According to another aspect of the embodiments of this disclosure, a computer program product is also provided, including a computer program that, when executed by a processor, implements the methods of various embodiments of this disclosure.

[0026] According to another aspect of the embodiments of this disclosure, a computer program product is also provided, including a non-volatile computer-readable storage medium storing a computer program that, when executed by a processor, implements the methods of various embodiments of this disclosure.

[0027] According to another aspect of the embodiments of this disclosure, a computer program is also provided, which, when executed by a processor, implements the methods of the various embodiments of this disclosure.

[0028] In this embodiment, scenario simulation data and lane line annotation data are acquired, and then the trajectory data to be updated is determined using preset filtering conditions and lane line annotation data. Subsequently, the trajectory data to be updated is updated to obtain the target trajectory data of candidate vehicles. Finally, scenario rendering is performed based on the target trajectory data and scenario simulation data to generate a target driving scenario video, thereby achieving efficient simulation verification of the intelligent driving system in complex long-tail scenarios. By fusing scenario simulation data and lane line annotation data, a dynamic and accurate virtual driving environment is constructed, effectively filtering and updating candidate vehicle trajectories that may affect the target vehicle's driving, ensuring a high degree of consistency between vehicle behavior in the generated long-tail scenario video and real-world vehicle behavior. In the update processing, the trajectories of candidate vehicles are updated to exhibit complex interference patterns (e.g., sudden lane changes, emergency stops, etc.) in the simulated scenario, while maintaining speed and smoothness. This significantly improves the realism and challenge of the scenario, enriching the test scenarios for the intelligent driving system and providing training opportunities for safe driving under extreme conditions, enhancing the robustness and adaptability of the intelligent driving system in the face of unforeseen events. Scene rendering based on target trajectory data and scene simulation data generates target driving scene videos, providing an effective simulation verification method for intelligent driving systems. This solves the technical problems of insufficient realism and effectiveness of autonomous driving scenarios and low scene generation efficiency in related technologies. Attached Figure Description

[0029] The accompanying drawings, which are included to provide a further understanding of this disclosure and form part of this disclosure, illustrate exemplary embodiments of the present disclosure and are used to explain the disclosure, but do not constitute an undue limitation of the disclosure. In the drawings:

[0030] Figure 1 is a flowchart of a vehicle driving scene video generation method according to an embodiment of the present disclosure;

[0031] Figure 2 is a structural block diagram of a vehicle driving scene video generation device according to an embodiment of the present disclosure. Detailed Implementation

[0032] To enable those skilled in the art to better understand the present disclosure, the technical solutions of the present disclosure will be clearly and completely described below with reference to the accompanying drawings of the embodiments. Obviously, the described embodiments are only some embodiments of the present disclosure, and not all embodiments. Based on the embodiments of the present disclosure, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present disclosure.

[0033] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this disclosure are intended to distinguish similar objects and are not necessarily intended to describe a specific order or sequence. It should be understood that such data are interchangeable where appropriate so that the embodiments of this disclosure described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or device that comprises a series of steps or components is not necessarily limited to those steps or components explicitly listed, but may include other steps or components not explicitly listed or inherent to such processes, methods, products, or devices.

[0034] According to an embodiment of this disclosure, a method embodiment for generating vehicle driving scene video is provided. It should be noted that the steps shown in the flowcharts of the accompanying drawings are executed in a computer system such as a set of computer-executable instructions. Although a logical order is shown in the flowcharts, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0035] This method embodiment is executed in an electronic device or similar computing device that includes memory and a processor. Taking a computer terminal as an example, the computer terminal includes one or more processors (processors include, but are not limited to, central processing units (CPUs), graphics processing units (GPUs), digital signal processing (DSP) chips, microcontroller units (MCUs), field programmable gate arrays (FPGAs), neural network processors (NPUs), tensor processors (TPUs), artificial intelligence (AI) type processors, etc.) and memory configured to store data. Optionally, the computer terminal also includes transmission devices, input / output devices, and display devices configured for communication functions. Those skilled in the art will understand that the above structural description is merely illustrative and does not limit the structure of the computer terminal. For example, a computer terminal may include more or fewer components than those described above, or have a different configuration than those described above.

[0036] The memory can be configured to store computer programs, such as application software programs and modules, like the computer program corresponding to the vehicle driving scene video generation method in this embodiment. The processor executes various functional applications and data processing by running the computer program stored in the memory, thereby realizing the aforementioned vehicle driving scene video generation method. The memory may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory may further include memory remotely located relative to the processor, and these remote memories are connected to the mobile terminal via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0037] The transmission device is configured to receive or send data via a network. Specific examples of the network described above may include a wireless network provided by the mobile terminal's communication provider. In one example, the transmission device includes a Network Interface Controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the transmission device is a Radio Frequency (RF) module configured to communicate with the Internet wirelessly.

[0038] Display devices include, for example, touchscreen liquid crystal displays (LCDs) and touch displays (also referred to as "touchscreens" or "touch displays"). These LCDs enable users to interact with the user interface of a mobile terminal. In some embodiments, the mobile terminal has a graphical user interface (GUI), through which users interact with the GUI by touching and / or gesturing on a touch-sensitive surface. Optional human-computer interaction functions include: creating web pages, drawing, word processing, creating electronic documents, playing games, video conferencing, instant messaging, sending and receiving emails, call interfaces, playing digital video, playing digital music, and / or web browsing, etc. Executable instructions configured to perform the above human-computer interaction functions are configured / stored in one or more processor-executable computer program products or readable storage media.

[0039] This embodiment provides a method for generating vehicle driving scene videos. Figure 1 is a flowchart of a method for generating vehicle driving scene videos according to an embodiment of this disclosure. As shown in Figure 1, the process includes the following steps.

[0040] Step S11: Obtain scene simulation data and lane line annotation data. The scene simulation data is set to represent virtual data generated by simulating at least one object located in the driving area in the current driving scenario. The lane line annotation data is set to represent the geometric attributes of lane lines in the driving road.

[0041] Step S12: Determine the trajectory data to be updated using preset filtering conditions and lane line marking data. The preset filtering conditions are set to filter out multiple candidate vehicles associated with the target vehicle during the target vehicle's movement. The trajectory data to be updated is set to represent the original trajectory data corresponding to the candidate vehicles.

[0042] Step S13: Update the trajectory data to be updated to obtain the target trajectory data of the candidate vehicles;

[0043] Step S14: Render the scene based on the target trajectory data and scene simulation data to generate a target driving scene video. The target driving scene video is set to simulate and verify the intelligent driving system of the target vehicle.

[0044] The aforementioned scenario simulation data is a type of virtual data designed to recreate a specific driving scenario in a computer environment. It includes information about various objects within the driving area, such as vehicles, pedestrians, and static obstacles, as well as the corresponding motion states and environmental conditions. The scenario simulation data is acquired using a simulation platform capable of simulating various driving environments, including but not limited to traffic flow, weather conditions, and road types. Highly customized scenario simulation data is generated by setting scenario parameters such as the number of vehicles, pedestrian behavior, and time points. The scenario simulation data includes the state information of all objects within the driving area (e.g., position, speed, acceleration) and environmental conditions within the driving area (e.g., lighting, weather, time). The scenario simulation data is also acquired using 3D reconstruction technology, employing techniques such as Street Gaussians to reconstruct the real-world road environment with high fidelity. This involves extracting information from images taken from multiple angles or from LiDAR point cloud data to reconstruct a 3D model of the environment, and then adding dynamic behaviors by editing vehicle instances to generate the scenario simulation data.

[0045] The lane line annotation data mentioned above records the geometric attributes of lane lines in roads. This data includes, but is not limited to, information such as the position, length, width, and curvature of lane lines, which is crucial for simulating vehicle driving routes and determining whether vehicles are adhering to lane rules. For existing datasets, lane line annotation data is manually generated by professionals using image annotation software to annotate lane lines in road images or videos. This manual annotation includes marking information such as the position, length, width, and curvature of lane lines. Lane line perception models built using deep learning techniques, such as convolutional neural networks, can automatically identify and annotate lane lines directly from videos or images. Combining data from multiple sensor sources, such as vehicle vision sensors, LiDAR, and radar, and using data fusion algorithms (e.g., Kalman filtering, particle filtering), more accurate lane line annotation data can be obtained.

[0046] The above preset filtering conditions are a set of predefined filtering criteria. The preset filtering conditions are set to automatically identify objects that may have a potential impact on the target vehicle during the target vehicle's driving process, especially other vehicles in lanes near the target vehicle, i.e., candidate vehicles.

[0047] The aforementioned trajectory data to be updated refers to the original data of the candidate vehicle traveling along a preset path in the original scenario. The trajectory data to be updated needs to be modified to form vehicle behavior that includes long-tail events. The target trajectory data of the candidate vehicle in the simulated scenario obtained after updating the trajectory data to be updated aims to simulate complex driving situations that pose a challenge to the target vehicle.

[0048] Based on preset filtering criteria, candidate vehicles in the scenario simulation data are located and identified. These criteria consider the relative position and speed between the target vehicle and candidate vehicles, ensuring that selected vehicles will trigger long-tail events. With the assistance of lane line annotation data, it is more accurate to determine whether candidate vehicles are located in lanes near the target vehicle, thereby identifying which vehicles' trajectories need to be updated.

[0049] After identifying candidate vehicles, a heuristic strategy is used to update the original trajectory data of each vehicle, generating a series of preset disturbance behaviors (e.g., sudden cut-in, emergency braking, etc.). The update process not only modifies the driving path of the candidate vehicles but also ensures that the new trajectory matches the road conditions, and that the changes in speed and angular velocity are smooth, making the simulation scenario more realistic and reliable, and improving the effectiveness of the autonomous driving system simulation verification.

[0050] After obtaining the target trajectory data, a target driving scene video is generated using 3D Gaussian splashing technology based on the target trajectory data and scene simulation data. During this scene rendering process, the updated vehicle trajectory data and complete scene details are integrated to create a highly realistic virtual driving environment, particularly suitable for testing the reaction capability of the target vehicle's intelligent driving system when encountering long-tail events. Therefore, this embodiment effectively generates a target driving scene video containing long-tail events, providing a novel simulation verification method for autonomous driving systems. Especially after updating the trajectory data of candidate vehicles, this embodiment not only enhances the realism and diversity of the scene but also ensures the rationality and safety of vehicle dynamic behavior, thereby greatly improving the simulation training and verification effect of intelligent driving systems in complex traffic conditions and promoting the safety and maturity of autonomous driving technology.

[0051] Based on steps S11 to S14 above, scenario simulation data and lane line annotation data are acquired, and then the trajectory data to be updated is determined using preset filtering conditions and lane line annotation data. Subsequently, the trajectory data to be updated is updated to obtain the target trajectory data of candidate vehicles. Finally, scenario rendering is performed based on the target trajectory data and scenario simulation data to generate a target driving scenario video, thereby achieving efficient simulation verification of the intelligent driving system in complex long-tail scenarios. By fusing scenario simulation data and lane line annotation data, a dynamic and accurate virtual driving environment is constructed, effectively filtering and updating candidate vehicle trajectories that may affect the target vehicle's driving, ensuring a high degree of consistency between vehicle behavior in the generated long-tail scenario video and real-world vehicle behavior. In the update processing, the trajectories of candidate vehicles are updated to exhibit complex interference patterns (e.g., sudden lane changes, emergency stops, etc.) in the simulation scenario, while maintaining speed and smoothness. This significantly improves the realism and challenge of the scenario, not only enriching the test scenarios for the intelligent driving system but also providing training opportunities for safe driving under extreme conditions, enhancing the robustness and adaptability of the intelligent driving system in the face of unforeseen events. Scene rendering based on target trajectory data and scene simulation data generates target driving scene videos, providing an effective simulation verification method for intelligent driving systems. This solves the technical problems of insufficient realism and effectiveness of autonomous driving scenarios and low scene generation efficiency in related technologies.

[0052] The method for generating vehicle driving scene videos in the embodiments of this disclosure will be further described below.

[0053] In an optional embodiment, step S11, obtaining scene simulation data includes the following steps.

[0054] Step S111: Obtain the initial driving dataset;

[0055] Step S112: Decouple the initial driving dataset to obtain first point cloud data and second point cloud data, wherein the first point cloud data is set to represent dynamic obstacle data in the road and the second point cloud data is set to represent static background data in the road.

[0056] Step S113: Initialize and reconstruct the first point cloud data in the first coordinate system to obtain vehicle object data, and initialize and reconstruct the second point cloud data in the second coordinate system to obtain background object data.

[0057] Step S114: Determine scene simulation data based on vehicle object data and background object data.

[0058] The aforementioned initial driving dataset is a collection of raw data collected by autonomous vehicles in real-world road environments, including but not limited to information captured by vehicle sensors such as the location, speed, and acceleration of roads, obstacles, other vehicles, and pedestrians.

[0059] After acquiring the initial driving dataset, decoupling the dataset involves separating the point cloud data into dynamic obstacle data and static background data. Dynamic obstacle data refers to objects that change over time (e.g., moving vehicles and pedestrians), while static background data encompasses unchanging environmental features (e.g., road surfaces, buildings). The first point cloud data specifically refers to the point cloud information associated with dynamic obstacles in the initial driving dataset, while the second point cloud data corresponds to the point cloud information associated with static background features. After decoupling, the first and second point cloud data represent the dynamic and static components of the scene, respectively.

[0060] In this embodiment, the first coordinate system is the vehicle's own coordinate system, and the second coordinate system is the world coordinate system. Vehicle object data is initialized and reconstructed in the vehicle's own coordinate system, while background object data is initialized and reconstructed in the world coordinate system, ensuring that the reconstructed vehicles and background can be correctly positioned and interacted with in the virtual environment. Vehicle object data represents vehicles and other dynamic obstacles on the road, while background object data represents a 3D model of the static background. Vehicle object data and background object data provide key elements for subsequent scene simulation.

[0061] The starting point of this disclosure is an existing initial driving dataset, which typically originates from the operational records of autonomous vehicles in real-world road environments and contains rich road and environmental information. Subsequently, this disclosure implements decoupling processing, classifying and separating point cloud data to distinguish dynamic obstacle data from static background data. This step of separating dynamic obstacle data from static background data is typically achieved using machine learning algorithms or traditional computer vision methods, ensuring the accuracy and efficiency of the reconstruction process.

[0062] After decoupling, this embodiment initializes and reconstructs point cloud data in both the vehicle coordinate system and the world coordinate system. This allows for detailed modeling of dynamic obstacles and static backgrounds separately. The vehicle object data, processed in the vehicle coordinate system, ensures accurate reconstruction of the vehicle model, while the background object data, processed in the world coordinate system, guarantees consistency in the scene space. Finally, by combining the vehicle object data and the background object data, a framework for scene simulation data is established. This not only preserves the basic structure and environment of the original scene but also provides greater flexibility and realism in subsequent scene editing and expansion.

[0063] Based on the above optional embodiments, by acquiring an initial driving dataset and then decoupling it to obtain first point cloud data and second point cloud data, the first point cloud data is then initialized and reconstructed in the first coordinate system to obtain vehicle object data, and the second point cloud data is initialized and reconstructed in the second coordinate system to obtain background object data. Finally, scene simulation data is determined based on the vehicle object data and background object data. This effectively separates dynamic elements from static elements, ensures the accuracy of the reconstructed model, and provides clear and orderly data support for subsequent vehicle behavior editing.

[0064] In an optional embodiment, step S12, determining the trajectory data to be updated using preset filtering conditions and lane line marking data includes the following steps.

[0065] Step S121: Obtain lane centerline data based on lane line annotation data;

[0066] Step S122: Determine the associated lane of the target vehicle based on the lane centerline data. The associated lane includes the current lane where the target vehicle is located, the lane adjacent to the left of the current lane, and the lane adjacent to the right of the current lane.

[0067] Step S123: In response to the relevant vehicles in the associated lane meeting the preset screening conditions, the relevant vehicles are identified as candidate vehicles, wherein the preset screening conditions indicate that the longitudinal distance between the relevant vehicles and the target vehicle is less than a preset distance threshold.

[0068] Step S124: The original trajectory data of the candidate vehicles is determined as the trajectory data to be updated.

[0069] The lane centerline data mentioned above is lane centerline information extracted from lane line labeling data. The lane centerline data is set to identify the center position of each lane and is crucial for determining vehicle travel lines and lane change analysis.

[0070] The aforementioned associated lanes are the current lane where the target vehicle is located, as well as the lanes immediately to the left and right of the current lane. The associated lanes help to focus on the dynamics of traffic participants that may affect the target vehicle, thereby filtering out more relevant and challenging scenario events.

[0071] The aforementioned preset filtering conditions are a set of predefined rules. These conditions are set to identify other vehicles that may potentially affect the target vehicle during its journey, i.e., candidate vehicles. The core consideration of these preset filtering conditions is the longitudinal distance between vehicles, ensuring that the selected candidate vehicles are close enough to the target vehicle to generate effective long-tail events.

[0072] The aforementioned raw trajectory data records information about the actual driving paths of candidate vehicles in the scene (including location, speed, and direction of travel), serving as the foundation for scene reconstruction and vehicle behavior analysis. Once a relevant vehicle is identified as a candidate vehicle, its raw trajectory data is considered as trajectory data to be updated. It will then be edited to introduce long-tail events (e.g., sudden lane changes or emergency braking) to generate new trajectories that have simulation and verification value for the target vehicle's intelligent driving system.

[0073] Lane centerline data is extracted from lane marking data, involving image processing techniques (e.g., edge detection, Hough transform). This centerline data is used to precisely locate the center position of the lane lines, providing a foundation for subsequent lane analysis. After acquiring the lane centerline data, the associated lanes of the target vehicle are further defined. By calculating the lateral distance between the vehicle and each lane centerline, the lane containing the target vehicle, as well as its adjacent left and right lanes, are determined, forming a set of lanes of interest. This facilitates subsequent analysis of vehicle behavior in these associated lanes.

[0074] In response to the discovery of relevant vehicles within the associated lane that meet preset screening criteria, these vehicles are marked as candidate vehicles. The preset screening criteria focus on the longitudinal distance between vehicles. When the distance between other vehicles and the target vehicle is less than a preset threshold, it indicates that the relevant vehicles may interact with the target vehicle during subsequent driving, thus becoming potential targets for generating long-tail events. Once candidate vehicles are identified, this embodiment of the disclosure immediately uses the original trajectory data of the candidate vehicles as the trajectory data to be updated. The trajectory data to be updated serves as the starting point for editing vehicle behavior. Subsequently, new trajectories will be generated based on the trajectory data to be updated to simulate long-tail scenarios and verify the response strategies of the target vehicle's intelligent driving system.

[0075] Specifically, for each segment of data in the initial driving dataset, its lane line annotation data is obtained, thereby obtaining all lane centerlines [L1, L2, ..., L]. N ], in This indicates the position of a point on the center line of the lane, and K represents the number of points on a center line.

[0076] Furthermore, the lane centerline closest to the target vehicle is designated as the current lane, i.e., current lane L. ego Based on the lateral position of the lane centerline, the nearest lane centerlines on the left and right sides are taken as the left and right lanes, respectively, that is, the left adjacent lane L of the current lane. left and the adjacent lane to the right of the current lane L right Specifically, first, change the center line of the lane [L1, L2, ..., L N To establish the vehicle's coordinate system, calculate the coordinates [p1, p2, ..., p] closest to the target vehicle for each centerline. N Then calculate [p1, p2, ..., p]; N The point closest to the target vehicle is selected as the current lane, denoted as L. ego Finally, according to [p1,p2,...,p N Sort by horizontal coordinates, and take the distance L. ego The center lines of the two nearest points to the left and right of the corresponding point are designated as the left and right lanes, respectively, and are denoted as L. left and L right Determine if the relevant vehicle is located at L. ego L left or L right And whether the longitudinal distance to the target vehicle is less than a set threshold (e.g., 30 meters), if the vehicle is located at L ego L left or L right If the longitudinal distance to the target vehicle is less than a set threshold, then the corresponding scenario data will be selected for editing and updating. If the vehicle is not located at L...ego L left or L right If the longitudinal distance from the target vehicle is not less than a set threshold, then skip that segment of data.

[0077] Based on the above optional embodiments, lane centerline data is obtained based on lane line annotation data, and then the associated lane of the target vehicle is determined based on the lane centerline data. Subsequently, in response to the relevant vehicles in the associated lanes meeting the preset screening conditions, the relevant vehicles are identified as candidate vehicles. Finally, the original trajectory data of the candidate vehicles is identified as the trajectory data to be updated. This lays a solid foundation for subsequent vehicle trajectory editing, ensuring that the generated target driving scene video can accurately reflect the performance of the autonomous driving system when encountering sudden and rare traffic events, and improving the efficiency and flexibility of scene generation.

[0078] In an optional embodiment, step S13 involves updating the trajectory data to be updated to obtain the target trajectory data of the candidate vehicle, which includes the following steps.

[0079] Step S131: Obtain the action type to be updated corresponding to the candidate vehicle, wherein the action type to be updated includes at least one of the following: lane change cut-in type, emergency braking type;

[0080] Step S132: Update the trajectory data to be updated based on the action type to be updated to obtain intermediate trajectory data;

[0081] Step S133: Optimize the intermediate trajectory data to obtain the target trajectory data.

[0082] The aforementioned candidate vehicles are other vehicles identified as potentially affecting the target vehicle's driving. These candidate vehicles meet certain distance conditions, making them the objects of trajectory editing in the simulation.

[0083] Furthermore, a series of simulated actions designed for the candidate vehicle are intended to create interference with the target vehicle's driving, in order to verify the autonomous driving system's ability to respond. The types of actions to be updated mainly include two types: lane change and emergency braking. The lane change simulates the candidate vehicle suddenly changing lanes and cutting into the target vehicle's lane; the emergency braking simulates the candidate vehicle suddenly applying emergency braking while driving.

[0084] This embodiment first determines the types of actions that candidate vehicles need to simulate, selecting either lane-changing or emergency braking as the updated actions. Determining the types of actions that candidate vehicles need to simulate is based on analysis of the current scene and the driving state of the target vehicle. Selecting the action type to be updated using random or preset probabilities increases the diversity and unpredictability of the scene.

[0085] For a selected action type to be updated, embodiments of this disclosure edit the candidate vehicle's trajectory data to generate intermediate trajectory data. For example, if the lane change / cut-in type is selected, the vehicle trajectory is adjusted so that it cuts into the current lane at a preset speed and preset angle at a specific time point; if the emergency braking type is selected, the required deceleration and braking distance of the vehicle are calculated to ensure a safe stop in front of the target vehicle. Editing the candidate vehicle's trajectory data to generate intermediate trajectory data specifically involves Bézier interpolation algorithms or other curve generation techniques to ensure the continuity of the trajectory and the reasonableness of the speed.

[0086] To improve the quality of intermediate trajectory data, embodiments of this disclosure further optimize processing operations, correcting potential speed abrupt changes or physical inconsistencies to generate target trajectory data. Optimization operations typically include, but are not limited to, speed smoothing and angular velocity smoothing, trajectory alignment with the road, and consideration of actual vehicle dynamics constraints to ensure that the edited vehicle behavior is not only challenging but also realistic and consistent with vehicle behavior in real-world driving scenarios.

[0087] Based on the above optional embodiments, by obtaining the action type to be updated corresponding to the candidate vehicle, and then updating the trajectory data to be updated based on the action type to be updated, intermediate trajectory data is obtained. Finally, the intermediate trajectory data is optimized to obtain the target trajectory data. This can simulate complex scenarios such as lane changing and emergency braking under precise control. This not only enhances the diversity of scenarios, but also ensures the physical rationality and behavioral smoothness in the simulation process. It further improves the test accuracy and robustness of the autonomous driving system when facing long-tail events, and significantly improves the efficiency and quality of scenario generation.

[0088] In an optional embodiment, step S131, obtaining the action type to be updated corresponding to the candidate vehicle includes the following steps.

[0089] In response to the candidate vehicle being located in the current lane of the target vehicle, the action type to be updated is determined to be emergency braking.

[0090] In response to the fact that the candidate vehicle is not located in the current lane of the target vehicle, the action type to be updated is determined to be a lane change cut-in type.

[0091] Specifically, the core principle of editing the behavior of other vehicles is to interfere with the normal driving of the target vehicle, that is, to control the target point of the candidate vehicle to be located at a relatively close distance in front of the own vehicle. If the candidate vehicle is located at L... left or L right If a candidate vehicle suddenly cuts into the current lane and is close to the target vehicle, the lane can be edited. If the candidate vehicle is located in lane L... egoAhead, you can edit the prompts for candidate vehicles to brake quickly. You can also trigger different combinations of behaviors probabilistically (e.g., a neighboring vehicle suddenly cutting in and braking). Additionally, you can edit the placement of candidate vehicles statically, vehicle rollover, and other actions.

[0092] Based on the above optional embodiments, when the candidate vehicle is in the current lane where the target vehicle is located, the action type to be updated is determined to be emergency braking. Then, when the candidate vehicle is not in the current lane where the target vehicle is located, the action type to be updated is determined to be lane change and cut-in. This not only improves the efficiency and relevance of scenario generation, but also ensures that the simulation verification can more comprehensively and deeply reveal the real performance and potential defects of the autonomous driving system when facing sudden traffic events.

[0093] In an optional embodiment, step S132, updating the trajectory data to be updated based on the action type to be updated to obtain intermediate trajectory data includes the following steps.

[0094] In response to the action type to be updated being a lane change cut-in type, the target interpolation control point is determined based on the action type to be updated;

[0095] Interpolation processing is performed based on the target interpolation control points to obtain the first interpolated trajectory data;

[0096] The first interpolated trajectory data is used to update the trajectory data to be updated, thus obtaining intermediate trajectory data.

[0097] The aforementioned target interpolation control points are the control points for Bezier interpolation. After determining the lane change / cut-in type as the action type to be updated, the control points for Bezier interpolation are calculated based on the vehicle's initial and final poses, as well as preset lane change / cut-in parameters (such as lane change speed and angle). The control points for Bezier interpolation determine the smoothness and rationality of the lane change / cut-in action, and are crucial for generating realistic lane change / cut-in trajectories.

[0098] The aforementioned first interpolated trajectory data is processed by performing Bezier interpolation to calculate the continuous trajectory data (i.e., the first interpolated trajectory data) of the candidate vehicle's lane change and cut-in based on the target interpolation control points. The first interpolated trajectory data includes information on the vehicle's position, velocity, and acceleration changes during the lane change process, providing a basis for subsequent trajectory updates. The intermediate trajectory data is the lane change and cut-in trajectory data obtained after Bezier interpolation, containing the preliminary trajectory of the candidate vehicle's lane change and cut-in action.

[0099] Specifically, the starting pose of the candidate vehicle's approach action is first set to the pose of the candidate vehicle in a certain frame when it approaches the target vehicle, and the ending pose is the vehicle's pose plus S meters in the driving direction (assuming it is the x-axis direction); the translation amount of the starting pose is taken as the starting point position p. startThe initial pose corresponds to the vehicle's frontal orientation (x-axis direction) as the direction vector v of the starting point. start The translation amount at the end pose is used as the end point position p. end The direction of the vehicle's front (x-axis direction) corresponding to the final pose is the direction vector v of the final point. end According to p start v start p end v end Calculate the control points for Bessel interpolation: p control_1 =p start +(p end,x -p start,x )*α*v start p contTol_2 =p end -(p end,x -p start,x )*β*v end

[0100] Where α and β are dimensionless constants, specifically taking the value 0.3. The first interpolation trajectory data, i.e. the trajectory pose, is obtained by Bessel interpolation.

[0101] Based on the above optional embodiments, in response to the action type to be updated being a lane change cut-in type, a target interpolation control point is determined based on the action type to be updated, and then interpolation processing is performed according to the target interpolation control point to obtain first interpolated trajectory data. Finally, the first interpolated trajectory data is used to update the trajectory data to be updated to obtain intermediate trajectory data. This can improve the realism of the scene and the effectiveness of the test. The intermediate trajectory data obtained at the end lays a solid foundation for creating realistic and targeted long-tail event scenarios.

[0102] In an optional embodiment, step S1332, updating the trajectory data to be updated based on the action type to be updated, and obtaining intermediate trajectory data includes the following steps.

[0103] In response to the action type to be updated being emergency braking, brake attribute data is determined based on the action type to be updated.

[0104] Interpolation processing is performed on braking attribute data to obtain second interpolated trajectory data;

[0105] The second interpolated trajectory data is used to update the trajectory data to be updated, thus obtaining intermediate trajectory data.

[0106] The aforementioned braking attribute data is a set of parameters related to the emergency braking type. The braking attribute data includes, but is not limited to, the braking start time, the braking end time, the speed of the candidate vehicle before braking, and the deceleration of the candidate vehicle during braking. The braking attribute data is set to calculate the trajectory changes of the candidate vehicle during emergency braking.

[0107] The second interpolated trajectory data mentioned above is continuous trajectory data of the candidate vehicle during emergency braking, generated through interpolation processing. By setting braking attribute data and applying the interpolation algorithm, a smooth transition of the candidate vehicle from normal driving to complete stop is achieved.

[0108] Specifically, firstly, the start time t of the candidate vehicle's braking is set. start and braking duration Δt brake And calculate the speed v at the start of braking. start Further calculate the braking distance d brake =0.5*v start *Δt brake This allows us to obtain the initial position M of the vehicle's braking action. start and termination pose M end Then, a uniform deceleration proportionality coefficient r = 2x - x is adopted. 2 (x∈[0,1]) interpolation M staTt and M end This yields the continuous trajectory poses of the candidate vehicle during emergency braking, i.e., the second interpolated trajectory data. For static behaviors, such as static vehicle placement or vehicle rollover, the target pose of the candidate vehicle can be directly set.

[0109] Based on the above optional embodiments, in response to the action type to be updated being emergency braking, braking attribute data is determined based on the action type to be updated, and then interpolation processing is performed based on the braking attribute data to obtain second interpolated trajectory data. Finally, the second interpolated trajectory data is used to update the trajectory data to be updated to obtain intermediate trajectory data. This can improve the realism of the scene and the effectiveness of the test. The final intermediate trajectory data lays a solid foundation for creating realistic and targeted long-tail event scenarios.

[0110] In an optional embodiment, step S133, optimizing the intermediate trajectory data to obtain the target trajectory data includes the following steps.

[0111] Step S1331: Fit the preset road surface height data to obtain local road surface height information;

[0112] Step S1332: Correct the intermediate trajectory data based on the local road surface height information to obtain the trajectory correction result;

[0113] Step S1333: Smooth the trajectory correction result to obtain the target trajectory data.

[0114] The aforementioned preset road surface height data is acquired through radar point clouds or other sensors and stored after pose fusion processing. This preset road surface height data is a dataset of road surface height information. The preset road surface height data is designed to accurately reflect the true undulations of the road surface and is the basis for correcting vehicle trajectories to conform to the road surface.

[0115] The aforementioned local road surface height information is extracted from preset road surface height data through fitting processing, representing the local road height features at the location corresponding to the candidate vehicle's new trajectory. Obtaining this local road surface height information ensures that every point on the vehicle's new trajectory takes into account changes in road surface height, thereby avoiding significant issues of floating or sinking into the road surface.

[0116] The trajectory correction result obtained by correcting intermediate trajectory data using local road surface height information aims to make the vehicle trajectory closer to actual road conditions and improve the realism and credibility of the scene.

[0117] Specifically, a road surface extraction algorithm is used to extract single-frame radar road surface points, and a Random Sample Consensus (RANSAC) algorithm is used to remove non-road noise points. Then, multi-frame radar road surface points are fused by pose to obtain preset road surface height data. Subsequently, road surface points corresponding to the preset road surface height data are extracted based on their positions in the intermediate trajectory data, and a local road surface plane is fitted to obtain local road surface height information. Furthermore, the intermediate trajectory data is corrected based on the local road surface height information to obtain the trajectory correction result.

[0118] Because intermediate trajectory data after Bezier interpolation or height adjustment may contain sudden changes in vehicle speed and angular velocity, resulting in abrupt and unrealistic behavior of candidate vehicles, this embodiment of the present disclosure employs an exponential moving average algorithm to smooth the speed and angular velocity of the trajectory correction result, thereby obtaining the target trajectory data. The speed of the trajectory correction result is calculated from the translation and rotation components of the speed of the trajectory correction result, and the angular velocity of the trajectory correction result is calculated from the translation and rotation components of the angular velocity of the trajectory correction result.

[0119] Based on the target trajectory data and combined with the scene simulation data, 3D Gaussian splash rendering is performed frame by frame to generate a target driving scene video. The target driving scene video is set to simulate and verify the intelligent driving system of the target vehicle.

[0120] Based on the above optional embodiments, by using preset road surface height data for fitting processing, local road surface height information is obtained. Then, based on the local road surface height information, intermediate trajectory data is corrected to obtain trajectory correction results. Finally, the trajectory correction results are smoothed to obtain target trajectory data. This significantly enhances the precision of scene generation, making long-tail event simulation more consistent with real-world driving scenarios and providing a more accurate and effective testing environment for autonomous driving systems.

[0121] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data set for analysis, stored data, displayed data, etc.) involved in this disclosure are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use and processing of the relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation entry points are provided for users to choose to authorize or refuse.

[0122] According to an embodiment of this disclosure, an apparatus embodiment for a vehicle driving scene video generation method is provided. It should be noted that the apparatus is configured to perform the above-described vehicle driving scene video generation method.

[0123] Figure 2 is a structural block diagram of a vehicle driving scene video generation device according to an embodiment of the present disclosure. As shown in Figure 2, the device includes the following modules.

[0124] The acquisition module 201 is configured to acquire scene simulation data and lane line annotation data. The scene simulation data represents virtual data generated by simulating at least one object located within the driving area in the current driving scenario, and the lane line annotation data represents the geometric attributes of lane lines in the driving road. The determination module 202 is configured to determine the trajectory data to be updated using preset filtering conditions and lane line annotation data. The preset filtering conditions are configured to filter out multiple candidate vehicles associated with the target vehicle during the target vehicle's movement, and the trajectory data to be updated represents the original trajectory data corresponding to the candidate vehicles. The update module 203 is configured to update the trajectory data to be updated to obtain the target trajectory data of the candidate vehicles. The generation module 204 is configured to perform scene rendering based on the target trajectory data and scene simulation data to generate a target driving scene video. The target driving scene video is configured to simulate and verify the intelligent driving system of the target vehicle.

[0125] Optionally, the acquisition module 201 is further configured to perform the following steps: acquire an initial driving dataset; decouple the initial driving dataset to obtain first point cloud data and second point cloud data, wherein the first point cloud data is configured to represent dynamic obstacle data in the road, and the second point cloud data is configured to represent static background data in the road; perform initialization and reconstruction processing on the first point cloud data in a first coordinate system to obtain vehicle object data, and perform initialization and reconstruction processing on the second point cloud data in a second coordinate system to obtain background object data; determine scene simulation data based on the vehicle object data and the background object data.

[0126] Optionally, the determining module 202 is further configured to perform the following steps: Obtain lane centerline data based on lane line marking data; determine the associated lanes of the target vehicle based on the lane centerline data, wherein the associated lanes include the current lane where the target vehicle is located, the left adjacent lane of the current lane, and the right adjacent lane of the current lane; in response to relevant vehicles in the associated lanes meeting preset filtering conditions, determine the relevant vehicles as candidate vehicles, wherein the preset filtering conditions indicate that the longitudinal distance between the relevant vehicles and the target vehicle is less than a preset distance threshold; determine the original trajectory data of the candidate vehicles as trajectory data to be updated.

[0127] Optionally, the update module 203 is further configured to perform the following steps: Obtain the action type to be updated corresponding to the candidate vehicle, wherein the action type to be updated includes at least one of the following: lane change cut-in type, emergency braking type; update the trajectory data to be updated based on the action type to be updated to obtain intermediate trajectory data; optimize the intermediate trajectory data to obtain target trajectory data.

[0128] Optionally, the update module 203 is further configured to perform the following steps: In response to the candidate vehicle being located in the current lane where the target vehicle is located, determine that the action type to be updated is an emergency braking type; in response to the candidate vehicle not being located in the current lane where the target vehicle is located, determine that the action type to be updated is a lane change / cut-in type.

[0129] Optionally, the update module 203 is further configured to perform the following steps: In response to the action type to be updated being a lane change cut-in type, a target interpolation control point is determined based on the action type to be updated; interpolation processing is performed based on the target interpolation control point to obtain first interpolated trajectory data; the first interpolated trajectory data is used to update the trajectory data to be updated to obtain intermediate trajectory data.

[0130] Optionally, the update module 203 is further configured to perform the following steps: In response to the action type to be updated being an emergency braking type, brake attribute data is determined based on the action type to be updated; interpolation processing is performed based on the brake attribute data to obtain second interpolated trajectory data; the second interpolated trajectory data is used to update the trajectory data to be updated to obtain intermediate trajectory data.

[0131] Optionally, the update module 203 is further configured to perform the following steps: Fitting the preset road surface height data to obtain local road surface height information; correcting the intermediate trajectory data based on the local road surface height information to obtain a trajectory correction result; and smoothing the trajectory correction result to obtain the target trajectory data.

[0132] It should be noted that the above modules are implemented through software or hardware. For hardware, they are implemented in the following ways, but not limited to these: all the above modules are located in the same processor; or, the above modules are located in different processors in any combination.

[0133] Embodiments of this disclosure also provide an electronic device, including: a memory storing an executable program; and a processor configured to run the program, wherein the program executes the methods of various embodiments of this disclosure when it runs.

[0134] Optionally, in this embodiment, the processor is configured to perform the following steps via a computer program.

[0135] S1, acquire scene simulation data and lane line annotation data, wherein the scene simulation data is set to represent virtual data generated to simulate at least one object located in the driving area in the current driving scene, and the lane line annotation data is set to represent the geometric properties of lane lines in the driving road;

[0136] S2, determine the trajectory data to be updated using preset filtering conditions and lane line marking data. The preset filtering conditions are set to filter out multiple candidate vehicles associated with the target vehicle during the target vehicle's movement, and the trajectory data to be updated is set to represent the original trajectory data corresponding to the candidate vehicles.

[0137] S3, update the trajectory data to be updated to obtain the target trajectory data of the candidate vehicles;

[0138] S4, based on target trajectory data and scene simulation data, performs scene rendering to generate target driving scene video, wherein the target driving scene video is set to simulate and verify the intelligent driving system of the target vehicle.

[0139] Embodiments of this disclosure also provide a computer-readable storage medium including a stored executable program, wherein, when the executable program is executed, it controls the device on which the computer-readable storage medium resides to perform the methods of the various embodiments of this disclosure.

[0140] Optionally, in this embodiment, the storage medium is configured to store a computer program configured to perform the following steps.

[0141] S1, acquire scene simulation data and lane line annotation data, wherein the scene simulation data is set to represent virtual data generated to simulate at least one object located in the driving area in the current driving scene, and the lane line annotation data is set to represent the geometric properties of lane lines in the driving road;

[0142] S2, determine the trajectory data to be updated using preset filtering conditions and lane line marking data. The preset filtering conditions are set to filter out multiple candidate vehicles associated with the target vehicle during the target vehicle's movement, and the trajectory data to be updated is set to represent the original trajectory data corresponding to the candidate vehicles.

[0143] S3, update the trajectory data to be updated to obtain the target trajectory data of the candidate vehicles;

[0144] S4, based on target trajectory data and scene simulation data, performs scene rendering to generate target driving scene video, wherein the target driving scene video is set to simulate and verify the intelligent driving system of the target vehicle.

[0145] Embodiments of this disclosure also provide a computer program product, including a computer program that, when executed by a processor, implements the methods of various embodiments of this disclosure.

[0146] Embodiments of this disclosure also provide a computer program product including a non-volatile computer-readable storage medium configured to store a computer program that, when executed by a processor, implements the methods described in various embodiments of this disclosure.

[0147] Embodiments of this disclosure also provide a computer program that, when executed by a processor, implements the methods described in the various embodiments of this disclosure.

[0148] In the above embodiments of this disclosure, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0149] In the several embodiments provided in this disclosure, it should be understood that the disclosed technical content can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of components is a logical functional division, and in actual implementation, there may be other division methods, such as multiple components or components combined or integrated into another system, or some features being ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed between components or modules is through some interface, indirect coupling or communication connection, and is electrical or other forms.

[0150] The components described as separate parts may or may not be physically separate. The parts shown as components may or may not be physical components; that is, they may be located in one place or distributed across multiple components. Some or all of the components are selected according to actual needs to achieve the purpose of this embodiment.

[0151] Furthermore, in the various embodiments of this disclosure, the functional components are integrated into a single processing component, or each component exists physically separately, or two or more components are integrated into one component. The integrated components are implemented either in hardware or as software functional components.

[0152] If the integrated components are implemented as software functional components and sold or used as independent products, they are stored in a computer-readable storage medium. Based on this understanding, the technical solution of this disclosure, in essence, or the part that contributes to the prior art, or all or part of the technical solution, is embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this disclosure. The aforementioned storage medium includes various media for storing program code, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0153] The above description is only a preferred embodiment of this disclosure. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principles of this disclosure, and these improvements and modifications should also be considered within the scope of protection of this disclosure. Industrial applicability

[0154] In this embodiment, scenario simulation data and lane line annotation data are acquired, and then the trajectory data to be updated is determined using preset filtering conditions and lane line annotation data. Subsequently, the trajectory data to be updated is updated to obtain the target trajectory data of candidate vehicles. Finally, scenario rendering is performed based on the target trajectory data and scenario simulation data to generate a target driving scenario video, thereby achieving efficient simulation verification of the intelligent driving system in complex long-tail scenarios. By fusing scenario simulation data and lane line annotation data, a dynamic and accurate virtual driving environment is constructed, effectively filtering and updating candidate vehicle trajectories that may affect the target vehicle's driving, ensuring a high degree of consistency between vehicle behavior in the generated long-tail scenario video and real-world vehicle behavior. In the update processing, the trajectories of candidate vehicles are updated to exhibit complex interference patterns (e.g., sudden lane changes, emergency stops, etc.) in the simulated scenario, while maintaining speed and smoothness. This significantly improves the realism and challenge of the scenario, enriching the test scenarios for the intelligent driving system and providing training opportunities for safe driving under extreme conditions, enhancing the robustness and adaptability of the intelligent driving system in the face of unforeseen events. Scene rendering based on target trajectory data and scene simulation data generates target driving scene videos, providing an effective simulation verification method for intelligent driving systems. This solves the technical problems of insufficient realism and effectiveness of autonomous driving scenarios and low scene generation efficiency in related technologies.

Claims

1. A method for generating a vehicle driving scene video, comprising: Acquire scene simulation data and lane line annotation data, wherein the scene simulation data is set to represent virtual data generated to simulate at least one object located in the driving area in the current driving scenario, and the lane line annotation data is set to represent the geometric attributes of lane lines in the driving road; The trajectory data to be updated is determined using preset filtering conditions and the lane line marking data. The preset filtering conditions are set to filter out multiple candidate vehicles associated with the target vehicle during the target vehicle's movement. The trajectory data to be updated is set to represent the original trajectory data corresponding to the candidate vehicles. The trajectory data to be updated is updated to obtain the target trajectory data of the candidate vehicle; Based on the target trajectory data and the scene simulation data, scene rendering is performed to generate a target driving scene video, wherein the target driving scene video is set to simulate and verify the intelligent driving system of the target vehicle.

2. The method according to claim 1, wherein, Obtaining the scenario simulation data includes: Obtain the initial driving dataset; The initial driving dataset is decoupled to obtain first point cloud data and second point cloud data, wherein the first point cloud data is set to represent dynamic obstacle data in the road, and the second point cloud data is set to represent static background data in the road. The first point cloud data is initialized and reconstructed in a first coordinate system to obtain vehicle object data, and the second point cloud data is initialized and reconstructed in a second coordinate system to obtain background object data, wherein the first coordinate system is the vehicle coordinate system and the second coordinate system is the world coordinate system. The scene simulation data is determined based on the vehicle object data and the background object data.

3. The method according to claim 1, wherein, Determining the trajectory data to be updated using the preset filtering conditions and the lane line marking data includes: Obtain lane centerline data based on the lane line marking data; The associated lane of the target vehicle is determined based on the lane centerline data, wherein the associated lane includes the current lane where the target vehicle is located, the left adjacent lane of the current lane, and the right adjacent lane of the current lane. In response to a relevant vehicle in the associated lane satisfying the preset filtering condition, the relevant vehicle is identified as the candidate vehicle, wherein the preset filtering condition indicates that the longitudinal distance between the relevant vehicle and the target vehicle is less than a preset distance threshold; The original trajectory data of the candidate vehicle is determined as the trajectory data to be updated.

4. The method according to claim 3, wherein, The associated lanes of the target vehicle are determined based on the lane centerline data, including: Calculate the lateral distances between the target vehicle and the center lines of multiple lanes; The associated lane of the target vehicle is determined based on the lateral distance.

5. The method according to claim 1, wherein, The process of updating the trajectory data to be updated to obtain the target trajectory data of the candidate vehicle includes: Obtain the action type to be updated corresponding to the candidate vehicle, wherein the action type to be updated includes at least one of the following: lane change cut-in type, emergency braking type; The trajectory data to be updated is updated based on the action type to be updated to obtain intermediate trajectory data; The intermediate trajectory data is optimized to obtain the target trajectory data.

6. The method according to claim 5, wherein, The lane change cut-in type is set to simulate the action of the candidate vehicle changing lanes and cutting into the lane where the target vehicle is located, and the emergency braking type is set to simulate the behavior of the candidate vehicle braking suddenly while driving.

7. The method according to claim 5, wherein, The types of actions to be updated corresponding to the candidate vehicles include: In response to the candidate vehicle being located in the current lane where the target vehicle is located, the action type to be updated is determined to be the emergency braking type; In response to the fact that the candidate vehicle is not located in the current lane where the target vehicle is located, the action type to be updated is determined to be the lane change cut-in type.

8. The method according to claim 5, wherein, The intermediate trajectory data is obtained by updating the trajectory data based on the action type to be updated, including: In response to the action type to be updated being the lane change cut-in type, a target interpolation control point is determined based on the action type to be updated; Interpolation processing is performed based on the target interpolation control points to obtain the first interpolation trajectory data; The intermediate trajectory data is obtained by updating the trajectory data to be updated using the first interpolated trajectory data.

9. The method according to claim 8, wherein, In response to the action type to be updated being the lane change cut-in type, determining the target interpolation control point based on the action type to be updated includes: In response to the action type to be updated being the lane change entry type, the lane change parameters of the candidate vehicle are obtained, wherein the lane change parameters include the vehicle's starting pose, the vehicle's ending pose, and preset lane change entry parameters. The target interpolation control point is calculated based on the updated parameters.

10. The method according to claim 5, wherein, The intermediate trajectory data is obtained by updating the trajectory data based on the action type to be updated, including: In response to the fact that the action type to be updated is the emergency braking type, braking attribute data is determined based on the action type to be updated; Interpolation processing is performed on the braking attribute data to obtain the second interpolated trajectory data; The intermediate trajectory data is obtained by updating the trajectory data to be updated using the second interpolated trajectory data.

11. The method according to claim 10, wherein, In response to the action type to be updated being the emergency braking type, determining the braking attribute data based on the action type to be updated includes: In response to the action type to be updated being the emergency braking type, the braking parameters of the candidate vehicle are obtained, wherein the braking parameters include the braking start time, the braking end time, the speed of the candidate vehicle before braking, and the deceleration of the candidate vehicle during braking. The braking attribute data is determined based on the braking parameters.

12. The method according to claim 5, wherein, The intermediate trajectory data is optimized to obtain the target trajectory data, which includes: Local road surface height information is obtained by fitting the preset road surface height data. Based on the local road surface height information, the intermediate trajectory data is corrected to obtain the trajectory correction result; The trajectory correction result is smoothed to obtain the target trajectory data.

13. The method according to claim 12, wherein, The method further includes: Extracting initial radar road surface points from multiple frames based on a road surface extraction algorithm; By using a preset sampling algorithm to remove non-road noise points from the initial radar road points in the multiple frames, multiple radar road points are obtained. The multi-frame radar road surface points are subjected to pose fusion processing to obtain the preset road surface height data.

14. An electronic device, comprising: Memory, which stores executable programs; A processor configured to run the program, wherein the program, when running, performs the method according to any one of claims 1 to 13.

15. A computer-readable storage medium comprising a stored executable program, wherein, When the executable program is executed, it controls the device containing the storage medium to perform the method described in any one of claims 1 to 13.