Traffic scene generation method and device, equipment, storage medium and program product

Abstract

The invention relates to the technical field of artificial intelligence, in particular to a traffic scene generation method and device, equipment, a storage medium and a program product, and the method comprises the steps: generating a reference path of a completely controlled vehicle based on preset road structure information, scene description information and initial position information of the completely controlled vehicle, and generating a traffic scene based on the reference path. Calculating a cost function corresponding to the basic behavior; and based on the cost function, the road structure information, the scene description information and the initial position information of the vehicle, track data of a completely controlled vehicle and track data of an incompletely controlled vehicle are generated by using a pre-trained diffusion model. By calculating the reference path, constructing the cost functions for the basic behaviors, and controlling the longitudinal distance of the vehicle running along the reference path and the transverse distance of the vehicle deviating from the reference path, the trajectory data finally generated by the diffusion model can be more real and accord with the scene description information, and the success rate of generating the complex traffic scene can be improved.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of artificial intelligence, and in particular, to a traffic scene generation method and device, equipment, a storage medium and a program product. BACKGROUND

[0002] With the development of science and technology, deep learning models are increasingly widely used in autonomous driving, and have shown good performance in various tasks such as perception, decision-making, and control. The traffic scene data required for training these deep learning models is a key factor in determining the performance of the deep learning models and also affects the overall performance of the autonomous driving algorithm. However, the current traffic scene data has the problems of high acquisition cost, limited data scale, and low proportion of complex traffic scenes, which limits the performance of the deep learning model.

[0003] To solve these problems, some commonly used methods are to generate a large number of complex traffic scenes through generative models, and to collect the required traffic scene data based on the generated traffic scenes. However, the performance of these generative models is limited, and the generated traffic scenes often do not meet the scene description of complex traffic scenes, with a low success rate. SUMMARY

[0004] The present disclosure is proposed in view of the above problems, and provides a traffic scene generation method and device, equipment, a storage medium and a program product.

[0005] According to one aspect of the present disclosure, a traffic scene generation method is provided, comprising: generating a reference path of a fully controlled vehicle based on preset road structure information, scene description information, and initial position information of the fully controlled vehicle, the scene description information including behavior description information of the fully controlled vehicle; calculating a cost function corresponding to a basic behavior of the fully controlled vehicle based on the reference path of the fully controlled vehicle, the basic behavior being obtained based on the behavior description information, the cost function being used to control a longitudinal distance of the fully controlled vehicle along the reference path and a lateral distance of the fully controlled vehicle deviating from the reference path; generating trajectory data of the fully controlled vehicle and trajectory data of an incompletely controlled vehicle based on the cost function, the road structure information, the scene description information, the initial position information of the fully controlled vehicle, and the initial position information of the incompletely controlled vehicle, using a pre-trained diffusion model; obtaining a traffic scene meeting the scene description information based on the trajectory data of the fully controlled vehicle, the trajectory data of the incompletely controlled vehicle, and the road structure information.

[0006] Furthermore, according to one aspect of the traffic scene generation method of this disclosure, based on the reference path of the fully controlled vehicle, a cost function corresponding to the basic behavior of the fully controlled vehicle is calculated, including: Based on the position information of multiple projection points of the reference path, the Frenet coordinates corresponding to each projection point are calculated; the projection point is a point on the initial trajectory of the fully controlled vehicle traveling under the instruction of the behavior description information; the abscissa of the Frenet coordinate is used to characterize the longitudinal distance traveled by the fully controlled vehicle along the reference path; the ordinate of the Frenet coordinate is used to characterize the lateral distance of the fully controlled vehicle deviating from the reference path. Based on the Frenet coordinates corresponding to each of the projection points, calculate the first cost function corresponding to the horizontal distance and the second cost function corresponding to the vertical distance; Based on the first cost function and the second cost function, the cost function corresponding to the basic behavior of the fully controlled vehicle is determined.

[0007] Furthermore, according to a traffic scene generation method of one aspect of this disclosure, based on the Frenet coordinates corresponding to each of the projection points, a first cost function corresponding to the lateral distance and a second cost function corresponding to the longitudinal distance are calculated, including: Based on the Frenet coordinates corresponding to each projection point, the ordinate values ​​corresponding to each projection point are obtained. The sum of the absolute values ​​of the ordinate values ​​of each projection point is calculated to obtain a first cost function corresponding to the horizontal distance. Furthermore, based on the Frenet coordinates corresponding to the target projection point, the target vertical distance is calculated, where the target projection point is the start and end point of the initial trajectory within a preset time period. The abscissa value corresponding to the end point of the initial trajectory is obtained, and the difference between the target vertical distance and the abscissa value is calculated. If the difference is greater than a preset threshold, a second cost function corresponding to the vertical distance is obtained based on the difference. If the difference is less than or equal to a preset threshold, the second cost function corresponding to the vertical distance is obtained based on the preset threshold.

[0008] Furthermore, according to a traffic scene generation method of one aspect of this disclosure, based on the first cost function and the second cost function, a cost function corresponding to the basic behavior of the fully controlled vehicle is determined, including: When the basic behavior of the fully controlled vehicle is to go straight or change lanes, the cost function corresponding to the basic behavior of the fully controlled vehicle is the first cost function; When the basic behavior of the fully controlled vehicle is turning or making a U-turn, the product of the second cost function and the function coefficients is calculated, and the cost function corresponding to the basic behavior of the fully controlled vehicle is the sum of the first cost function and the product.

[0009] Furthermore, according to one aspect of the traffic scene generation method of this disclosure, a reference path for the fully controlled vehicle is generated based on preset road structure information, scene description information, and initial position information of the fully controlled vehicle, including: Based on preset road structure information, a road topology map is obtained. The road structure information includes the location information of lane segments, and the road topology map is used to represent the connection relationship between the lane segments. Based on the initial position information of the fully controlled vehicle and the position information of the lane segment, the current lane segment where the fully controlled vehicle is located is determined; Based on the behavioral description information included in the scenario description information, the basic behavior of the fully controlled vehicle is obtained; Based on the basic behavior of the fully controlled vehicle and its current lane segment, a reference path for the fully controlled vehicle is searched and generated in the road topology map.

[0010] Furthermore, according to a traffic scene generation method of one aspect of this disclosure, based on the behavior description information included in the scene description information, the basic behavior of the fully controlled vehicle is obtained, including: Based on a pre-built basic behavior library, the behavior description information included in the scene description information is decomposed to obtain the basic behavior of the fully controlled vehicle. The basic actions include at least one of going straight, changing lanes, turning, and making a U-turn.

[0011] According to another aspect of this disclosure, a traffic scene generation apparatus is provided, comprising: The path generation module is used to generate a reference path for the fully controlled vehicle based on preset road structure information, scene description information, and initial position information of the fully controlled vehicle. The scene description information includes the behavior description information of the fully controlled vehicle. The calculation module is used to calculate the cost function corresponding to the basic behavior of the fully controlled vehicle based on the reference path of the fully controlled vehicle. The basic behavior is obtained based on the behavior description information. The cost function is used to control the longitudinal distance traveled by the fully controlled vehicle along the reference path and the lateral distance deviating from the reference path. The trajectory generation module is used to generate trajectory data of the fully controlled vehicle and trajectory data of the incompletely controlled vehicle based on the cost function, the road structure information, the scene description information, the initial position information of the fully controlled vehicle and the initial position information of the incompletely controlled vehicle, using a pre-trained diffusion model. The scene generation module is used to obtain a traffic scene that conforms to the scene description information based on the trajectory data of the fully controlled vehicle, the trajectory data of the partially controlled vehicle, and the road structure information.

[0012] According to another aspect of this disclosure, a computer device is provided, including a memory, a processor, and a computer program stored in the memory, the processor executing the computer program to implement the method of one aspect above.

[0013] According to another aspect of this disclosure, a computer-readable storage medium is provided having a computer program stored thereon that, when executed by a processor, implements the method of one aspect above.

[0014] According to another aspect of this disclosure, a computer program product is provided, including a computer program that, when executed by a processor, implements the method of the above-described aspect.

[0015] As will be described in detail below, a traffic scene generation method, apparatus, device, storage medium, and program product according to embodiments of the present disclosure construct cost functions for the basic behaviors of fully controlled vehicles by calculating reference paths, and control the longitudinal distance traveled by fully controlled vehicles along the reference paths and the lateral distance deviating from the reference paths. This enables the trajectory data finally generated by the diffusion model to be more realistic and consistent with the scene description information, which is beneficial to improving the success rate of generating complex traffic scenes.

[0016] It should be understood that both the foregoing general description and the following detailed description are exemplary and intended to provide further illustration of the claimed technology. Attached Figure Description

[0017] The above and other objects, features, and advantages of this disclosure will become more apparent from the more detailed description of the embodiments thereof in conjunction with the accompanying drawings. The drawings are provided to further illustrate the embodiments of this disclosure and form part of the specification. They are used together with the embodiments of this disclosure to explain the disclosure and do not constitute a limitation thereof. In the drawings, the same reference numerals generally represent the same components or steps.

[0018] Figure 1 This is a flowchart illustrating a traffic scene generation method according to an embodiment of the present disclosure.

[0019] Figure 2 This is a flowchart illustrating another method for generating traffic scenes according to an embodiment of the present disclosure.

[0020] Figure 3 This is a schematic diagram illustrating the application of a cost function according to an embodiment of the present disclosure.

[0021] Figure 4 This is a schematic diagram of the structure of a traffic scene generation device according to an embodiment of the present disclosure.

[0022] Figure 5 This is a schematic diagram illustrating the structure of a computer device according to an embodiment of the present disclosure.

[0023] Figure 6 This is a schematic diagram illustrating a computer program product according to an embodiment of the present disclosure. Detailed Implementation

[0024] To make the objectives, technical solutions, and advantages of this disclosure more apparent, exemplary embodiments according to this disclosure will now be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are merely some embodiments of this disclosure, and not all embodiments of this disclosure. It should be understood that this disclosure is not limited to the exemplary embodiments described herein.

[0025] With technological advancements, deep learning models are increasingly being used in autonomous driving, demonstrating excellent performance, particularly in tasks such as perception, decision-making, and control. The traffic scene data required to train these deep learning models is a key factor determining their performance and also influences the overall performance of autonomous driving algorithms. However, current traffic scene data suffers from high collection costs, limited data scale, and a small proportion of complex traffic scenarios, thus limiting the performance of deep learning models.

[0026] To address these issues, a common solution is to generate large batches of complex traffic scenarios using generative models, and then collect the required traffic scenario data based on these generated scenarios. However, these generative models have limited performance, and the generated traffic scenarios often do not accurately reflect the characteristics of complex traffic scenarios, resulting in a low success rate.

[0027] The above description, with reference to the accompanying drawings, illustrates a traffic scene generation method, apparatus, device, storage medium, and program product according to embodiments of the present disclosure. By calculating a reference path, cost functions are constructed for the basic behaviors of a fully controlled vehicle, controlling the longitudinal distance traveled by the fully controlled vehicle along the reference path and the lateral distance deviating from the reference path. This enables the trajectory data ultimately generated by the diffusion model to be more realistic and consistent with the scene description information, thereby improving the success rate of generating complex traffic scenes.

[0028] For generating conditions in complex traffic scenarios, a basic behavior library including going straight, changing lanes, turning, and making a U-turn is pre-built. The behavior description of vehicles in complex traffic scenarios is decomposed into a combination of multiple basic behaviors, which simplifies the condition generation problem.

[0029] A cost function framework based on Frenet coordinates is constructed to control the lateral and longitudinal distances between the vehicle and the reference path for condition generation, which facilitates simple and unified condition control. Compared with existing technologies, it can handle more complex condition generation problems and improves the generation effect.

[0030] To facilitate understanding of this embodiment, a detailed description of the traffic scene generation method disclosed in this disclosure is provided first. The execution entity of the traffic scene generation method provided in this disclosure is generally a computer device with certain computing capabilities. This computer device may include, for example, a terminal device, a server, or other processing devices. The terminal device may be a user equipment (UE), mobile device, user terminal, terminal, cellular phone, cordless phone, personal digital assistant (PDA), handheld device, computing device, in-vehicle device, wearable device, etc. In some possible implementations, this traffic scene generation method can be implemented by a processor calling computer-readable instructions stored in memory.

[0031] like Figure 1 The diagram shows a flowchart of a traffic scene generation method provided in this embodiment of the present disclosure, the method comprising S101-S104: S101: Generate a reference path for the fully controlled vehicle based on preset road structure information, scene description information, and the initial position information of the fully controlled vehicle.

[0032] The road structure information describes the road information in the traffic scene, such as the number, location, and connection relationships of lane segments, as well as the setting of intersections and traffic lights. Scene description information includes behavioral descriptions of fully controlled vehicles, such as vehicle A first going straight, then overtaking, and then turning left at the next intersection. Initial position information could be, for example, the lane segment and orientation currently occupied by the fully controlled vehicle.

[0033] Regarding fully controlled vehicles, assuming the user defines traffic participants in the traffic scenario as ABCDE, where vehicles ABC need to be controlled, that is, the behavior description information of vehicles ABC needs to be given, then vehicles ABC are fully controlled vehicles. Vehicles D and E do not need to be controlled by behavior description information, only initial position information needs to be set, then vehicles D and E are not fully controlled vehicles.

[0034] Specifically, generating a reference path includes the following steps 1-4: Step 1: Obtain a road topology map based on the preset road structure information.

[0035] The road structure information includes the location information of lane segments, and the road topology map is used to represent the connection relationship between lane segments. Optionally, a lane segment can be understood as follows: assuming a road is 2km long and includes three lanes (left, center, and right), and this road is divided longitudinally into 50 segments, then the number of lane segments is 50 × 3 = 150.

[0036] Specifically, road structure information is expressed as a combination of lane segments, each lane segment including... A directed line segment, represented by a tensor, is:

[0037] in This represents the total number of lane segments. It represents the two-dimensional coordinates of the start and end points of each directed line segment in each lane segment.

[0038] In addition, a road topology map can be used with a adjacency matrix express:

[0039] in, , j represent the first and second digits respectively. The j-th lane segment. For example, =0 means the first... The j-th lane segment is not connected.

[0040] Step 2: Based on the initial position information of the fully controlled vehicle and the position information of the lane segment, determine the current lane segment where the fully controlled vehicle is located.

[0041] Step 3: Based on the behavior description information included in the scene description information, obtain the basic behavior of the fully controlled vehicle.

[0042] Specifically, a basic behavior database of traffic participants in traffic scenarios is pre-built. The basic behaviors in the database include: going straight, changing lanes, turning, and making a U-turn. Optionally, these can be further defined as going straight, changing lanes, turning left, turning right, making a U-turn left, and making a U-turn right. User-provided behavior descriptions are broken down into behaviors from the basic behavior database. For example, overtaking can be seen as changing lanes to either side, and then changing back to the original lane combination after passing the vehicle in front; while a cut-in can be seen as the vehicle in front changing lanes to the lane where the vehicle behind is located, while the vehicle behind continues straight.

[0043] Step 4: Based on the basic behavior of the fully controlled vehicle and its current lane segment, search for and generate a reference path for the fully controlled vehicle in the road topology map.

[0044] Specifically, starting from the current lane segment where the fully controlled vehicle is located, a complete reference path can be generated by utilizing the connection relationship between lane segments. For example: (1) For straight-going behavior, select the current lane segment and continuously search for its subsequent lane segments to obtain a straight-going reference path; (2) For lane-changing behavior, start from the left / right neighbor of the current lane segment and continuously search for its subsequent lane segments to obtain the entire left / right lane; (3) For turning and U-turn behavior, select the current lane segment and perform a breadth-first search for its successors. By comparing the angles between lane segments, determine and select the turning / U-turn lane segment to form a reference path.

[0045] This embodiment preferably uses depth-first search and breadth-first search, but other search algorithms can also be selected to achieve the same purpose.

[0046] S102: Based on the reference path of the fully controlled vehicle, calculate the cost function corresponding to the basic behavior of the fully controlled vehicle.

[0047] The basic behavior is obtained based on behavior description information, and the cost function is used to control the longitudinal distance traveled by the fully controlled vehicle along the reference path and the lateral distance deviating from the reference path.

[0048] Specifically, S102 includes the following steps: Step 1: Calculate the Frenet coordinates of each projection point based on the position information of multiple projection points of the reference path.

[0049] In this system, the abscissa of the Frenet coordinates represents the longitudinal distance traveled by the fully controlled vehicle along the reference path, while the ordinate of the Frenet coordinates represents the lateral distance of the fully controlled vehicle deviating from the reference path. The projection point is a point on the initial trajectory of the fully controlled vehicle under the guidance of the behavior description information.

[0050] Specifically, for a collection Reference path for each point This can be considered as its origin. It consists of directed line segments, that is

[0051] Optionally, the initial trajectory, measured in units of time, consists of multiple projection points, each of which can be projected onto a different directed line segment. Assume one of the projection points... Its reference path The projection falls on the directed line segment Above. Directed line segment The slope-intercept form of the line is:

[0052] Calculate projection points The projection point on the straight line:

[0053]

[0054] Therefore, the lateral distances can be obtained separately. and longitudinal distance for:

[0055]

[0056] in, For directed line segments The coordinates of the starting point, ( The projection point is... The corresponding Frenet coordinates.

[0057] Step 2: Based on the Frenet coordinates corresponding to each projection point, calculate the first cost function corresponding to the horizontal distance and the second cost function corresponding to the vertical distance.

[0058] Assume an initial trajectory of T seconds. initial trajectory To determine the trajectory for a fully controlled vehicle to complete a U-turn or turn, the initial trajectory can be determined according to the procedure in step 1 above. Convert to Frenet coordinates .

[0059] Specifically, the first cost function for calculating the lateral distance includes: Based on the Frenet coordinates corresponding to each projection point, the value of the ordinate corresponding to each projection point is obtained; based on the absolute value of the ordinate values ​​of each projection point, the sum of the absolute values ​​is calculated to obtain the first cost function corresponding to the horizontal distance.

[0060] First cost function The calculation formula is:

[0061] Calculate the second cost function corresponding to the longitudinal distance, including: Based on the Frenet coordinates corresponding to the target projection point, calculate the longitudinal distance of the target. The target projection point is the start and end point of the initial trajectory within a preset time period. Obtain the x-coordinate value corresponding to the end point of the initial trajectory and calculate the difference between the longitudinal distance of the target and the x-coordinate value. If the difference is greater than a preset threshold, a second cost function corresponding to the vertical distance is obtained based on the difference; if the difference is less than or equal to the preset threshold, a second cost function corresponding to the vertical distance is obtained based on the preset threshold.

[0062] Assuming the preset threshold is 0, the second cost function The calculation formula is:

[0063] in, The target longitudinal distance, i.e., the curve length between the start and end points of the initial trajectory, is used to encourage vehicles to complete turning or U-turn actions. This is the x-coordinate value corresponding to the end point of the initial trajectory. Used to penalize lateral deviations between fully controlled vehicles and the reference path. Used to encourage fully controlled vehicles to move longitudinally along a reference path.

[0064] Step 3: Based on the first cost function and the second cost function, determine the cost function corresponding to the basic behavior of the fully controlled vehicle. For example... Figure 3 As shown, specifically: When the basic behavior of a fully controlled vehicle is to go straight or change lanes, the cost function corresponding to the basic behavior of the fully controlled vehicle is... For the first cost function ; Calculate the second cost function and function coefficients when the basic behavior of a fully controlled vehicle is turning or making a U-turn. The product of the ... The sum of the first cost function and the product, i.e. In this embodiment, it is optional. 2.0.

[0065] S103: Based on the cost function, road structure information, scene description information, initial position information of fully controlled vehicles and initial position information of partially controlled vehicles, the trajectory data of fully controlled vehicles and partially controlled vehicles are generated respectively using a pre-trained diffusion model.

[0066] Specifically, S103 includes the following steps: Step 1: Establish a model of the attention mechanism among road and traffic participants.

[0067] The attention mechanism model is built into the diffusion model. Its role is to model the interactive information in the traffic scene and remove noise from the input data.

[0068] Step 2: Input the cost function, road structure information, scene description information, initial position information of fully controlled vehicles, initial position information of partially controlled vehicles, and noisy trajectory data into the diffusion model. Using the established cost function and attention mechanism model, generate trajectory data of fully controlled vehicles and trajectory data of partially controlled vehicles using the sampling paradigm of the diffusion model.

[0069] Let the constructed attention mechanism model be denoted as , This represents noisy trajectory data. The noisy trajectory data then satisfies the following ordinary differential equation:

[0070] in, For variance, This is the cost function. In this embodiment, it is optional. Any numerical method can be chosen to solve the above ordinary differential equations numerically. The final result... These are the trajectory data of fully controlled vehicles and incompletely controlled vehicles, which are free of noise and constrained by a cost function.

[0071] S104: Based on the trajectory data of fully controlled vehicles, the trajectory data of partially controlled vehicles, and road structure information, obtain a traffic scene that matches the scene description information.

[0072] A complete traffic scenario consists of road structure information and trajectory data of traffic participants. The road structure information can be provided in advance by the user. Therefore, after obtaining the trajectory data of fully controlled vehicles and the trajectory data of partially controlled vehicles, a traffic scenario can be generated.

[0073] like Figure 2 The diagram shown is another flowchart of a traffic scene generation method provided in this disclosure, the method including S201-S206: S201: Obtain the initial conditions of the traffic scenario provided by the user, including road structure information and the initial state of traffic participants, as well as the scenario description information to be generated.

[0074] Traffic participants include fully controlled vehicles and partially controlled vehicles, and the initial position of each traffic participant is represented by a tensor:

[0075] in, Let represent the initial state of the i-th traffic participant. The initial state includes initial location information, representing the total number of traffic participants. and initial orientation , that is

[0076] Road structure information can be expressed as a combination of lane segments, each lane segment including... A directed line segment, represented by a tensor, is:

[0077] in This represents the total number of lane segments. It represents the two-dimensional coordinates of the start and end points of each directed line segment in each lane segment.

[0078] In addition, a road topology map can be used with a adjacency matrix express:

[0079] in, , j represent the first and second digits respectively. The j-th lane segment. For example, =1 means the first... The j-th lane segment is located before the j-th lane segment.

[0080] S202: Based on the basic behavior database of traffic participants in the constructed traffic scenario, the scenario description information provided by the user is decomposed into a combination of basic behaviors.

[0081] The basic behavior library is defined as seven behaviors: going straight, changing lanes left or right, turning left or right, and making U-turns left or right. For example, overtaking in the scene description information can be seen as changing lanes to either side, and then changing back to the original lane after passing the vehicle in front. Cut-in in the scene description information can be seen as the vehicle in front changing lanes to the lane where the vehicle behind is located, while the vehicle behind continues to go straight.

[0082] S203: For each basic action, use a search algorithm to search the road topology map to obtain a reference path for the fully controlled vehicle.

[0083] The reference path includes A point, expressed as , This represents the two-dimensional coordinates of each point. During the search process, many lane segments are traversed; the two-dimensional coordinates of each point on all lane segments are recorded, ultimately forming a vector. .

[0084] Starting from the current lane segment where the fully controlled vehicle is located, the complete reference path of the fully controlled vehicle can be obtained by utilizing the connection relationships between lane segments represented by the road topology map. This embodiment uses depth-first search and breadth-first search, but other search algorithms can be selected to achieve the same goal.

[0085] S204: Construct a cost function based on Frenet coordinates based on the reference path of a fully controlled vehicle.

[0086] The cost function penalizes vehicle trajectories that do not conform to the reference path, guiding the generated trajectories to conform to the given basic behavior. The reference path can be understood as the centerline of the lane, while the trajectory is the actual path traveled by the vehicle, composed of multiple points. The trajectory does not necessarily have to strictly follow the centerline; the trajectory is time-dependent, for example, one point every 0.1 seconds.

[0087] Specifically, S204 includes the following steps 1-2: Step 1: Calculate Frenet coordinates. Frenet coordinates express the horizontal and vertical distances of a point relative to a reference path. For a given path containing... Reference path for each point This can be considered as its origin. It consists of directed line segments, that is

[0088] The initial trajectory, measured in units of time, consists of multiple projection points, each of which can be projected onto a different directed line segment. Assume one of these projection points... Its reference path The projection falls on the directed line segment Above. Directed line segment The slope-intercept form of the line is:

[0089] Calculate projection points The projection point on the straight line:

[0090]

[0091] Therefore, the lateral distances can be obtained separately. and longitudinal distance for:

[0092]

[0093] in, For directed line segments The coordinates of the starting point, ( The projection point is... The corresponding Frenet coordinates.

[0094] Step 2: Construct the cost function .

[0095] For one The trajectory of seconds Convert it to Frenet coordinates Construct cost functions for the horizontal and vertical distances respectively:

[0096]

[0097] in, Used to penalize lateral deviations between fully controlled vehicles and the reference path. Used to encourage fully controlled vehicles to move longitudinally along a reference path. For the target longitudinal distance, Set at the end of a curve / U-turn to encourage vehicles to complete the turn / U-turn.

[0098] like Figure 3 As shown, for straight-ahead and lane-changing behaviors, For turning and U-turn actions, ,in, Take 2.0.

[0099] S205: Construct an attention mechanism model.

[0100] An attention mechanism model among road and traffic participants is established. This model is trained based on the initial conditions of the traffic scenario given by the user and is embedded in the diffusion model.

[0101] The role of the attention mechanism model is to model the interaction information in traffic scenarios and remove noise from the input data. The input of the attention mechanism model is noisy trajectory data and the initial conditions of the traffic scenario.

[0102] The working principle of the attention mechanism model is as follows: (1) Each trajectory data and each lane segment are encoded into a feature vector using a multilayer perceptron. The features obtained after encoding the trajectory and lane segments are denoted as follows: as well as , where d is the dimension of the model's latent variables.

[0103] (2) Use attention mechanisms to model road topology interactions, interactions between traffic participants and roads, and interactions among traffic participants. Taking the interaction between traffic participants and road topology as an example, the following operations are performed:

[0104]

[0105] Where Q, K, and V are the query, key, and value in the attention mechanism model, respectively. Each of these represents a linear layer, which performs a linear transformation on the input, with d being a latent variable. Modeling is performed sequentially for three types of interactions: road topology interaction, interaction between traffic participants and the road, and interaction among traffic participants. During the update process, each feature vector incorporates information from other feature vectors, resulting in each feature vector containing rich interaction information from the traffic scenario.

[0106] (3) Apply the multilayer perceptron to the updated The noise-removed trajectory is obtained from the above.

[0107] in, =MLP( ), output This is the denoised trajectory data. The whole process starts with the noisy trajectory, goes through multiple layers of attention mechanism modeling between the road and the trajectory, and finally goes through a multi-layer perceptron to output the denoised trajectory.

[0108] S206: Using the established cost function and attention mechanism model, traffic scenarios that conform to the user-given scenario description information are generated using the diffusion model sampling paradigm.

[0109] This paradigm utilizes synchronous optimization of the cost function during the sampling step to guide the sampling process to satisfy the conditions expressed by the cost function. By using S202 to decompose the complex target scene into basic behaviors and the cost function constructed in S204, the diffusion model can be better guided to sample and generate traffic scenes that conform to the description.

[0110] Let the attention mechanism model constructed by S205 be denoted as , This represents noisy trajectory data. The noisy trajectory data then satisfies the following ordinary differential equation:

[0111] in, For variance, This is the cost function. In this embodiment, it is optional. Any numerical method can be chosen to solve the above ordinary differential equations numerically. The final result... These are the trajectory data of fully controlled vehicles and incompletely controlled vehicles, which are free of noise and constrained by a cost function.

[0112] To verify that this embodiment can generate more complex scenes and improve the success rate, experiments were conducted under three conditions: lane change, left turn, and right turn. The baseline method selected was CTG++, a cost function method based on projection points. The results of the comparison with the baseline method are shown in Table 1.

[0113] Table 1 Results of Condition Generation As can be seen from Table 1, the generation method in this embodiment significantly improves the success rate and can better generate conditions for complex scenes.

[0114] Furthermore, to demonstrate the application value of the traffic scene generated in this embodiment for deep learning models in autonomous driving, the generated data was used as augmented data to train the trajectory prediction model. Specifically, 20,000 data points generated in this embodiment were used to train the model together with the data used to train the initial trajectory prediction model. The average position deviation (ADE) and final position deviation (FDE) were used as evaluation criteria, and the results are shown in Tables 2 and 3:

[0115] Table 2. Experimental Results of Trajectory Prediction

[0116] Table 3 Comparison of trajectory prediction results under turning and lane changing categories As can be seen from Tables 2 and 3, the performance of the trajectory prediction model has been improved to a certain extent after training with the generated augmented data, especially for lane changing and turning behaviors that are difficult to learn, which shows a significant improvement, proving that the generated traffic scenarios have practical value.

[0117] According to another aspect of the embodiments of this disclosure, a traffic scene generation apparatus is provided, such as... Figure 4 As shown, the device includes: The path generation module 401 is used to generate a reference path for the fully controlled vehicle based on preset road structure information, scene description information and initial position information of the fully controlled vehicle, wherein the scene description information includes the behavior description information of the fully controlled vehicle. The calculation module 402 is used to calculate a cost function corresponding to the basic behavior of the fully controlled vehicle based on the reference path of the fully controlled vehicle. The basic behavior is obtained based on the behavior description information. The cost function is used to control the longitudinal distance traveled by the fully controlled vehicle along the reference path and the lateral distance deviating from the reference path. The trajectory generation module 403 is used to generate trajectory data of the fully controlled vehicle and trajectory data of the incompletely controlled vehicle based on the cost function, the road structure information, the scene description information, the initial position information of the fully controlled vehicle and the initial position information of the incompletely controlled vehicle, using a pre-trained diffusion model. The scene generation module 404 is used to obtain a traffic scene that conforms to the scene description information based on the trajectory data of the fully controlled vehicle, the trajectory data of the partially controlled vehicle, and the road structure information.

[0118] In one or more embodiments, the computing module 402 is used for: Based on the position information of multiple projection points of the reference path, the Frenet coordinates corresponding to each projection point are calculated; the projection point is a point on the initial trajectory of the fully controlled vehicle traveling under the instruction of the behavior description information; the abscissa of the Frenet coordinate is used to characterize the longitudinal distance traveled by the fully controlled vehicle along the reference path; the ordinate of the Frenet coordinate is used to characterize the lateral distance of the fully controlled vehicle deviating from the reference path. Based on the Frenet coordinates corresponding to each of the projection points, calculate the first cost function corresponding to the horizontal distance and the second cost function corresponding to the vertical distance; Based on the first cost function and the second cost function, the cost function corresponding to the basic behavior of the fully controlled vehicle is determined.

[0119] In one or more embodiments, the calculation module 402 is further configured to: obtain the value of the ordinate corresponding to each projection point based on the Frenet coordinates corresponding to each projection point; calculate the sum of the absolute values ​​of the ordinate values ​​of each projection point to obtain a first cost function corresponding to the horizontal distance; and calculate the target vertical distance based on the Frenet coordinates corresponding to the target projection point, wherein the target projection point is the start and end point of the initial trajectory within a preset time period; obtain the value of the abscissa corresponding to the end point of the initial trajectory; calculate the difference between the target vertical distance and the abscissa value; if the difference is greater than a preset threshold, obtain a second cost function corresponding to the vertical distance based on the difference; and if the difference is less than or equal to a preset threshold, obtain the second cost function corresponding to the vertical distance based on the preset threshold.

[0120] In one or more embodiments, the calculation module 402 is further configured to: when the basic behavior of the fully controlled vehicle is to go straight or change lanes, the cost function corresponding to the basic behavior of the fully controlled vehicle is the first cost function; When the basic behavior of the fully controlled vehicle is turning or making a U-turn, the product of the second cost function and the function coefficients is calculated, and the cost function corresponding to the basic behavior of the fully controlled vehicle is the sum of the first cost function and the product.

[0121] In one or more embodiments, the path generation module 401 is used to: Based on preset road structure information, a road topology map is obtained. The road structure information includes the location information of lane segments, and the road topology map is used to represent the connection relationship between the lane segments. Based on the initial position information of the fully controlled vehicle and the position information of the lane segment, the current lane segment where the fully controlled vehicle is located is determined; Based on the behavioral description information included in the scenario description information, the basic behavior of the fully controlled vehicle is obtained; Based on the basic behavior of the fully controlled vehicle and its current lane segment, a reference path for the fully controlled vehicle is searched and generated in the road topology map.

[0122] In one or more embodiments, the path generation module 401 is further configured to: decompose the behavior description information included in the scene description information based on a pre-built basic behavior library to obtain the basic behavior of the fully controlled vehicle; The basic actions include at least one of going straight, changing lanes, turning, and making a U-turn.

[0123] The traffic scene generation device and the traffic scene generation method provided in this disclosure are based on the same inventive concept and have the same beneficial effects as the methods they adopt, operate or implement.

[0124] This disclosure also provides a computer device for executing the traffic scene generation method described above. Please refer to... Figure 5 It illustrates a schematic diagram of a computer device provided by some embodiments of this disclosure. For example... Figure 5 As shown, the computer device 50 includes: a processor 500, a memory 501, a bus 502, and a communication interface 503. The processor 500, the communication interface 503, and the memory 501 are connected via the bus 502. The memory 501 stores a computer program that can run on the processor 500. When the processor 500 runs the computer program, it executes the traffic scene generation method provided in any of the foregoing embodiments of this disclosure.

[0125] The memory 501 may include high-speed random access memory (RAM) or non-volatile memory, such as at least one disk storage device. Communication between this device network element and at least one other network element is achieved through at least one communication interface 503 (which can be wired or wireless), such as the Internet, wide area network, local area network, metropolitan area network, etc.

[0126] Bus 502 can be an ISA bus, PCI bus, or EISA bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. The memory 501 is used to store programs. After receiving an execution instruction, the processor 500 executes the program. The traffic scene generation method disclosed in any of the foregoing embodiments of this disclosure can be applied to the processor 500, or implemented by the processor 500.

[0127] The processor 500 may be an integrated circuit chip with signal processing capabilities. In implementation, each step of the above method can be completed by the integrated logic circuitry in the hardware of the processor 500 or by instructions in software form. The processor 500 may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), an off-the-shelf programmable gate array (FPTA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this disclosure. The general-purpose processor may be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this disclosure can be directly embodied in the execution of a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software modules may reside in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. The storage medium is located in memory 501. The processor 500 reads the information in memory 501 and, in conjunction with its hardware, completes the steps of the above method.

[0128] The computer device provided in this disclosure and the traffic scene generation method provided in this disclosure are based on the same inventive concept and have the same beneficial effects as the methods they adopt, operate or implement.

[0129] This disclosure also provides a computer-readable storage medium corresponding to the traffic scene generation method provided in the foregoing embodiments. The computer-readable storage medium is an optical disc, on which a computer program (i.e., a computer program product) is stored. When the computer program is run by a processor, it executes the traffic scene generation method provided in any of the foregoing embodiments.

[0130] It should be noted that examples of the computer-readable storage medium may also include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other optical and magnetic storage media, which will not be elaborated here.

[0131] The computer-readable storage medium provided in the above embodiments of this disclosure and the traffic scene generation method provided in the embodiments of this disclosure are based on the same inventive concept and have the same beneficial effects as the methods adopted, run or implemented by the applications stored therein.

[0132] This disclosure also provides a computer program product; please refer to [reference needed]. Figure 6 The computer program product 600 carries program code, namely computer program 601. The instructions included in the computer program 601 can be used to execute the steps of the traffic scene generation method described in the above method embodiments. For details, please refer to the above method embodiments, which will not be repeated here.

[0133] The aforementioned computer program product can be implemented through hardware, software, or a combination thereof. In one optional embodiment, the computer program product is specifically embodied in a computer storage medium; in another optional embodiment, the computer program product is specifically embodied in a software product, such as a software development kit (SDK), etc.

[0134] The basic principles of this disclosure have been described above with reference to specific embodiments. However, it should be noted that the advantages, benefits, and effects mentioned in this disclosure are merely examples and not limitations, and should not be considered as essential features of each embodiment of this disclosure. Furthermore, the specific details disclosed above are for illustrative and facilitative purposes only, and are not limitations. These details do not limit the scope of this disclosure to the necessity of employing the aforementioned specific details for implementation.

[0135] The block diagrams of devices, apparatuses, devices, and systems disclosed herein are merely illustrative examples and are not intended to require or imply that they must be connected, arranged, or configured in the manner shown in the block diagrams. As those skilled in the art will recognize, these devices, apparatuses, devices, and systems can be connected, arranged, and configured in any manner. Words such as “comprising,” “including,” “having,” etc., are open-ended terms meaning “including but not limited to,” and are used interchangeably with them. The terms “or” and “and” as used herein refer to the terms “and / or,” and are used interchangeably with them unless the context clearly indicates otherwise. The term “such as” as used herein refers to the phrase “such as but not limited to,” and is used interchangeably with it.

[0136] Additionally, as used herein, the "or" used in a list of items beginning with "at least one" indicates a separate list, such that a list of, for example, "at least one of A, B, or C" means A or B or C, or AB or AC or BC, or ABC (i.e., A and B and C). Furthermore, the word "exemplary" does not imply that the described example is preferred or better than other examples.

[0137] It should also be noted that in the systems and methods of this disclosure, the components or steps can be decomposed and / or recombined. These decompositions and / or recombinations should be considered as equivalent solutions to this disclosure.

[0138] Various changes, substitutions, and modifications can be made to the technology described herein without departing from the teachings defined by the appended claims. Furthermore, the scope of the claims of this disclosure is not limited to the specific aspects of the processes, machines, manufactures, events, means, methods, and actions described above. Currently existing or later-developed processes, machines, manufactures, events, means, methods, or actions that perform substantially the same function or achieve substantially the same result as the corresponding aspects described herein can be utilized. Therefore, the appended claims include such processes, machines, manufactures, events, means, methods, or actions within their scope.

[0139] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use this disclosure. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein can be applied to other aspects without departing from the scope of this disclosure. Therefore, this disclosure is not intended to be limited to the aspects shown herein, but rather to be carried out within the widest scope consistent with the principles and novel features disclosed herein.

[0140] The above description has been given for purposes of illustration and description. Furthermore, this description is not intended to limit the embodiments of this disclosure to the forms disclosed herein. Although numerous exemplary aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, alterations, additions, and sub-combinations thereof.

Claims

1. A traffic scenario generation method, characterized by, The method comprises the following steps: generating a reference path of a fully controlled vehicle based on preset road structure information, scene description information and initial position information of the fully controlled vehicle, wherein the scene description information comprises behavior description information of the fully controlled vehicle; calculating a cost function corresponding to a basic behavior of the fully controlled vehicle based on the reference path of the fully controlled vehicle, wherein the basic behavior is obtained based on the behavior description information, and the cost function is used to control a longitudinal distance of the fully controlled vehicle along the reference path and a transverse distance of the fully controlled vehicle deviating from the reference path; generating trajectory data of the fully controlled vehicle and trajectory data of an incompletely controlled vehicle respectively by using a pre-trained diffusion model based on the cost function, the road structure information, the scene description information, the initial position information of the fully controlled vehicle and initial position information of the incompletely controlled vehicle; obtaining a traffic scene conforming to the scene description information based on the trajectory data of the fully controlled vehicle, the trajectory data of the incompletely controlled vehicle and the road structure information.

2. The traffic scenario generation method of claim 1, wherein, The method for calculating the cost function corresponding to the basic behavior of the fully controlled vehicle based on the reference path of the fully controlled vehicle comprises the following steps: calculating Frenet coordinates corresponding to each projection point based on position information of a plurality of projection points of the reference path; the projection points are points on an initial trajectory of the fully controlled vehicle traveling under the indication of the behavior description information, and the transverse coordinate of the Frenet coordinates is used to represent the longitudinal distance of the fully controlled vehicle along the reference path, and the longitudinal coordinate of the Frenet coordinates is used to represent the transverse distance of the fully controlled vehicle deviating from the reference path; calculating a first cost function corresponding to the transverse distance and a second cost function corresponding to the longitudinal distance based on the Frenet coordinates corresponding to each projection point; confirming the cost function corresponding to the basic behavior of the fully controlled vehicle based on the first cost function and the second cost function.

3. The traffic scenario generation method of claim 2, wherein, The method for calculating the first cost function corresponding to the transverse distance and the second cost function corresponding to the longitudinal distance based on the Frenet coordinates corresponding to each projection point comprises the following steps: obtaining the value of the longitudinal coordinate corresponding to each projection point based on the Frenet coordinates corresponding to each projection point, calculating the sum of absolute values based on the absolute values of the values of the longitudinal coordinates of each projection point, obtaining the first cost function corresponding to the transverse distance; and calculating a target longitudinal distance based on the Frenet coordinates corresponding to a target projection point, wherein the target projection point is the starting point and the ending point of the initial trajectory in a preset time period; obtaining the value of the transverse coordinate corresponding to the ending point of the initial trajectory, calculating the difference between the target longitudinal distance and the value of the transverse coordinate; in the case that the difference is greater than a preset threshold, obtaining the second cost function corresponding to the longitudinal distance based on the difference; in the case that the difference is less than or equal to the preset threshold, obtaining the second cost function corresponding to the longitudinal distance based on the preset threshold.

4. The traffic scenario generation method of claim 2, wherein, Confirming a cost function corresponding to the basic behavior of the fully controlled vehicle based on the first cost function and the second cost function, comprising: In the case that the basic behavior of the fully controlled vehicle is straight driving or lane changing, the cost function corresponding to the basic behavior of the fully controlled vehicle is the first cost function; In the case that the basic behavior of the fully controlled vehicle is turning or U-turning, calculating a product of the second cost function and a function coefficient, and the cost function corresponding to the basic behavior of the fully controlled vehicle is a sum of the first cost function and the product.

5. The traffic scenario generation method of claim 1, wherein, Generating a reference path of the fully controlled vehicle based on preset road structure information, scene description information and initial position information of the fully controlled vehicle, comprising: Obtaining a road topology graph based on the preset road structure information, wherein the road structure information comprises position information of lane segments, and the road topology graph is used to represent connection relationships between the lane segments; Determining a current lane segment where the fully controlled vehicle is located based on the initial position information of the fully controlled vehicle and the position information of the lane segments; Obtaining a basic behavior of the fully controlled vehicle based on behavior description information included in the scene description information; Searching and generating the reference path of the fully controlled vehicle in the road topology graph based on the basic behavior of the fully controlled vehicle and the current lane segment where the fully controlled vehicle is located.

6. The traffic scenario generation method of claim 5, wherein, Obtaining a basic behavior of the fully controlled vehicle based on behavior description information included in the scene description information, comprising: Decomposing the behavior description information included in the scene description information based on a pre-constructed basic behavior library to obtain the basic behavior of the fully controlled vehicle; The basic behavior comprises at least one of straight driving, lane changing, turning and U-turning.

7. A traffic scenario generation apparatus characterized by comprising: Comprising: A path generation module configured to generate a reference path of the fully controlled vehicle based on preset road structure information, scene description information and initial position information of the fully controlled vehicle, wherein the scene description information comprises behavior description information of the fully controlled vehicle; A calculation module configured to calculate a cost function corresponding to a basic behavior of the fully controlled vehicle based on the reference path of the fully controlled vehicle, wherein the basic behavior is obtained based on the behavior description information, and the cost function is used to control a longitudinal distance of the fully controlled vehicle along the reference path and a transverse distance of the fully controlled vehicle deviating from the reference path; A trajectory generation module configured to generate trajectory data of the fully controlled vehicle and trajectory data of the incompletely controlled vehicle respectively by using a pre-trained diffusion model based on the cost function, the road structure information, the scene description information, the initial position information of the fully controlled vehicle and initial position information of the incompletely controlled vehicle; A scene generation module configured to obtain a traffic scene conforming to the scene description information based on the trajectory data of the fully controlled vehicle, the trajectory data of the incompletely controlled vehicle and the road structure information.

8. A computer-embedded device comprising a memory, a processor and a computer program stored on the memory, characterized in that, The processor executes the computer program to implement the method of any one of claims 1-6.

9. A computer readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the method of any one of claims 1-6.

10. A computer program product comprising a computer program, characterized in that, The computer program, which is executed by a processor, implements the method of any one of claims 1 to 6.

Images