Data generation method and apparatus

Abstract

A data generation method and device, the method comprising: obtaining motion object control information, the motion object control information at least indicating a first motion attribute of a first motion object and a second motion attribute of a virtual vehicle, the first motion object being an object having a first interaction relationship with the virtual vehicle; generating at least one set of trajectory points according to the motion object control information, each set of trajectory points in the at least one set of trajectory points comprising a plurality of pairs of trajectory points, each pair of trajectory points in the plurality of pairs of trajectory points respectively indicating positions of the first motion object and the virtual vehicle at a time instant; and generating a first set of images according to a first set of trajectory points in the at least one set of trajectory points, the first set of images being used for training a first algorithm model, the first algorithm model being used for planning a driving path for the first vehicle. The technical solution can be applied to the field of neural network model training, and helps to improve the ability of the vehicle to cope with long-tail problems in the automatic driving process, thereby improving the driving safety of the vehicle.

Description

Technical Field

[0001] This application relates to the field of neural network model training, and more specifically, to a data generation method and apparatus. Background Technology

[0002] With the development of vehicle intelligence, more and more intelligent driving technologies have been developed to alleviate the stress on drivers during driving. The reliability of the aforementioned intelligent driving technologies generally depends on the reliability of the relevant intelligent driving algorithms. In order to improve the reliability of intelligent driving algorithms, they are usually trained using massive amounts of scenario data encountered in real-world driving.

[0003] However, the aforementioned massive amount of scenario data includes a small number of driving scenarios with extremely low probability of occurrence, which makes the vehicle's intelligent driving algorithm insufficient in dealing with such scenarios, thus affecting the vehicle's driving safety.

[0004] Therefore, a data generation solution that can improve vehicles' ability to cope with long-tail scenarios is urgently needed. Summary of the Invention

[0005] This application provides a data generation method and apparatus that can generate driving scenario data with low probability of occurrence in actual driving scenarios, so as to train the intelligent driving algorithm model of the vehicle based on such scenario data, which helps to improve the vehicle's ability to deal with long-tail problems in the process of autonomous driving, thereby improving the vehicle's driving safety.

[0006] Firstly, a data generation method is provided that can be applied in a computer, for example, it can be executed by a chip or circuit used in a computer.

[0007] The method includes: acquiring moving object control information, the moving object control information indicating at least a first motion attribute of a first moving object and a second motion attribute of a virtual vehicle, the first moving object being an object having a first interaction relationship with the virtual vehicle; generating at least one set of trajectory points based on the moving object control information, each set of trajectory points including multiple pairs of trajectory points, each pair of trajectory points indicating the position of the first moving object and the virtual vehicle at a certain moment.

[0008] Generally, scenarios associated with data that have a low probability of occurrence in real-world driving are often quite dangerous, potentially causing vehicle damage or even threatening the safety of occupants. Therefore, acquiring such low-probability data is costly. The aforementioned technical solution generates driving scenario data with a low probability of occurrence in real-world driving based on moving object control information, addressing the issue of high costs associated with acquiring real-world data. Furthermore, training the vehicle's intelligent driving algorithm model using this generated data helps improve the model's generalization ability to rare scenarios, thereby enhancing the vehicle's ability to handle low-probability events during autonomous driving and ultimately improving driving safety.

[0009] In conjunction with the first aspect, in some implementations of the first aspect, the method further includes: acquiring static element information, the static element information at least indicating the road boundary of the first road; generating at least one set of trajectory points based on moving object control information, including: generating at least one set of trajectory points based on moving object control information and static element information; each pair of trajectory points also respectively indicates the position of the first moving object and the virtual vehicle in the first road at a certain moment.

[0010] In some implementations, static feature information also indicates at least one of the following: the location and type (e.g., color and line type) of lane lines in the first road, the intersection boundaries associated with the first road, the topological relationship between the first road and other roads, or the location of static obstacles in the road.

[0011] In the above technical solution, the boundaries of the vehicle's drivable area are restricted based on the static elements of the road, thereby improving the accessibility of the generated trajectory points and the compatibility between the trajectory points and road constraints.

[0012] In conjunction with the first aspect, in some implementations of the first aspect, the method further includes: performing a first processing on the first motion attribute to obtain N sets of motion parameters, the N sets of motion parameters indicating N sets of trajectory points of the first moving object; performing a first processing on the second motion attribute to obtain M sets of motion parameters, the M sets of motion parameters indicating M sets of trajectory points of the virtual vehicle; obtaining P sets of trajectory points based on the N sets of motion trajectories and the M sets of motion trajectories, the P sets of trajectory points including at least one set of trajectory points; wherein M and N are both positive integers, and P is the product of N and M.

[0013] For example, the first processing can be augmentation processing, such as initializing the motion object control information based on different random noises to obtain multiple sets of processed motion object control information.

[0014] In the above technical solution, multiple sets of trajectory points can be generated based on a set of moving object control information, thereby covering different driving styles and interaction strategies and avoiding the limitations of single decision-making. Furthermore, training the vehicle based on the generated multiple sets of trajectory points enables the vehicle to cope with more complex dynamic scenarios and helps to select the optimal driving path for the virtual vehicle to deal with the first moving object.

[0015] In conjunction with the first aspect, in some implementations of the first aspect, the N sets of motion parameters and the M sets of motion parameters are the parameters obtained after removing invalid motion parameters.

[0016] Among them, illegal motion parameters include, but are not limited to: parameters that do not conform to the laws of vehicle kinematics and dynamics, parameters that are incompatible with the static elements of the road (such as trajectory points exceeding the road boundary), parameters that do not meet safety requirements, or parameters that cause the vehicle's behavior to differ too much from the characteristics of operation under human driver control.

[0017] In the above technical solutions, safe, human-like, and reasonable driving data are selected so that the intelligent driving algorithm model trained based on such driving data is safer and its decision-making and behavior are more human-like.

[0018] In conjunction with the first aspect, in some implementations of the first aspect, the moving object control information includes spoken text, which indicates the motion characteristics of the first moving object and the virtual vehicle; the first motion attribute and the second motion attribute are determined based on the spoken text.

[0019] In the above technical solution, determining the control information of moving objects based on language text can improve the flexibility of the input control information of moving objects, reduce the complexity of constructing the control information of moving objects, and thus reduce the workload in the process of generating driving scene data.

[0020] In conjunction with the first aspect, in some implementations of the first aspect, the motion object control information indicates the starting and ending positions of the first moving object and the virtual vehicle, and the position, velocity, or orientation angle of the first moving object and the virtual vehicle at intermediate moments.

[0021] In some implementations, the moving object control information also indicates the speed and / or orientation angle of the first moving object and the virtual vehicle at the starting and ending positions, respectively.

[0022] The moving object control information can indicate the starting and ending positions of the first moving object and the virtual vehicle directly or indirectly (or implicitly). For example, when the moving object control information is in the form of an array, it can indirectly indicate the possible starting and ending positions of the first moving object by indicating the coordinates of a certain position that the first moving object needs to pass through from the starting position to the ending position and the vehicle's orientation angle at that position.

[0023] In some implementations, the motion attributes of the first moving object and the virtual vehicle indicated by the moving object control information are matched with the first interaction relationship.

[0024] In conjunction with the first aspect, in some implementations of the first aspect, the moving object control information also indicates the type of the first moving object and / or the virtual vehicle.

[0025] The type of the first moving object can include any of the following: motor vehicle, non-motor vehicle, or pedestrian. If the first moving object is a motor vehicle, the type of the first moving object can also indicate its vehicle type (e.g., bus, truck, or motorcycle) and / or size. Furthermore, the type of the virtual vehicle can also indicate its vehicle type and / or size.

[0026] In the above technical solution, by indicating the type of the first moving object and / or the type of the virtual vehicle through the moving object control information, more driving scenarios can be covered, so that the generalization ability of the intelligent driving algorithm model trained based on such driving data is stronger.

[0027] In conjunction with the first aspect, in some implementations of the first aspect, the moving object control information also indicates the interaction between the first moving object and / or the virtual vehicle and the static road elements in the first road.

[0028] The interaction between the first moving object and / or virtual vehicle and the static road elements includes, but is not limited to: the first moving object and / or virtual vehicle crossing the solid lane line, driving against the flow of traffic, or not driving in the designated lane.

[0029] In the above technical solution, by using moving object control information to indicate the interaction between the first moving object and / or virtual vehicle and static road elements, more driving scenarios can be covered, so that the generalization ability of the intelligent driving algorithm model trained based on such driving data is stronger.

[0030] In conjunction with the first aspect, in some implementations of the first aspect, the first interaction relationship includes any of the following: the first moving object cuts into front of the virtual vehicle; the first moving object is located in front of the virtual vehicle, and the virtual vehicle cuts out of the lane where the first moving object is located; or, the first moving object and the virtual vehicle are in the same lane, and there is a risk of collision between the virtual vehicle and the first moving object.

[0031] In conjunction with the first aspect, in some implementations of the first aspect, the method further includes: generating a first set of images based on a first set of trajectory points, the first set of images being used to train a first algorithm model, the first algorithm model being used to plan a driving path for a first vehicle; wherein at least one set of trajectory points includes the first set of trajectory points, the first set of images includes multiple frames of images, each frame of the multiple frames of images indicating the position of the first moving object and the virtual vehicle in the first road at a certain moment, and the first vehicle and the virtual vehicle being associated.

[0032] In some implementations, the first algorithm model is an end-to-end neural network model.

[0033] Generally speaking, intelligent driving algorithm models for vehicles are based on image and / or point cloud data for path planning and vehicle control. In the above technical solution, generating relevant images based on the first set of trajectory points helps to improve the ability of vehicles with this intelligent driving algorithm model to respond to driving scenarios indicated by relevant images.

[0034] In conjunction with the first aspect, in some implementations of the first aspect, the method further includes: acquiring environmental information, which indicates the weather conditions and / or lighting conditions of the first road; and generating a first set of images based on a first set of trajectory points, including: generating the first set of images based on the first set of trajectory points and the environmental information.

[0035] In the above technical solution, the generated images take into account the effects of different weather conditions (such as rain, snow, fog and haze) and lighting conditions (such as backlighting and nighttime), which helps the intelligent driving algorithm model trained by such data to plan more reasonable and compliant driving paths for vehicles under different weather and lighting conditions.

[0036] Secondly, a data generation apparatus is provided, comprising an acquisition unit and a processing unit, wherein the acquisition unit is configured to: acquire moving object control information, the moving object control information indicating at least a first motion attribute of a first moving object and a second motion attribute of a virtual vehicle, the first moving object being an object having a first interaction relationship with the virtual vehicle; the processing unit is configured to: generate at least one set of trajectory points based on the moving object control information, each set of trajectory points including multiple pairs of trajectory points, each pair of trajectory points indicating the positions of the first moving object and the virtual vehicle at a given moment.

[0037] In conjunction with the second aspect, in some implementations of the second aspect, the acquisition unit is further configured to: acquire static element information, the static element information at least indicating the road boundary of the first road; the processing unit is configured to: generate at least one set of trajectory points based on the moving object control information and the static element information; each pair of trajectory points further indicates the position of the first moving object and the virtual vehicle in the first road at a certain moment.

[0038] In conjunction with the second aspect, in some implementations of the second aspect, the processing unit is further configured to: perform a first processing on the first motion attribute to obtain N sets of motion parameters, the N sets of motion parameters indicating N sets of trajectory points of the first moving object; perform a first processing on the second motion attribute to obtain M sets of motion parameters, the M sets of motion parameters indicating M sets of trajectory points of the virtual vehicle; and obtain P sets of trajectory points based on the N sets of motion trajectories and the M sets of motion trajectories, the P sets of trajectory points including at least one set of trajectory points; wherein M and N are both positive integers, and P is the product of N and M.

[0039] In conjunction with the second aspect, in some implementations of the second aspect, the N sets of motion parameters and the M sets of motion parameters are the parameters obtained after removing invalid motion parameters.

[0040] In conjunction with the second aspect, in some implementations of the second aspect, the moving object control information includes spoken text, which indicates the motion characteristics of the first moving object and the virtual vehicle; the first motion attribute and the second motion attribute are determined based on the spoken text.

[0041] In conjunction with the second aspect, in some implementations of the second aspect, the moving object control information indicates the starting and ending positions of the first moving object and the virtual vehicle, and the first moving object and the virtual vehicle at intermediate moments as at least one of the following: position, speed, or, orientation angle.

[0042] In conjunction with the second aspect, in some implementations of the second aspect, the moving object control information also indicates the type of the first moving object and / or the virtual vehicle.

[0043] In conjunction with the second aspect, in some implementations of the second aspect, the moving object control information also indicates the interaction relationship between the first moving object and / or virtual vehicle and static road elements in the first road.

[0044] In conjunction with the second aspect, in some implementations of the second aspect, the first interaction relationship includes any of the following: the first moving object cuts into front of the virtual vehicle; the first moving object is located in front of the virtual vehicle, and the virtual vehicle cuts out of the lane where the first moving object is located; or, the first moving object and the virtual vehicle are in the same lane, and there is a risk of collision between the virtual vehicle and the first moving object.

[0045] In conjunction with the second aspect, in some implementations of the second aspect, the processing unit is further configured to: generate a first set of images based on the first set of trajectory points, the first set of images being used to train a first algorithm model, the first algorithm model being used to plan a driving path for a first vehicle; wherein at least one set of trajectory points includes the first set of trajectory points, the first set of images includes multiple frames of images, each frame of the multiple frames of images indicating the position of the first moving object and the virtual vehicle in the first road at a certain moment, and the first vehicle and the virtual vehicle being associated.

[0046] In conjunction with the second aspect, in some implementations of the second aspect, the acquisition unit is further configured to: acquire environmental information, the environmental information indicating the weather conditions and / or lighting conditions of the first road; the processing unit is further configured to: generate a first set of images based on the first set of trajectory points and the environmental information.

[0047] Thirdly, a data generation apparatus is provided, the apparatus comprising: a processor for executing a computer program stored in the memory, such that the apparatus performs the method in any possible implementation of the first aspect described above.

[0048] In conjunction with the third aspect, in some implementations of the third aspect, the device also includes a memory.

[0049] Fourthly, a computer program product is provided, comprising: computer program code, which, when executed on a computer or processor, causes the computer or processor to perform the method in any possible implementation of the first aspect.

[0050] It should be noted that the above computer program code can be stored in whole or in part on a storage medium, which can be packaged together with the processor or packaged separately from the processor.

[0051] Fifthly, a computer-readable storage medium is provided, the computer-readable medium storing instructions that, when executed by a processor, cause the processor to implement the method in any possible implementation of the first aspect.

[0052] In a sixth aspect, a chip is provided, the chip including circuitry for performing the method in any of the possible implementations of the first aspect described above.

[0053] For the beneficial effects not described in detail in aspects two through six, please refer to the description in aspect one, which will not be repeated here. Attached Figure Description

[0054] Figure 1 This is a schematic block diagram of the data generation system provided in the embodiments of this application;

[0055] Figure 2This is a schematic diagram of the structure of the motion object behavior generation model provided in the embodiments of this application;

[0056] Figure 3 This is another schematic diagram of the data generation system provided in the embodiments of this application;

[0057] Figure 4 This is a schematic diagram illustrating the application scenario provided in the embodiments of this application;

[0058] Figure 5 This is yet another schematic diagram illustrating an application scenario provided in the embodiments of this application;

[0059] Figure 6 This is yet another schematic diagram illustrating the application scenario provided in the embodiments of this application;

[0060] Figure 7 This is a schematic flowchart of the data generation method provided in the embodiments of this application;

[0061] Figure 8 This is a schematic block diagram of the data generation apparatus provided in the embodiments of this application;

[0062] Figure 9 This is another schematic block diagram of the data generation apparatus provided in the embodiments of this application. Detailed Implementation

[0063] To facilitate understanding of the technical solution of this application, some concepts involved in this application are introduced below:

[0064] (1) Long-tail scenarios (or long-tail problems): In the field of autonomous driving, long-tail problems refer to edge cases encountered by intelligent vehicles during operation, i.e., scenarios with low probability of occurrence. These low-probability scenarios are often overlooked in the datasets used to train algorithm models for intelligent driving because of their low occurrence rate and special nature. For example, long-tail scenarios include, but are not limited to: other vehicles with protruding trucks or irregularly shaped vehicles, other vehicles cutting in front of the vehicle, the risk of rear-end collision between other vehicles and the vehicle itself, or obstacles in front of the vehicle after it cuts out of its lane.

[0065] (2) Neural Network (or Neural Network Model): A neural network can be composed of neural units. A neural unit can refer to a computational unit that takes xs and an intercept of 1 as inputs, and the output of the computational unit can be:

[0066]

[0067] Where s = 1, 2, ..., n, n is a natural number greater than 1, W s For x sThe weights are denoted by b, where b is the bias of the neural unit. f is the activation function of the neural unit, used to introduce nonlinear characteristics into the neural network to convert the input signal in the neural unit into an output signal. The output signal of this activation function can be used as the input of the next convolutional layer; the activation function can be the sigmoid function. A neural network is a network formed by connecting multiple of the above-mentioned individual neural units together; that is, the output of one neural unit can be the input of another neural unit. The input of each neural unit can be connected to the local receptive field of the previous layer to extract the features of the local receptive field, which can be a region composed of several neural units.

[0068] (3) Convolutional Neural Network (CNN): A deep neural network with convolutional structures. A CNN contains a feature extractor consisting of convolutional layers and subsampling layers, which can be viewed as a filter. A convolutional layer refers to the layer of neurons in a CNN that performs convolutional processing on the input signal. In a convolutional layer of a CNN, a neuron can be connected to only some of the neurons in its neighboring layers. A convolutional layer typically contains several feature planes, each composed of rectangularly arranged neural units. Neural units on the same feature plane share weights, which are the convolutional kernel. Shared weights can be understood as the way image information is extracted regardless of location. The convolutional kernel can be initialized as a matrix of random size, and during the training process of the CNN, the kernel can learn reasonable weights. Furthermore, the direct benefit of shared weights is reducing the connections between layers in the CNN, while also reducing the risk of overfitting.

[0069] (4) Intelligent driving algorithm model: This is a neural network model that processes the perception information obtained by the vehicle's cameras, lidar, and other sensors to obtain the processing results, so as to plan the driving path for the vehicle. Intelligent driving algorithm models can include end-to-end models, large-scale autonomous driving models, etc.

[0070] As mentioned earlier, a large number of real-world driving scenarios contain relatively few driving scenario data points with low probability of occurrence, which limits the vehicle's ability to handle long-tail scenarios in intelligent driving mode. Therefore, this application provides a data generation method capable of generating driving scenario data with low probability of occurrence in real-world driving scenarios. This data is then used to train the vehicle's intelligent driving algorithm model, thereby improving the vehicle's ability to handle long-tail problems.

[0071] The technical solutions in this application will now be described with reference to the accompanying drawings.

[0072] Figure 1A schematic block diagram of a data generation system provided in an embodiment of this application is shown. The system includes a moving object behavior generation model and an image rendering model. The moving object behavior generation model generates trajectory points of a moving object based on moving object control conditions, which indicate the motion parameters of the moving object. In some implementations, the moving object behavior generation model can also combine static element conditions to generate trajectory points of the moving object, wherein the static element conditions at least indicate the road boundary of the road where the moving object is located. Furthermore, inputting the trajectory points of the moving object into the image rendering model yields an image indicating the changes in the moving object's position within the road.

[0073] More specifically, the structure of the moving object behavior generation model is as follows: Figure 2 As shown, it includes an input layer, a scene encoder, a trajectory generation model, a fine-tuning module, and an output layer. The roles of each part in the moving object behavior generation model are described in items (I) to (V):

[0074] (i) Input Layer: Receives the behavior conditions of moving objects and performs feature extraction and preprocessing based on these conditions, providing structured input to downstream modules (such as scene encoders). When the information input to the moving object behavior generation model also includes static feature conditions, the input layer can also perform feature extraction and preprocessing on the static feature conditions.

[0075] In some implementations, the behavior conditions of moving objects can be information in the form of an array, where the values ​​indicate the position of each of at least one moving object at different times. For example, a multidimensional array can be used to represent the behavior conditions of moving objects. Taking a 3-dimensional array n*m*p as an example, each element in n indicates each of the n moving objects, each element in m indicates a moment in the time series, and each element in p indicates the motion parameter characteristics (such as position, velocity, orientation angle, etc.) of each moving object at each moment. Each element in p can indicate a single attribute of the moving object (such as position), or it can indicate a composite attribute of the moving object (such as indicating position, velocity, and orientation angle).

[0076] In some other implementations, the behavioral conditions of a moving object can also be information in text form, for example, such as... Figure 3 As shown, the large language model (LLM) can obtain the behavioral conditions of moving objects in text form, process them to embed the text information into a high-dimensional vector space, and input the processed text information into the input layer.

[0077] For example, static feature conditions can be map information, which can be a vectorized map indicating road vectors, lane vectors, intersection vectors, etc. In some implementations, static feature conditions can also be text information, which is processed by LLM and then input into the input layer for feature extraction and preprocessing.

[0078] (ii) Scene encoder: used to generate a vector representation of the scene in which the moving object is located.

[0079] In some implementations, when the input information to the moving object behavior generation model only includes moving object behavior conditions, the feature representation of the scene can be obtained based on the content indicated by the moving object behavior conditions. For example, when the moving object behavior conditions indicate that two moving objects with an interactive relationship are in the same lane, a vector representation of the boundary of a lane can be generated based on the positions of the two moving objects. This lane is used for the two moving objects to travel, meaning that the scene can contain at least one lane in this case. As another example, when the moving object behavior conditions indicate that one moving object cuts in front of another moving object, vector representations of the boundaries of two lanes can be generated based on the positions of the two moving objects. Each lane is used for the two moving objects to travel, meaning that the scene can contain at least two lanes in this case.

[0080] In this context, the interaction between two moving objects can be understood as one moving object influencing the path or state (such as speed, orientation angle, etc.) of the other moving object.

[0081] In some other implementations, when the information input to the moving object behavior generation model also includes static feature conditions, the scene encoder can parse static features such as road topology, lane boundaries, lane line types (such as color and / or line type), and obstacle positions from the static feature conditions to generate a vector representation of the scene.

[0082] (iii) Trajectory Generation Model: Used to generate at least one set of trajectory points for a moving object. For example, the trajectory generation model encodes the starting coordinates and ending coordinates of the moving object, or the driving intention (such as lane change direction), into a conditional vector. Through a cross-attention mechanism, it interacts with the intermediate features of the trajectory generation process to generate a trajectory from the starting coordinates to the ending coordinates that meets the target constraints. The trajectory includes multiple trajectory points, each indicating the position coordinates of the moving object at a certain moment.

[0083] For example, the aforementioned target constraints may include, but are not limited to: the coordinates of the intermediate position of the moving object between the starting point and the ending point, the orientation angle and velocity of the moving object at the starting point and / or the ending point, or, the vehicle must not exceed the boundary of the drivable area determined based on static feature information during driving.

[0084] In some implementations, the trajectory generation model can be a CNN-based neural network model, or it can be a Transformer-based neural network model. By using the Transformer architecture to fuse information such as static road elements, the trajectory can be accurately aligned with the scene topology.

[0085] In some implementations, the trajectory generation model initializes the control conditions of the moving object input into the model with different random noises (such as Gaussian noise), so that the trajectory generation model can generate multiple candidate trajectories based on the control conditions of the moving object initialized with random noise. For example, in a lane-changing scenario, noise differences can trigger trajectory branches with different strategies such as aggressive entry, conservative following, and delayed lane changing.

[0086] In some implementations, the trajectory generation model can be a diffusion model, or it can be any other model.

[0087] (iv) Fine-tuning module: After the trajectory generation model generates candidate trajectories, the fine-tuning module performs trajectory fine-tuning and screening based on the reward function to improve the reliability of decision-making.

[0088] The reward function can define the physical rules and other constraints that the trajectory must satisfy. For example, physical rules may include restrictions on safe following distance, lateral acceleration, and longitudinal acceleration. The lateral and longitudinal directions can be determined relative to the direction of travel of the moving object or the vehicle coordinate system of the moving object, respectively. For instance, the lateral direction is perpendicular to the direction of travel of the moving object, and the longitudinal direction is parallel to the direction of travel of the moving object; or, for example, when the moving object is a vehicle, the lateral direction can be parallel to the Y-axis of the vehicle coordinate system, and the longitudinal direction can be parallel to the X-axis of the vehicle coordinate system. It should be noted that the origin O of the vehicle coordinate system can be located at the projection point of the rear axle center of the vehicle onto the ground, and the positive directions of the X and Z axes can be the direction of the vehicle's front end and the direction perpendicular to the vehicle's plane, respectively.

[0089] In some implementations, the aforementioned physical rules and semantic constraints (such as traffic sign priority) can be combined to quantify trajectory quality, thereby guiding the training direction of the fine-tuning module and the trajectory generation model.

[0090] (v) Output layer: Based on the trajectory points output by the trajectory generation model, the trajectory points can be processed such as trajectory generation, scene completion, and path planning, and the processed trajectory can be output.

[0091] The results of the neural network model on which the data generation method provided in the embodiments of this application depends have been described in detail above. The training process of the neural network model involved in this application is described below.

[0092] In some implementations, vehicle trajectory data is collected and represented as a time series (pose-time pairs). A noise scheduling strategy suitable for sequence data is designed. After generating the original trajectory x0, Gaussian noise is progressively added to the original trajectory x0 to generate a noisy trajectory x. t The noise trajectory x t Satisfies a Gaussian distribution:

[0093]

[0094] in, The mean of the distribution; It is the covariance matrix; These are noise scheduling parameters β1, β2, ..., β t The product function, β i These are noise scheduling parameters, typically set according to linear or cosine rules.

[0095] Furthermore, the trained model predicts the noise ε added during the forward process and calculates the loss L based on the loss function shown in the following formula:

[0096] L=||ε-ε θ (x t ,t)|| 2 ;

[0097] Where ε is the actual Gaussian noise added to the data during the forward diffusion process (sampled from a standard normal distribution), ε θ (x t (t) is the trajectory generation model (parameter θ) based on the current noisy data x. t Given a time step t, the predicted noise estimate. ||·|| 2 It is the square of the L2 norm, which is used to calculate the mean square error between the predicted noise and the actual noise.

[0098] After determining the loss L, the gradient of the loss L with respect to each parameter in the trajectory generation model is calculated through backpropagation, and the parameter θ in the trajectory generation model is updated based on the gradient descent algorithm.

[0099] In some implementations, when the behavior conditions of moving objects can also be in text form, pre-training can be performed using text prompts and static scene conditions to align the static scene conditions to the high-dimensional vector space embedded with the semantic information of the text. During this alignment pre-training process, the weight parameters of models such as the input layer, scene encoder, and trajectory generation model can be frozen.

[0100] It should be noted that the aforementioned training methods for neural network models are merely illustrative examples. In actual implementation, other methods can also be used to train the neural network models. Figure 2 and Figure 3 The model shown is trained.

[0101] The data generation system provided in this application has been described in detail above. The following section, in conjunction with... Figures 4 to 7 This section details the process of generating relevant data based on the aforementioned system.

[0102] Figure 4 This illustration shows a schematic flowchart of a data generation method provided in an embodiment of this application. The method can be generated by… Figure 1 The system shown can be executed, or the method can also be performed by... Figure 2 or Figure 3 The model shown is executed, and method 300 includes:

[0103] S301, Obtain moving object control information, the moving object control information at least indicates the first motion attribute of the first moving object and the second motion attribute of the virtual vehicle, the first moving object being an object that has a first interaction relationship with the virtual vehicle.

[0104] For example, the moving object control information may include the moving object control conditions in the foregoing embodiments.

[0105] In some implementations, the first interaction relationship includes any of the following: the first moving object cuts into front of the virtual vehicle; the first moving object is in front of the virtual vehicle, and the virtual vehicle cuts out of the lane where the first moving object is located; or, the first moving object and the virtual vehicle are in the same lane, and there is a risk of collision between the virtual vehicle and the first moving object. It is understood that this first interaction relationship can be indicated by moving object control information.

[0106] For example, when the virtual vehicle cuts out of the lane where the first moving object is located, the first moving object may be located in front of the virtual vehicle.

[0107] In some implementations, the motion object control information can be in the form of an array. In this case, the motion object control information can indicate the starting and ending positions of the first moving object and the virtual vehicle, as well as at least one of the following at intermediate moments: position, velocity, acceleration, angular velocity, angular acceleration, or orientation angle. In actual implementation, the motion object control information can also indicate other lateral motion parameters and / or longitudinal motion parameters of the first moving object and / or the virtual vehicle.

[0108] In some other implementations, the motion control information can be in text form. In this case, the motion control information includes spoken text that indicates the motion characteristics of the first moving object and the virtual vehicle; the first motion attribute and the second motion attribute are determined based on the spoken text.

[0109] For example, if the first interaction relationship is that the first moving object cuts into the front of the virtual vehicle from the left side, and the first moving object is a vehicle, then the language text can be "A vehicle cuts into the front of the virtual vehicle from the left side"; or, if the virtual vehicle cuts out of the lane where the first moving object is located to the right, then the language text can be "The vehicle cuts out to the right".

[0110] S302, based on the control information of the moving object, generate at least one set of trajectory points, each set of trajectory points including multiple pairs of trajectory points, each pair of trajectory points indicating the position of the first moving object and the virtual vehicle at a certain moment.

[0111] More specifically, S302 can be refined to: based on the control information of the moving object, based on Figure 2 or Figure 3 The moving object behavior generation model shown generates at least one set of trajectory points.

[0112] For example, taking the first interaction relationship as the first moving object cutting into front of the virtual vehicle as an example, Figure 5 The object 1 shown can be regarded as an example of a virtual vehicle, and the object 2 can be regarded as an example of a first moving object. If the moving object control information is in the form of an array, the moving object control information can indicate the starting positions of the first moving object and the virtual vehicle as position a' and position a, respectively, and the moving object control information can indicate the ending positions of the first moving object and the virtual vehicle as position c' and position c, respectively. Figure 5 The positions b' and b shown can be indicated by the control information of the moving object, or they can be indicated by the initial trajectory generated by the behavior generation model of the moving object.

[0113] In some implementations, the method further includes: performing a first processing on a first motion attribute to obtain N sets of motion parameters, the N sets of motion parameters indicating N sets of trajectory points of the first moving object; performing a first processing on a second motion attribute to obtain M sets of motion parameters, the M sets of motion parameters indicating M sets of trajectory points of the virtual vehicle; obtaining P sets of trajectory points based on the N sets of motion trajectories and the M sets of motion trajectories, the P sets of trajectory points including at least one set of trajectory points; wherein M and N are both positive integers, and P is the product of N and M.

[0114] For example, the first process can be noise initialization processing of the control information of the moving object based on different random noise, such as initialization of the first motion attribute and / or the second motion attribute based on different random noise.

[0115] For example, Figure 6 The paths a to d shown can be obtained by performing a first processing on the first motion attribute. Furthermore, the paths obtained after removing unreasonable paths from paths a to d can be regarded as some examples of the trajectory points of the first moving object included in the aforementioned P groups of trajectory points.

[0116] In some implementations, the N sets of motion parameters and the M sets of motion parameters are the parameters obtained after removing invalid motion parameters.

[0117] For example, illegal motion parameters include, but are not limited to: parameters that violate vehicle kinematics and dynamics, parameters that are incompatible with static elements of the road (such as trajectory points exceeding road boundaries), parameters that do not meet safety requirements, or parameters that make the vehicle behave inhumanly.

[0118] In some implementations, the method further includes: acquiring static element information, which at least indicates the road boundary of the first road; generating at least one set of trajectory points based on moving object control information, including: generating at least one set of trajectory points based on moving object control information and static element information; each pair of trajectory points also indicates the position of the first moving object and the virtual vehicle in the first road at a certain moment.

[0119] For example, the static element information may include the static element conditions in the foregoing embodiments. In addition to indicating the road boundary, the static element information may also indicate at least one of the following: the position and type (such as color and line type) of the lane lines in the first road, the intersection boundary associated with the first road, the topological relationship between the first road and other roads, or the position of static obstacles in the road.

[0120] In some implementations, the moving object control information also indicates the type of the first moving object and / or virtual vehicle.

[0121] For example, the type of the first moving object may include any of the following: motor vehicle, non-motor vehicle, or pedestrian. In the case that the first moving object is a motor vehicle, the type of the first moving object may also indicate the vehicle type (e.g., bus, truck, or motorcycle) and / or size of the first moving object.

[0122] For example, the type of virtual vehicle can indicate the vehicle type (such as bus, truck, etc.) and / or size of the virtual vehicle.

[0123] The size of the first moving object and the virtual vehicle can be indicated by the h parameter and the w parameter, where the value of the h parameter indicates the longitudinal dimension of the first moving object and the virtual vehicle, and the w parameter indicates the lateral dimension of the first moving object and the virtual vehicle.

[0124] In some implementations, the moving object control information also indicates the interaction between the first moving object and / or virtual vehicle and static road elements in the first road.

[0125] For example, the interaction between the first moving object and / or virtual vehicle and the static road element includes, but is not limited to: the first moving object and / or virtual vehicle crossing the solid lane line, driving against the flow of traffic, or not driving in the designated lane.

[0126] For example, if the first interaction relationship is that a first moving object cuts into front of a virtual vehicle from the left side of the virtual vehicle, the first moving object is a vehicle, and the first moving object crosses the solid lane line during the process of cutting into front of the virtual vehicle, then the language text can be "A vehicle crosses the solid line and cuts into front of my vehicle from the left side".

[0127] In the aforementioned implementation methods, the example of the interaction relationship between the first moving object and the virtual vehicle indicated by the moving object control information is used for illustration. In actual implementation, during the vehicle's driving process, there may be more than one moving object that interacts with the vehicle at the same time. If more complex driving scenario data needs to be generated, the moving object control information can also indicate the motion parameters of other moving objects that interact with the vehicle.

[0128] In one example, if it is necessary to generate driving scene data in which there is a vehicle on each side of the vehicle cutting in front of the vehicle, the motion object control information can indicate the motion parameters of the vehicle, the motion parameters of the vehicle on the left side of the vehicle, and the motion parameters of the vehicle on the right side of the vehicle.

[0129] In another example, consider generating driving scenario data where the vehicle changes lanes and another slow-moving vehicle blocks its path in the target lane. Figure 7Taking the following example, if object 1 is the vehicle itself, object 2 is another vehicle in front of the vehicle in the lane it was in before it cut out, and object 3 is another vehicle in the target lane that is obstructing the vehicle, then the motion object control information can indicate the starting positions of object 1, object 2 and object 3 as position 1, position 1' and position 1' respectively, and the motion object control information can indicate the ending positions of object 1, object 2 and object 3 as position 3, position 3' and position 3 respectively. Figure 7 The positions 2, 2', and 2" in the text can be indicators of the control information of the moving object, or they can be indicators of the initial trajectory generated by the behavior generation model of the moving object.

[0130] It is understandable that when the control information for a moving object is in the form of an array, it can indicate... Figure 7 The detailed positions of each object are shown; when the control information for the moving object is in the form of spoken text, it is described... Figure 7 The verbal text describing the behavior of each object shown could be: "The vehicle cuts to the left due to a slow-moving vehicle blocking its path. After cutting out, there is a slow-moving vehicle in the target lane that is obstructing the vehicle's movement."

[0131] Furthermore, in the aforementioned embodiments, the motion parameters of the virtual vehicle and the moving objects that interact with it are indicated by the motion object control information. In actual implementation, the motion object control information can also indicate the motion parameters of the moving objects that do not interact with the virtual vehicle. For example, the motion object control information can indicate the motion parameters of the moving objects that do not interact with the virtual vehicle through rules and strategies such as log replay or intelligent driver model (IDM).

[0132] In some implementations, a first set of images is generated based on a first set of trajectory points. The first set of images is used to train a first algorithm model, which is used to plan a driving path for a first vehicle. At least one set of trajectory points includes the first set of trajectory points, and the first set of images includes multiple frames. Each frame in the multiple frames indicates the position of the first moving object and the virtual vehicle in the first road at a certain moment, and the first vehicle and the virtual vehicle are associated.

[0133] It should be noted that the first vehicle is a physical vehicle. The association between the first vehicle and the virtual vehicle can be understood as follows: the position and position changes of the virtual vehicle can indicate the position and position changes of the first vehicle; or, the driving path of the virtual vehicle in response to the first moving object can be used to train the behavior of the first vehicle when using the first moving object.

[0134] In some implementations, each frame can also indicate the position of the first moving object relative to the virtual vehicle, that is, the position of the first moving object in the first road from the perspective of the virtual vehicle. Furthermore, since the first vehicle and the virtual vehicle are associated, the image showing the position of the first moving object in the first road from the perspective of the virtual vehicle can be used to simulate the situation where the first vehicle encounters the first moving object while driving.

[0135] In some implementations, the first vehicle is a vehicle in a broad sense, such as transportation vehicles (e.g., commercial vehicles, passenger cars, motorcycles, flying cars, trains, etc.), industrial vehicles (e.g., forklifts, trailers, tractors, etc.), engineering vehicles (e.g., excavators, bulldozers, cranes, etc.), agricultural equipment (e.g., lawnmowers, harvesters, etc.), amusement equipment, toy vehicles, etc. In actual implementation, the vehicle can also be a road vehicle, a water vehicle, an air vehicle, industrial equipment, agricultural equipment, or other intelligent driving equipment such as entertainment equipment.

[0136] For example, each frame of image can be generated based on a pair of trajectory points from a first set of trajectory points.

[0137] In some implementations, it is possible to... Figure 1 The image rendering model generates the first set of images. In one example, the image rendering model may include a background generation model and an object generation model. The background generation model is used to generate a point cloud structure describing a static scene of the first road based on static feature information. Specifically, a point cloud structure is used to describe static elements such as roads and buildings. Each point in the point cloud structure can be optimized to generate corresponding 3D Gaussian parameters (covariance matrix, position vector) to form a Gaussian sphere. Furthermore, for each point (or Gaussian sphere), transparency, spherical harmonic coefficients (for lighting modeling), and semantic information for object classification can be set. The object generation model is used to model moving objects and virtual vehicles, which are represented by point clouds in a local coordinate system and aligned with the global coordinate system through optimizable tracking poses (position, rotation). Additionally, a 4D spherical harmonic model (i.e., encoded via Fourier transform in the time dimension) can be used to capture changes in the appearance of virtual vehicles and moving objects over time (such as light reflection and motion blur). Furthermore, based on the positions of the first moving object and the virtual vehicle indicated by each pair of trajectory points in the first set of trajectory points, the positions of the Gaussian spheres indicating the moving object and the virtual vehicle are adjusted. After determining the positions of the moving object and the virtual vehicle in the first road at different times, the moving object and the virtual vehicle are rendered based on three-dimensional (3D) Gaussian sputtering to obtain image information.

[0138] In another example, the image rendering model could also be a diffusion-based image rendering model, or other models used for rendering images.

[0139] In some implementations, environmental information is acquired, which indicates the weather conditions and / or lighting conditions of the first road; a first set of images is generated based on a first set of trajectory points, including: generating the first set of images based on the first set of trajectory points and the environmental information.

[0140] The data generation method provided in this application can generate driving scenario data with a low probability of occurrence in actual driving scenarios based on the control information of moving objects, thus solving the problem of scarce real data. Furthermore, training the vehicle's intelligent driving algorithm model based on the aforementioned generated data helps improve the model's generalization ability to rare scenarios, thereby enhancing the vehicle's ability to cope with low-probability events during autonomous driving and ultimately improving vehicle driving safety.

[0141] In the various embodiments of this application, unless otherwise specified or in case of logical conflict, the terminology and / or descriptions between the various embodiments are consistent and can be referenced by each other. Technical features in different embodiments can be combined to form new embodiments according to their inherent logical relationships.

[0142] The above text combines Figures 1 to 7 The methods provided in the embodiments of this application are described in detail below. Figure 8 and Figure 9 The apparatus provided in the embodiments of this application is described in detail. It should be understood that the description of the apparatus embodiments corresponds to the description of the method embodiments. Therefore, for content not described in detail, please refer to the method embodiments above. For the sake of brevity, it will not be repeated here.

[0143] Figure 8 A schematic block diagram of a data generation apparatus 2000 provided in an embodiment of this application is shown. The apparatus 2000 may include units for executing the embodiments described in the foregoing method. Furthermore, each unit in the apparatus 2000 implements a corresponding process of the above-described method embodiments. The apparatus 2000 includes an acquisition unit 2010, which can be used to implement corresponding data acquisition or transmission / reception functions. The apparatus 2000 also includes a processing unit 2020, which can be used to implement corresponding processing functions.

[0144] Optionally, the device 2000 further includes a storage unit, which can be used to store instructions and / or data. The processing unit 2020 can read the instructions and / or data in the storage unit so that the device can perform the relevant actions in the aforementioned method embodiments.

[0145] It should be understood that the specific process of each unit performing the above-mentioned corresponding steps has been described in detail in the above method embodiments, and will not be repeated here for the sake of brevity.

[0146] It should also be understood that the device 2000 described herein is embodied in the form of a functional unit. The terms “module” or “unit” may refer to application-specific ASICs, electronic circuits, processors (e.g., shared processors, proprietary processors, or group processors) and memory for executing one or more software or firmware programs, integrated logic circuits, and / or other suitable components that support the described functions.

[0147] The apparatuses described above are capable of implementing the corresponding steps performed by the computing platform 150 in the methods described above. These functions can be implemented in hardware or by hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the functions described above; for example, the acquisition unit 2010 can be replaced by a transceiver, and other units, such as processing units, can be replaced by a processor, used to execute the relevant processing operations in each method embodiment.

[0148] For example, the acquisition unit 2010 and the processing unit 2020 can be set in Figure 1 In the vehicle 100 shown, or it can also be set in Figure 2 In the system shown, for example, the operations performed by the acquisition unit 2010 and the processing unit 2020 can be executed by a single processor, or by different processors. In specific implementations, the one or more processors can be configured as follows: Figure 1 The processor in the vehicle 100 shown; or, the device 2000 described above may be a chip disposed in the vehicle 100.

[0149] In the specific implementation process, the units in the above device can be fully or partially integrated together, or they can be implemented independently. In one implementation, these units are integrated together and implemented in the form of a system-on-a-chip (SoC).

[0150] Figure 9 This is another schematic block diagram of the data generation apparatus provided in the embodiments of this application. Figure 9 The illustrated device 2100 may include a processor 2110, a transceiver 2120, and a memory 2130. The processor 2110, transceiver 2120, and memory 2130 are connected via internal interconnects. The memory 2130 stores instructions, and the processor 2110 executes the instructions stored in the memory 2130 to implement the methods described in the above embodiments. Optionally, the memory 2130 may be coupled to the processor 2110 via an interface or integrated with the processor 2110.

[0151] It should be noted that the transceiver 2120 mentioned above may include, but is not limited to, transceiver devices such as input / output interfaces, to realize communication between device 2100 and other devices or communication networks.

[0152] Memory 2130 can be volatile memory and / or non-volatile memory. Non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory can be random access memory (RAM). For example, RAM can be used as an external cache. By way of example and not limitation, RAM includes various forms such as: static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous linked dynamic random access memory (SLDRAM), and direct rambus RAM (DR RAM).

[0153] Transceiver 2120 uses transceiver devices, such as but not limited to transceivers, to enable communication between device 2100 and other devices or communication networks to receive / send data / information for implementing the methods in the above embodiments.

[0154] This application also provides a computer program product, which includes computer program code. When the computer program code is run on a computer, it causes the computer to implement the methods described in the above embodiments of this application.

[0155] This application also provides a computer-readable storage medium storing computer instructions that, when executed on a computer, cause the computer to implement the methods described in the above embodiments of this application.

[0156] This application also provides a chip, including circuitry, for performing the methods described in the above embodiments of this application.

[0157] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0158] In the description of the embodiments of this application, unless otherwise stated, " / " means "or", for example, A / B can mean A or B; "and / or" in this document describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. In this application, "at least one" means one or more, and "more" means two or more. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of single or multiple items. For example, at least one of a, b, or c can represent: a, b, c, ab, ac, bc, or abc, where a, b, and c can be single or multiple.

[0159] The use of prefixes such as "first" and "second" in this application embodiment is solely for distinguishing different descriptive objects and does not limit the position, order, priority, quantity, or content of the described objects. The use of ordinal numbers and other prefixes to distinguish descriptive objects in this application embodiment does not constitute a limitation on the described objects. The description of the described objects is found in the claims or the context of the embodiments, and the use of such prefixes should not constitute unnecessary restrictions.

[0160] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0161] In the various embodiments of this application, unless otherwise specified or in case of logical conflict, the terminology and / or descriptions between the various embodiments are consistent and can be referenced by each other. Technical features in different embodiments can be combined to form new embodiments according to their inherent logical relationships.

[0162] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0163] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0164] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A data generation method, characterized in that, include: Acquire moving object control information, which at least indicates a first motion attribute of a first moving object and a second motion attribute of a virtual vehicle, wherein the first moving object is an object that has a first interaction relationship with the virtual vehicle; Based on the moving object control information, at least one set of trajectory points is generated. Each set of trajectory points includes multiple pairs of trajectory points, and each pair of trajectory points indicates the position of the first moving object and the virtual vehicle at a certain moment.

2. The method according to claim 1, characterized in that, The method further includes: Obtain static element information, wherein the static element information at least indicates the road boundary of the first road; Based on the moving object control information, at least one set of trajectory points is generated, including: Based on the moving object control information and the static element information, the at least one set of trajectory points is generated; Each pair of trajectory points also indicates the position of the first moving object and the virtual vehicle in the first road at a given moment.

3. The method according to claim 1 or 2, characterized in that, The method further includes: The first motion attribute is processed to obtain N sets of motion parameters, and the N sets of motion parameters indicate N sets of trajectory points of the first moving object; The second motion attribute is processed by the first method to obtain M sets of motion parameters, wherein the M sets of motion parameters indicate the M sets of trajectory points of the virtual vehicle; Based on the N sets of motion trajectories and the M sets of motion trajectories, P sets of trajectory points are obtained, wherein the P sets of trajectory points include the at least one set of trajectory points; Where M and N are both positive integers, and P is the product of N and M.

4. The method according to claim 3, characterized in that, The N sets of motion parameters and the M sets of motion parameters are obtained after removing invalid motion parameters.

5. The method according to any one of claims 1 to 4, characterized in that, The moving object control information includes language text, which indicates the motion characteristics of the first moving object and the virtual vehicle. The first motion attribute and the second motion attribute are determined based on the language text.

6. The method according to any one of claims 1 to 4, characterized in that, The moving object control information indicates the starting and ending positions of the first moving object and the virtual vehicle, and the position of the first moving object and the virtual vehicle at an intermediate moment, including at least one of the following: position, speed, or orientation angle.

7. The method according to any one of claims 1 to 6, characterized in that, The moving object control information also indicates the type of the first moving object and / or the virtual vehicle.

8. The method according to any one of claims 1 to 7, characterized in that, The moving object control information also indicates the interaction between the first moving object and / or the virtual vehicle and static road elements in the first road.

9. The method according to any one of claims 1 to 8, characterized in that, The first interaction relationship includes any one of the following: The first moving object cuts into the front of the virtual vehicle; The first moving object is located in front of the virtual vehicle, and the virtual vehicle cuts out of the lane where the first moving object is located; or, The first moving object and the virtual vehicle are in the same lane, and there is a risk of collision between the virtual vehicle and the first moving object.

10. The method according to any one of claims 1 to 9, characterized in that, The method further includes: Based on the first set of trajectory points, a first set of images is generated. The first set of images is used to train a first algorithm model. The first algorithm model is used to plan a driving path for the first vehicle. Wherein, the at least one set of trajectory points includes the first set of trajectory points, the first set of images includes multiple frames of images, each frame of the multiple frames of images indicates the position of the first moving object and the virtual vehicle in the first road at a certain moment, and the first vehicle and the virtual vehicle are associated.

11. The method according to claim 10, characterized in that, The method further includes: Obtain environmental information, which indicates the weather conditions and / or lighting conditions of the first road; The step of generating a first set of images based on a first set of trajectory points includes: The first set of images is generated based on the first set of trajectory points and the environmental information.

12. A data generation apparatus, characterized in that, include: The acquisition unit is used to acquire moving object control information, which at least indicates a first motion attribute of a first moving object and a second motion attribute of a virtual vehicle, wherein the first moving object is an object that has a first interaction relationship with the virtual vehicle. The processing unit is configured to generate at least one set of trajectory points based on the control information of the moving object, wherein each set of trajectory points includes multiple pairs of trajectory points, and each pair of trajectory points indicates the position of the first moving object and the virtual vehicle at a certain moment.

13. The apparatus according to claim 12, characterized in that, The acquisition unit is also used for: Obtain static element information, wherein the static element information at least indicates the road boundary of the first road; The processing unit is used for: Based on the moving object control information and the static element information, the at least one set of trajectory points is generated; Each pair of trajectory points also indicates the position of the first moving object and the virtual vehicle in the first road at a given moment.

14. The apparatus according to claim 12 or 13, characterized in that, The processing unit is also used for: The first motion attribute is processed to obtain N sets of motion parameters, and the N sets of motion parameters indicate N sets of trajectory points of the first moving object; The second motion attribute is processed by the first method to obtain M sets of motion parameters, wherein the M sets of motion parameters indicate the M sets of trajectory points of the virtual vehicle; Based on the N sets of motion trajectories and the M sets of motion trajectories, P sets of trajectory points are obtained, wherein the P sets of trajectory points include the at least one set of trajectory points; Where M and N are both positive integers, and P is the product of N and M.

15. The apparatus according to claim 14, characterized in that, The N sets of motion parameters and the M sets of motion parameters are obtained after removing invalid motion parameters.

16. The apparatus according to any one of claims 12 to 15, characterized in that, The moving object control information includes language text, which indicates the motion characteristics of the first moving object and the virtual vehicle. The first motion attribute and the second motion attribute are determined based on the language text.

17. The apparatus according to any one of claims 12 to 16, characterized in that, The moving object control information indicates the starting and ending positions of the first moving object and the virtual vehicle, and the position of the first moving object and the virtual vehicle at an intermediate moment, including at least one of the following: position, speed, or orientation angle.

18. The apparatus according to any one of claims 12 to 17, characterized in that, The moving object control information also indicates the type of the first moving object and / or the virtual vehicle.

19. The apparatus according to any one of claims 12 to 18, characterized in that, The moving object control information also indicates the interaction between the first moving object and / or the virtual vehicle and the static road elements in the first road.

20. The apparatus according to any one of claims 12 to 19, characterized in that, The first interaction relationship includes any one of the following: The first moving object cuts into the front of the virtual vehicle; The first moving object is located in front of the virtual vehicle, and the virtual vehicle cuts out of the lane where the first moving object is located; or, The first moving object and the virtual vehicle are in the same lane, and there is a risk of collision between the virtual vehicle and the first moving object.

21. The apparatus according to any one of claims 12 to 20, characterized in that, The processing unit is also used for: Based on the first set of trajectory points, a first set of images is generated. The first set of images is used to train a first algorithm model. The first algorithm model is used to plan a driving path for the first vehicle. Wherein, the at least one set of trajectory points includes the first set of trajectory points, the first set of images includes multiple frames of images, each frame of the multiple frames of images indicates the position of the first moving object and the virtual vehicle in the first road at a certain moment, and the first vehicle and the virtual vehicle are associated.

22. The apparatus according to claim 21, characterized in that, The acquisition unit is also used for: Obtain environmental information, which indicates the weather conditions and / or lighting conditions of the first road; The processing unit is used for: The first set of images is generated based on the first set of trajectory points and the environmental information.

23. A data generation apparatus, characterized in that, include: A processor for executing a computer program stored in memory to cause the apparatus to perform the method as described in any one of claims 1 to 11.

24. A computer-readable storage medium, characterized in that, It stores instructions that, when executed by a processor, implement the method as described in any one of claims 1 to 11.

25. A chip, characterized in that, The chip includes circuitry for performing the method as described in any one of claims 1 to 11.

26. A computer program product, characterized in that, The computer program product includes: computer program code, which, when executed by a processor, implements the method as described in any one of claims 1 to 11.

Images