Automatic driving scene generation method and related model training method and device

Abstract

The application discloses a method for generating an automatic driving scene and a training method and device for a related model, the training method comprising: obtaining a training data set for describing an automatic driving scene, the automatic driving scene comprising static map elements and dynamic objects, the training data set comprising a first data subset representing the static map elements and a second data subset representing original trajectories of the dynamic objects; inputting the first data subset and the second data subset into an automatic driving scene generation model to perform training according to a target function, to obtain a trained automatic driving scene generation model, the target function being a function comprising conditional variables and modeling variables, the conditional variables representing the static map elements, and the modeling variables representing modeling trajectories of the dynamic objects. The above scheme can improve the accuracy of the trained automatic driving scene generation model during model training.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to an autonomous driving scene generation method and a training method and device for related models. Background Technology

[0002] Deep generative models are an important research direction in artificial intelligence based on neural networks. Unlike discriminative models such as regression and classification, generative models can model a series of observational data and then randomly generate new observational data, especially given certain implicit parameters. In recent years, deep generative models have been widely used in fields such as image synthesis, speech synthesis, text generation, and data augmentation. In the field of autonomous driving simulation, the commonly used framework is based on a domain-specific language (DSL) to describe the scene, parameterizing the important characteristics of the scene, and then constructing a large number of new scenes through parameter transformation and permutation. However, this approach is highly dependent on the scene description; if the scene description is not accurate enough, the constructed new scene will be inaccurate. Summary of the Invention

[0003] This application provides at least one method for generating autonomous driving scenes and a training method and device for related models, in order to solve the problem of inaccurate scene generation in related technologies.

[0004] The first aspect of this application provides a method for training an autonomous driving scene generation model, including:

[0005] A training dataset for describing an autonomous driving scenario is obtained, wherein the autonomous driving scenario includes static map elements and dynamic objects, and the training dataset includes a first data subset representing the static map elements and a second data subset representing the original trajectories of the dynamic objects; the first data subset and the second data subset are input into the autonomous driving scenario generation model to train it according to an objective function to obtain a trained autonomous driving scenario generation model, wherein the objective function is a function containing condition variables and modeling variables, the condition variables representing the static map elements and the modeling variables representing the modeled trajectories of the dynamic objects.

[0006] The autonomous driving scene change model includes an encoder and a decoder. Inputting the first data subset and the second data subset into the autonomous driving scene generation model includes: inputting the first data subset and the second data subset into the encoder respectively to obtain a first encoded dataset corresponding to the static map element and a second encoded dataset corresponding to the original trajectory of the dynamic object; wherein, in the objective function, the condition variable represents the first encoded dataset corresponding to the static map element, and the modeling variable represents the second encoded dataset corresponding to the original trajectory of the dynamic object; inputting the first encoded dataset and the second encoded dataset into the decoder to obtain the modeled trajectory corresponding to the dynamic object; training according to the objective function to obtain the autonomous driving scene generation model includes: when the objective function is minimized, obtaining the network parameters of the autonomous driving scene generation model to obtain the autonomous driving scene generation model.

[0007] The encoder includes at least two convolutional layers and a max-pooling layer. The at least two convolutional layers are connected sequentially, and the last convolutional layer is connected to the max-pooling layer. The output dimensions of the at least two convolutional layers increase sequentially. The max-pooling layer sets the dimension to 1 when the sequence length output by the last convolutional layer is greater than 1. The encoder inputs the first data subset and the second data subset to obtain a first encoded dataset corresponding to the static map element and a second encoded dataset corresponding to the original trajectory of the dynamic object. This includes performing convolution operations on the first data subset and the second data subset through the at least two convolutional layers and the max-pooling layer to obtain the first encoded dataset and the second encoded dataset. Both the first encoded dataset and the second encoded dataset are two-dimensional arrays with the same sequence length.

[0008] The decoder includes a multi-head attention mechanism and a linear residual decoder. Inputting the first encoded dataset and the second encoded dataset into the decoder to obtain the modeling trajectory corresponding to the dynamic object includes: processing the first encoded dataset and the second encoded dataset through the multi-head attention mechanism to obtain the attention vector of the dynamic object, wherein the attention vector represents the attention vector between the dynamic objects and the attention vector between the dynamic object and the static map element; merging the attention vectors of the dynamic objects and inputting them into the linear residual decoder to generate the modeling trajectory corresponding to the dynamic object.

[0009] The step of training the autonomous driving scene generation model according to the objective function to obtain the trained autonomous driving scene generation model further includes: adjusting the network parameters of the autonomous driving scene generation model in response to the error between the modeled trajectory and the original trajectory not meeting a preset value, so as to update the autonomous driving scene generation model.

[0010] The second aspect of this application provides a method for generating an autonomous driving scenario, applied to a target vehicle, including:

[0011] The process involves: acquiring an input scene; inputting the input scene into an autonomous driving scene generation model to obtain the modeling trajectory of the target vehicle, thereby obtaining the autonomous driving scene; the autonomous driving scene generation model includes an encoder and a decoder; wherein the autonomous driving scene generation model is trained using the training method for the autonomous driving scene generation model described in the first aspect of this application.

[0012] The input scenario includes static map elements. The step of inputting the input scenario into the autonomous driving scenario generation model to obtain the modeling trajectory of the target vehicle includes: inputting the data of the static map elements into the encoder to obtain the encoded data of the static map elements; inputting the trajectory data of the target vehicle into the encoder to obtain the trajectory encoded data of the target vehicle, and adding a random vector to the trajectory encoded data of the target vehicle to obtain the trajectory stitching vector of the target vehicle; merging the encoded data of the static map elements and the trajectory stitching vector of the target vehicle and inputting them into the decoder to obtain the modeling trajectory of the target vehicle.

[0013] The step of inputting the input scene into the autonomous driving scene generation model to obtain the modeling trajectory of the target vehicle includes: inputting the data of the static map elements into the encoder to obtain the encoded data of the static map elements; obtaining a random vector and using the random vector as the encoded data of the target vehicle; merging the encoded data of the static map elements and the encoded data of the target vehicle and inputting them into the decoder to obtain the modeling trajectory of the target vehicle.

[0014] A third aspect of this application provides an electronic device including a memory and a processor coupled to each other. The processor is configured to execute program instructions stored in the memory to implement the training method of the training model of the autonomous driving scene generation model described in the first aspect of this application, or to implement the autonomous driving scene generation method described in the second aspect of this application.

[0015] The fourth aspect of this application provides a non-volatile computer-readable storage medium storing program instructions thereon, which, when executed by a processor, implement the training method for the autonomous driving scene generation model described in the first aspect of this application, or implement the autonomous driving scene generation method described in the second aspect of this application.

[0016] The above scheme trains the autonomous driving scene generation model using a training dataset. Since the training dataset includes a first subset of data representing static map elements and a second subset of data representing the original trajectories of dynamic objects, training with specific descriptive data makes the trained model more accurate. Furthermore, training is performed using an objective function, which associates static map elements with the modeling trajectories of dynamic objects, making the modeling trajectories output by the autonomous driving scene generation model more accurate.

[0017] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this application. Attached Figure Description

[0018] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with this application and, together with the specification, serve to explain the technical solutions of this application.

[0019] Figure 1 This is a flowchart illustrating an embodiment of the training method for the autonomous driving scene generation model of this application;

[0020] Figure 2 This is a flowchart illustrating the first embodiment of the method for generating autonomous driving scenarios according to this application:

[0021] Figure 3 This is a flowchart illustrating the second embodiment of the method for generating autonomous driving scenarios according to this application;

[0022] Figure 4 This is a flowchart illustrating the third embodiment of the method for generating autonomous driving scenarios in this application;

[0023] Figure 5 This is a schematic diagram of the framework of an embodiment of the electronic device of this application;

[0024] Figure 6 This is a schematic diagram of a framework of an embodiment of the non-volatile computer-readable storage medium of this application. Detailed Implementation

[0025] The embodiments of this application will now be described in detail with reference to the accompanying drawings.

[0026] In the following description, specific details such as particular system architectures, interfaces, and technologies are presented for illustrative purposes rather than for limiting purposes, in order to provide a thorough understanding of this application.

[0027] In this document, the term "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Additionally, the character " / " generally indicates that the preceding and following related objects have an "or" relationship. Furthermore, "many" in this document means two or more. Moreover, the term "at least one" in this document means any combination of at least two of any one or more of a plurality of objects. For example, including at least one of A, B, and C can mean including any one or more elements selected from the set consisting of A, B, and C.

[0028] Please see Figure 1 , Figure 1 This is a flowchart illustrating an embodiment of the training method for the autonomous driving scene generation model of this application. The method is applied to an electronic device, which can be installed in a vehicle and can be an in-vehicle device. It should be noted that if substantially the same result is obtained, the method of this application is not necessarily identical. Figure 1 The sequence of processes shown is limited.

[0029] In some possible implementations, the training method for this autonomous driving scenario generation model can be implemented by the processor calling computer-readable instructions stored in memory. This may include the following steps:

[0030] Step S11: Obtain a training dataset for describing autonomous driving scenarios, wherein the autonomous driving scenarios include static map elements and dynamic objects, and the training dataset includes a first data subset representing static map elements and a second data subset representing the original trajectories of dynamic objects.

[0031] Static map elements include elements such as road centerlines, while dynamic objects include vehicles and pedestrians. The selection of static map elements and dynamic objects depends on the requirements and is not limited here. The first data subset includes the position coordinates of each static map element, and the second data subset includes the position coordinates, speed, and orientation angle of each dynamic object. The original trajectory is described by a series of position coordinates. This autonomous driving scenario can be a simulation scenario based on actual road conditions or an actual road condition scenario; there is no restriction.

[0032] The training dataset can be obtained from the raw data of the autonomous driving scenario. For example, the raw data can be processed to obtain the training dataset.

[0033] Specifically, the process involves extracting trajectory data of dynamic objects and static map data of static map elements from the raw data. This can include data corresponding to n dynamic objects and m static map elements. The trajectory data is used to extract attribute information such as position, velocity, and orientation angle of the dynamic objects, while the static map data is used to extract position information for the static map elements representing lane centerlines. The trajectory data of each dynamic object is processed according to its attribute information and maximum time steps to obtain a trajectory array for each dynamic object. Similarly, each lane centerline is processed according to its position information and maximum number of coordinate points to obtain a static map array for each lane centerline. The second data subset includes the trajectory array for each dynamic object, and the first data subset includes the static map array for each lane centerline.

[0034] This embodiment includes n dynamic objects, and the resulting second data subset is an array of (n x T x 7), where T represents the maximum number of time steps, and 7 represents the number of position information (x, y, z), velocity (Vx, Vy, Vz), and orientation angle values. Taking an autonomous driving scenario with a total time of 10 seconds as an example, it is divided into 10 time periods, each lasting 1 second, and each time period is divided into 10 frames. Therefore, the maximum number of time steps is 10 * 10 = 100. If the movement time of some dynamic objects is 3 seconds, less than 10 seconds, the time step count is 30, meaning the maximum number of time steps has not been reached. In this case, the subsequent 7 seconds of time are padded with the data from the last second of the 3-second period. This process of neighborhood padding is performed on the trajectory data of the dynamic objects that do not meet the 10-second time requirement, ensuring the consistency of the data dimensions.

[0035] This embodiment includes m static map elements, resulting in a first data subset of an array (m x L x 7), where L represents the maximum number of coordinate points and 7 represents the number of position information (x, y, z) values. All road centerlines in the autonomous driving scenario are acquired, and each road centerline is divided into coordinates according to a preset coordinate system. Each centerline is discontinuous according to preset values, and each road centerline may include multiple coordinate points. The value with the most coordinate points is taken as the maximum number of coordinate points, i.e., the number of coordinate points corresponding to the longest road centerline. For other road centerlines where the number of coordinate points does not reach the maximum number, the position information of the last coordinate point of each road centerline is used to supplement the difference in coordinate points. The number of coordinate points of each road centerline is then padded to the maximum number of coordinate points; that is, neighborhood padding is performed on the data of road centerlines that do not meet the maximum number of coordinate points to ensure data dimensionality consistency.

[0036] Step S12: Input the first data subset and the second data subset into the autonomous driving scene generation model to train it according to the objective function, thereby obtaining the trained autonomous driving scene generation model. The objective function is a function that includes condition variables and modeling variables. The condition variables represent static map elements, and the modeling variables represent the modeling trajectories of dynamic objects.

[0037] The objective function is constructed based on the Conditional Variational Autoencoder (CVAE). This objective function serves as the loss function. Data representing static map elements indicates conditional variables, while data representing dynamic objects indicates modeling variables. This process trains an autonomous driving scene generation model, which outputs the modeled trajectories of dynamic objects. The objective function is designed to ensure that the modeled trajectory output by the autonomous driving scene generation model is almost identical to the original trajectory; that is, the error between the two is sufficiently small. For example, the average positional error can be less than 0.5 meters. The average positional error is calculated using the corresponding position coordinates.

[0038] In this embodiment, the autonomous driving scene generation model is trained using a training dataset. Since the training dataset includes a first subset of data representing static map elements and a second subset of data representing the original trajectories of dynamic objects, training with specific descriptive data makes the trained model more accurate. Furthermore, training is performed using an objective function, which associates static map elements with the modeling trajectories of dynamic objects, making the modeling trajectories output by the autonomous driving scene generation model more accurate.

[0039] As described above, the first and second data subsets are input into the autonomous driving scene generation model for training according to the objective function to obtain a trained autonomous driving scene generation model. In some embodiments, the autonomous driving scene change model includes an encoder and a decoder. Inputting the first and second data subsets into the autonomous driving scene generation model includes: inputting the first and second data subsets into the encoder respectively to obtain a first encoded dataset corresponding to static map elements and a second encoded dataset corresponding to the original trajectory of dynamic objects; wherein, in the objective function, the condition variable represents the first encoded dataset corresponding to static map elements, and the modeling variable represents the second encoded dataset corresponding to the original trajectory of dynamic objects; the first encoded dataset and the second encoded dataset are input into the decoder to obtain the modeled trajectory corresponding to the dynamic objects.

[0040] In this embodiment, the first encoded dataset corresponding to the first data subset is used as the condition variable of the objective function, and the second encoded dataset corresponding to the second data subset is used as the modeling variable of the objective function. According to the objective function, the training operation is performed so that the modeling trajectory output by the decoder is exactly the same as the original trajectory.

[0041] At this point, step S12, which involves training according to the objective function to obtain an autonomous driving scene generation model, may include:

[0042] When the objective function is minimized, the network parameters of the autonomous driving scenario generation model are obtained, thus yielding the autonomous driving scenario generation model.

[0043] In this embodiment, minimizing the objective function represents minimizing the error between the modeled trajectory and the original trajectory. The closer the reconstructed trajectory output by the autonomous driving scene generation model is to the original trajectory, for example, when the error is less than 0.5%, it indicates that the training is complete. The network parameters at this time are obtained to obtain the driving scene generation model, thus the trained autonomous driving scene generation model is more accurate.

[0044] As described above, the first data subset and the second data subset are respectively input into the encoder to obtain the first encoded dataset corresponding to the static map elements and the second encoded dataset corresponding to the original trajectory of the dynamic object. In some embodiments, the encoder includes at least two convolutional layers and a max pooling layer. The at least two convolutional layers are connected in sequence, and the last convolutional layer in the at least two convolutional layers is connected to the max pooling layer. The output dimensions of the at least two convolutional layers increase sequentially. The max pooling layer is used to set the dimension to 1 when the length of the sequence output by the last convolutional layer is greater than 1.

[0045] At this point, the first data subset and the second data subset are respectively input into the encoder to obtain the first encoded dataset corresponding to the static map elements and the second encoded dataset corresponding to the original trajectory of the dynamic objects. This includes: performing convolution operations on the first data subset and the second data subset through at least two convolutional layers and a max pooling layer to obtain the first encoded dataset and the second encoded dataset. Both the first encoded dataset and the second encoded dataset are two-dimensional arrays and have the same sequence length.

[0046] In the objective function, the condition variable represents the first encoded dataset corresponding to the static map element, and the modeling variable represents the second encoded dataset corresponding to the original trajectory of the dynamic object. The first encoded dataset and the second encoded dataset are input into the decoder to obtain the modeled trajectory corresponding to the dynamic object.

[0047] The following description uses a four-layer network structure as an example. This four-layer network is a four-layer one-dimensional convolutional structure, with output dimensions of 64, 128, 256, and 512 for each layer, respectively. The kernel size is 3, and the stride is 2. The first and second data subsets are input into the encoder and convolved sequentially according to their output dimensions, resulting in a final output dimension of 512. After convolution, the length of the sequence output by the last convolutional layer is checked. If the length exceeds 1, a max pooling layer (MAX layer) is used to reduce this output dimension to 1, resulting in an output dimension of (1x512). When there are m static map elements, the first encoded dataset corresponding to the first data subset is (m x 512); when there are n dynamic objects, the second encoded dataset corresponding to the second data subset is (n x 512). The decoding process is described in the above embodiment and will not be repeated here.

[0048] In this embodiment, the data of the first data subset and the second data subset are encoded to obtain corresponding two-dimensional arrays, which facilitates subsequent decoding calculations. Finally, the two two-dimensional arrays are concatenated and then decoded to obtain the modeling trajectory.

[0049] In some embodiments, the decoder includes a multi-head attention mechanism and a linear residual decoder; inputting a first encoded dataset and a second encoded dataset into the decoder to obtain the modeling trajectory corresponding to the dynamic object may include:

[0050] The first and second encoded datasets are processed using a multi-head attention mechanism to obtain attention vectors for dynamic objects. These attention vectors represent the attention vectors between dynamic objects and the attention vectors between dynamic objects and static map elements. The attention vectors of the dynamic objects are then merged and input into a linear residual decoder to generate the modeling trajectory corresponding to the dynamic objects.

[0051] In this embodiment, the first and second encoded datasets are merged and then processed using a multi-head attention mechanism. The multi-head attention mechanism can calculate a first attention vector between each dynamic object and other dynamic objects based on the second data encoding set, and calculate a second attention vector between each dynamic object and each static map element based on the first and second data encoding sets. The first and second attention vectors of each dynamic object are then merged and input into the linear residual decoder.

[0052] The linear residual decoder in this embodiment includes a dense layer and a residual structure. It merges the attention vectors of dynamic objects, applies a non-linear transformation to the dense layer, extracts the correlations between these features, and finally maps them onto the output space. The resulting reconstructed trajectory is then obtained. The residual structure addresses the vanishing and exploding gradient problems encountered during feature mapping, making the output modeled trajectory more accurate.

[0053] As described above, the first and second data subsets are input into the autonomous driving scene generation model to train it according to the objective function, thereby obtaining a trained autonomous driving scene generation model. In some embodiments, training according to the objective function to obtain a trained autonomous driving scene generation model further includes:

[0054] In response to the error between the modeled trajectory and the original trajectory not meeting the preset value, the network parameters of the autonomous driving scene generation model are adjusted to update the autonomous driving scene generation model.

[0055] This embodiment optimizes the autonomous driving scene generation model by analyzing the error between the modeled trajectory and the original trajectory. The preset values ​​can be set according to requirements. For example, the similarity between the modeled trajectory and the original trajectory is calculated. When the similarity does not reach 95%, the network parameters of the autonomous driving scene generation model need to be adjusted, that is, the network parameters of the autonomous driving scene generation model need to be updated, so as to obtain a more accurate autonomous driving scene generation model, that is, a trained autonomous driving scene generation model.

[0056] Please see Figure 2 , Figure 2 This is a flowchart illustrating the first embodiment of the autonomous driving scenario generation method of this application, applied to a target vehicle, specifically, to a device on the target vehicle, such as an in-vehicle device. Figure 2 As shown, the method includes:

[0057] Step S21: Obtain the input scenario.

[0058] The input scenario can be obtained through sensor devices, including image sensors and radar sensors. The radar sensor can be a radar device used for autonomous driving that meets accuracy requirements and provides point cloud perception. Image sensors, such as cameras, can be used to acquire image data. Radar sensors, such as millimeter-wave radar and lidar, can be used to acquire point cloud data. Image sensors and radar sensors can be mounted on a mobile device, such as an autonomous vehicle. LiDAR can include mechanical lidar, semi-solid-state lidar, or solid-state lidar.

[0059] The input scenarios include data related to static maps and data related to dynamic objects. Data related to static maps may include road centerlines, while data related to dynamic objects may include vehicle data.

[0060] Step S22: Input the input scene into the autonomous driving scene generation model to obtain the modeling trajectory of the target vehicle, thereby obtaining the autonomous driving scene; the autonomous driving scene generation model includes an encoder and a decoder. The autonomous driving scene generation model is trained using the training method of the autonomous driving scene generation model in the above embodiment.

[0061] In this embodiment, the data in the input scene is processed by an autonomous driving scene generation model. This includes encoding the data in the input scene using an encoder to obtain scene encoded data, and decoding the scene encoded data using a decoder to obtain the modeled trajectories of the target vehicle and other vehicles, thereby obtaining the autonomous driving scene.

[0062] This embodiment processes the actual input scene using an autonomous driving scene generation model to obtain a more accurate modeling trajectory of the target scene.

[0063] Please see Figure 3 , Figure 3 This is a flowchart illustrating the second embodiment of the autonomous driving scenario generation method of this application, applied to a target vehicle, specifically, to a device on the target vehicle, such as an in-vehicle device. Figure 3 As shown, the method includes:

[0064] Step S31: Obtain the input scene; wherein, the input scene includes static map elements.

[0065] In this embodiment, static map elements include road centerlines, and the data for these static map elements includes the location information of each road centerline. The input scenario also includes the trajectory data of the target vehicle, which may include the target vehicle's location information, speed, and orientation angle.

[0066] Step S32: Input the trajectory data of the target vehicle into the encoder to obtain the trajectory encoding data of the target vehicle, and add a random vector to the trajectory encoding data of the target vehicle to obtain the trajectory splicing vector of the target vehicle.

[0067] In this embodiment, the trajectory data of the target vehicle can be processed array data, which includes the target vehicle's position, speed, and orientation angle in the input scene. When encoding the trajectory data, a random vector generator can be used to generate random vectors, adding random vector perturbation to the trajectory encoding data to obtain the target vehicle's trajectory splicing vector.

[0068] Step S33: Merge the encoded data of the static map elements with the trajectory splicing vector of the target vehicle and input it into the decoder to obtain the modeling trajectory of the target vehicle.

[0069] This embodiment combines the coded data of the static map with the trajectory stitching data of the target vehicle and inputs it into the decoder to generate data information corresponding to the target vehicle, i.e., the data information of the modeled trajectory. It can also output the modeled trajectories of other vehicles to obtain new scenes.

[0070] Please see Figure 4 , Figure 4 This is a flowchart illustrating the third embodiment of the autonomous driving scenario generation method of this application, applied to a target vehicle, specifically, to a device on the target vehicle, such as an in-vehicle device. Figure 4 As shown, the method includes:

[0071] Step S41: Obtain the input scenario.

[0072] This step has been described in the above embodiments and will not be repeated here.

[0073] Step S42: Obtain a random vector and use the random vector as the encoding data of the target vehicle.

[0074] Step S43: Merge the encoded data of the static map elements with the encoded data of the target vehicle and input them into the decoder to obtain the modeling trajectory of the target vehicle.

[0075] In this embodiment, a random vector is generated by a random noise generator. The random vector is then combined with the encoded data of static map elements as a disturbance and input into a decoder for decoding. The encoded data of static map elements is used to assist in generating the modeling trajectory, resulting in a more accurate modeling trajectory.

[0076] Those skilled in the art will understand that, in the above-described method of the specific implementation, the order in which each step is written does not imply a strict execution order and does not constitute any limitation on the implementation process. The specific execution order of each step should be determined by its function and possible internal logic.

[0077] Please see Figure 5 , Figure 5 This is a schematic diagram of a framework of an embodiment of the electronic device 50 of this application. The electronic device 50 is installed in a vehicle and can be an in-vehicle device. The electronic device 50 includes a memory 51 and a processor 52 coupled to each other. The processor 52 is used to execute program instructions stored in the memory 51 to implement the steps of any of the above-described embodiments of the training method for generating autonomous driving scenarios, or to implement the steps of any of the above-described embodiments of the method for generating autonomous driving scenarios. In a specific implementation scenario, the electronic device 50 includes, but is not limited to, a microcomputer or a server, and is not limited thereto.

[0078] The processor 52 can be used to control itself and the memory 51 to implement the steps of any of the above-described embodiments of the training method for generating autonomous driving scenarios, or to implement the steps of any of the above-described embodiments of the method for generating autonomous driving scenarios. The processor 52 can also be called a CPU (Central Processing Unit). The processor 52 may be an integrated circuit chip with signal processing capabilities. The processor 52 can also be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. The general-purpose processor can be a microprocessor or any conventional processor. Furthermore, the processor 52 can be implemented using integrated circuit chips.

[0079] Please see Figure 6 , Figure 6 This is a schematic diagram of a framework of an embodiment of the non-volatile computer-readable storage medium 60 of this application. The non-volatile computer-readable storage medium 60 stores program instructions 601 that can be executed by a processor. The program instructions 601 are used to implement the steps of any of the above-described embodiments of the training method for generating autonomous driving scenarios, or to implement the steps of any of the above-described embodiments of the method for generating autonomous driving scenarios.

[0080] In some embodiments, the functions or modules of the apparatus provided in this disclosure can be used to perform the methods described in the above method embodiments. The specific implementation can be referred to the description of the above method embodiments, and for the sake of brevity, it will not be repeated here.

[0081] The description of the various embodiments above tends to emphasize the differences between the various embodiments. The similarities or similarities between them can be referred to, and for the sake of brevity, they will not be repeated here.

[0082] In the several embodiments provided in this application, it should be understood that the disclosed methods and apparatus can be implemented in other ways. For example, the apparatus implementations described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection of devices or units may be electrical, mechanical, or other forms.

[0083] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0084] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute all or part of the steps of the methods of various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

Claims

1. A training method for an autonomous driving scene generation model, characterized in that, include: Obtain a training dataset for describing an autonomous driving scenario, wherein the autonomous driving scenario includes static map elements and dynamic objects, and the training dataset includes a first data subset representing the static map elements and a second data subset representing the original trajectories of the dynamic objects; The first data subset and the second data subset are input into the autonomous driving scene generation model to train it according to the objective function, thereby obtaining a trained autonomous driving scene generation model. The objective function is a function that includes conditional variables and modeling variables. The conditional variables represent the static map elements, and the modeling variables represent the modeling trajectories of the dynamic objects. The objective function is constructed according to a conditional variational autoencoder, and the autonomous driving scene generation model is used to output the modeling trajectory of the vehicle. The autonomous driving scene generation model includes an encoder and a decoder. The decoder includes a multi-head attention mechanism and a linear residual decoder; The step of inputting the first data subset and the second data subset into the autonomous driving scenario generation model includes: The first data subset and the second data subset are respectively input into the encoder to obtain the first encoded dataset corresponding to the static map element and the second encoded dataset corresponding to the original trajectory of the dynamic object; wherein, in the objective function, the condition variable represents the first encoded dataset corresponding to the static map element, and the modeling variable represents the second encoded dataset corresponding to the original trajectory of the dynamic object; By using a multi-head attention mechanism, the first encoded dataset and the second encoded dataset are processed to obtain the attention vector of the dynamic object. The attention vector represents the attention vector between the dynamic objects and the attention vector between the dynamic object and the static map element. The attention vectors of the dynamic objects are merged and input into a linear residual decoder to generate the modeling trajectory corresponding to the dynamic objects.

2. The method according to claim 1, characterized in that, The process of training according to the objective function to obtain the autonomous driving scene generation model includes: When the objective function is minimized, the network parameters of the autonomous driving scenario generation model are obtained, thereby obtaining the autonomous driving scenario generation model.

3. The method according to claim 2, characterized in that, The encoder includes at least two convolutional layers and a max pooling layer. The at least two convolutional layers are connected sequentially, and the last convolutional layer is connected to the max pooling layer. The output dimensions of the at least two convolutional layers increase sequentially. The max pooling layer is used to set the dimension to 1 when the sequence length output by the last convolutional layer is greater than 1. The first data subset and the second data subset are respectively input into the encoder to obtain a first encoded dataset corresponding to the static map element and a second encoded dataset corresponding to the original trajectory of the dynamic object, including: The first data subset and the second data subset are convolutionally processed by the at least two convolutional layers and the max pooling layer to obtain the first encoded dataset and the second encoded dataset. The first encoded dataset and the second encoded dataset are both two-dimensional arrays and have the same sequence length.

4. The method according to claim 2, characterized in that, The step of training according to the objective function to obtain the trained autonomous driving scene generation model further includes: In response to the error between the modeled trajectory and the original trajectory not meeting a preset value, the network parameters of the autonomous driving scene generation model are adjusted to update the autonomous driving scene generation model.

5. A method for generating autonomous driving scenarios, characterized in that, Applied to target vehicles, including: Obtain the input scenario; The input scenario is input into the autonomous driving scenario generation model to obtain the modeling trajectory of the target vehicle, thereby obtaining the autonomous driving scenario; the autonomous driving scenario generation model includes an encoder and a decoder; The autonomous driving scene generation model is trained using the training method for the autonomous driving scene generation model according to any one of claims 1 to 4.

6. The method according to claim 5, characterized in that, The input scenario includes static map elements; The step of inputting the input scenario into the autonomous driving scenario generation model to obtain the modeling trajectory of the target vehicle includes: The data of the static map element is input into the encoder to obtain the encoded data of the static map element; The trajectory data of the target vehicle is input into the encoder to obtain the trajectory encoded data of the target vehicle, and a random vector is added to the trajectory encoded data of the target vehicle to obtain the trajectory splicing vector of the target vehicle. The encoded data of the static map elements is merged with the trajectory splicing vector of the target vehicle and input into the decoder to obtain the modeled trajectory of the target vehicle.

7. The method according to claim 5, characterized in that, The step of inputting the input scenario into the autonomous driving scenario generation model to obtain the modeling trajectory of the target vehicle includes: The data of the static map element is input into the encoder to obtain the encoded data of the static map element; Obtain a random vector and use the random vector as the encoded data of the target vehicle; The encoded data of the static map elements are merged with the encoded data of the target vehicle and input into the decoder to obtain the modeling trajectory of the target vehicle.

8. An electronic device, characterized in that, It includes a memory and a processor that are coupled to each other, the processor being used to execute program instructions stored in the memory to implement the training method of the autonomous driving scene generation model according to any one of claims 1 to 4, or to implement the autonomous driving scene generation method according to any one of claims 5 to 7.

9. A non-volatile computer-readable storage medium storing program instructions thereon, characterized in that, When the program instructions are executed by the processor, they implement the training method of the autonomous driving scene generation model according to any one of claims 1 to 4, or the method of generating autonomous driving scenes according to any one of claims 5 to 7.

Images