Motion trajectory generation method and device, electronic equipment and storage medium

Abstract

Embodiments of the present application disclose a motion trajectory generation method and device, electronic equipment and storage medium. Applied to the field of automated testing. Including: obtaining historical motion trajectory data of at least one reference vehicle and a target vehicle. Using a trajectory generation model to determine a social information vector corresponding to the target vehicle according to the historical motion trajectory data of all reference vehicles. Using the trajectory generation model to determine a first historical trajectory feature vector corresponding to the target vehicle according to the historical motion trajectory data of the target vehicle. Using the trajectory generation model to generate a motion planning trajectory of the target vehicle according to the social information vector corresponding to the target vehicle and the trajectory feature vector corresponding to the target vehicle. Embodiments of the present application can quickly generate motion trajectories of multiple traffic participants using a trajectory generation model. In this way, multiple traffic participants can be added to the test scene to increase the complexity of the test scene.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of automated testing, and in particular, to a method and device for generating a motion trajectory, an electronic device, and a storage medium. BACKGROUND

[0002] With the rapid development of automatic driving technology, high-level automatic driving cars have gradually entered people's lives. According to the test process of the automatic driving car, it is estimated that the automatic driving car needs to be tested for 1.3 million miles to ensure the safety of the system, and the automatic driving car also needs to be tested in various complex test scenes. Therefore, the traditional site test and road test cannot meet the test needs of the automatic driving car.

[0003] This is because the traditional site test and road test need a large number of supporting equipment and related experienced technical personnel, which will cause huge consumption of manpower, material resources and financial resources in the test process. At the same time, due to the site reason, the traditional road test will have the problem of being unable to test complex and dangerous scenes. In view of the above difficulties, the in-loop test is adopted to test in the development and verification stage of the automatic driving car, which not only can greatly save time and cost, but also can carry out some dangerous test scenes that cannot be realized in actual site or road test.

[0004] However, the current in-loop test does not have a complex scene generation system that can support high-level automatic driving function test, and it is difficult to perform complete function test on the high-level automatic driving car. Therefore, how to generate a complex test scene in the in-loop test scene has become a problem to be solved. SUMMARY

[0005] The present application provides a method and device for generating a motion trajectory, an electronic device, and a storage medium. The trajectory generation model is used to analyze and integrate the historical motion trajectory data of at least one reference vehicle and a target vehicle, and then generate the motion trajectory of the target vehicle according to the analysis result. In this way, the trajectory generation model can be used to generate the motion trajectory of multiple traffic participants, so that the complexity of the test scene can be increased by adding multiple traffic participants to the test scene.

[0006] In a first aspect, an embodiment of the present application provides a method for generating a motion trajectory, which includes:

[0007] Obtaining historical motion trajectory data of at least one reference vehicle and a target vehicle, the at least one reference vehicle being a surrounding vehicle of the target vehicle in a driving scene.

[0008] Using a trajectory generation model to determine a social information vector corresponding to the target vehicle according to the historical motion trajectory data of all reference vehicles.

[0009] The trajectory generation model is used to determine a first historical trajectory feature vector corresponding to the target vehicle according to historical motion trajectory data of the target vehicle.

[0010] The trajectory generation model is used to generate a motion planning trajectory of the target vehicle according to the social information vector corresponding to the target vehicle and the first historical trajectory feature vector corresponding to the target vehicle.

[0011] In an optional implementation, the trajectory generation model comprises an encoder layer used to extract spatiotemporal sequence features. The method further comprises:

[0012] The historical motion trajectory data of the at least one reference vehicle is input into the encoder layer.

[0013] The second historical trajectory feature vector corresponding to each reference vehicle is obtained through the encoder layer.

[0014] The trajectory generation model is used to determine a first historical trajectory feature vector corresponding to the target vehicle according to historical motion trajectory data of the target vehicle, comprising:

[0015] The historical motion trajectory data of the target vehicle is input into the encoder layer.

[0016] The first historical trajectory feature vector corresponding to the target vehicle is obtained through the encoder layer.

[0017] In an optional implementation, the trajectory generation model comprises a convolutional pooling layer.

[0018] The trajectory generation model is used to determine a social information vector corresponding to the target vehicle according to historical motion trajectory data of all reference vehicles, comprising:

[0019] All second historical trajectory feature vectors are input into the convolutional pooling layer, and the convolutional pooling layer is used to perform feature fusion on all second historical trajectory feature vectors to obtain the social information vector.

[0020] In an optional implementation, the method further comprises:

[0021] The social information vector is subjected to a pooling process.

[0022] In an optional implementation, the trajectory generation model comprises a decoder layer.

[0023] The trajectory generation model is used to generate a motion planning trajectory of the target vehicle according to the social information vector corresponding to the target vehicle and the first historical trajectory feature vector corresponding to the target vehicle, comprising:

[0024] The social information vector subjected to the pooling process and the first historical trajectory feature vector are spliced to obtain a trajectory encoding vector.

[0025] The trajectory encoding vector is input into a decoder layer to obtain a motion planning trajectory of the target vehicle. The motion planning trajectory of the target vehicle is used to describe the positions of the predicted trajectory points corresponding to each time point within a preset time period.

[0026] In an optional implementation, the method further includes:

[0027] The trajectory generation model is trained. The training process of the trajectory generation model is as follows:

[0028] The vehicle driving sample data is obtained, and the vehicle driving sample data includes historical motion trajectory data corresponding to the first vehicle, historical motion trajectory data corresponding to the second vehicle, and current motion trajectory data corresponding to the second vehicle.

[0029] The historical motion trajectory data corresponding to the first vehicle and the historical motion trajectory data corresponding to the second vehicle are input into the trajectory generation model, and a motion trajectory prediction result corresponding to the second vehicle is obtained through the trajectory generation model.

[0030] According to the motion trajectory prediction result and the current motion trajectory data, a loss value is determined.

[0031] According to the loss value, the model parameters of the trajectory generation model are adjusted.

[0032] When a training condition is reached, the training process of the trajectory generation model is stopped.

[0033] In an optional implementation, the training condition is that the loss value is less than a preset threshold or the number of training times reaches a preset number.

[0034] In a second aspect, an embodiment of the present application provides a motion trajectory generation device, which includes:

[0035] An acquisition unit is configured to acquire historical motion trajectory data of at least one reference vehicle and a target vehicle. The at least one reference vehicle is a surrounding vehicle of the target vehicle in a driving scene.

[0036] A determination unit is configured to determine a social information vector corresponding to the target vehicle according to the historical motion trajectory data of all the reference vehicles by using a trajectory generation model.

[0037] The determination unit is further configured to determine a first historical trajectory feature vector corresponding to the target vehicle according to the historical motion trajectory data of the target vehicle by using the trajectory generation model.

[0038] A generation unit is configured to generate a motion planning trajectory of the target vehicle according to the social information vector corresponding to the target vehicle and the first historical trajectory feature vector corresponding to the target vehicle by using the trajectory generation model.

[0039] In an optional implementation, the trajectory generation model comprises an encoder layer configured to extract spatio-temporal sequence features. The generation apparatus further comprises an input-output unit.

[0040] The input-output unit is configured to input the historical motion trajectory data of the at least one reference vehicle into the encoder layer.

[0041] The obtaining unit is further configured to obtain, by using the encoder layer, second historical trajectory feature vectors corresponding to the reference vehicles respectively.

[0042] The input-output unit is further configured to input the historical motion trajectory data of the target vehicle into the encoder layer.

[0043] The determining unit is specifically configured to obtain, by using the encoder layer, a first historical trajectory feature vector corresponding to the target vehicle.

[0044] In an optional implementation, the trajectory generation model comprises a convolutional pooling layer.

[0045] The input-output unit is further configured to input all the second historical trajectory feature vectors into the convolutional pooling layer.

[0046] The determining unit is specifically configured to perform feature fusion on all the second historical trajectory feature vectors by using the convolutional pooling layer, to obtain a social information vector.

[0047] In an optional implementation, the determining unit is further configured to perform pooling processing on the social information vector.

[0048] In an optional implementation, the trajectory generation model comprises a decoder layer.

[0049] The generation unit is specifically configured to concatenate the social information vector after the pooling processing and the first historical trajectory feature vector, to obtain a trajectory encoding vector.

[0050] The input-output unit is further configured to input the trajectory encoding vector into the decoder layer.

[0051] The generation unit is specifically configured to obtain a motion planning trajectory of the target vehicle. The motion planning trajectory of the target vehicle is configured to describe positions of predicted trajectory points corresponding to respective time points within a preset time period.

[0052] In an optional implementation, the generation apparatus further comprises a training unit.

[0053] The training unit is specifically configured to train the trajectory generation model. The training process of the trajectory generation model comprises:

[0054] Obtain vehicle driving sample data, the vehicle driving sample data comprising historical motion trajectory data corresponding to the first vehicle, historical motion trajectory data corresponding to the second vehicle, and current motion trajectory data corresponding to the second vehicle.

[0055] Input the historical motion trajectory data corresponding to the first vehicle and the historical motion trajectory data corresponding to the second vehicle into a trajectory generation model, and obtain a motion trajectory prediction result corresponding to the second vehicle through the trajectory generation model.

[0056] Determine a loss value according to the motion trajectory prediction result and the current motion trajectory data.

[0057] Adjust model parameters of the trajectory generation model according to the loss value.

[0058] Stop the training process of the trajectory generation model when a training condition is reached.

[0059] In an optional implementation, the training condition is that the loss value is less than a preset threshold or the number of training times reaches a preset number.

[0060] The third aspect of the embodiments of the present application further provides an electronic device, comprising a memory and a processor, the memory and the processor being coupled.

[0061] The memory is configured to store one or more computer instructions.

[0062] The processor is configured to execute the one or more computer instructions to implement the motion trajectory generation method of the first aspect.

[0063] The fourth aspect of the embodiments of the present application further provides a computer readable storage medium, which stores one or more computer instructions, and the instructions are executed by a processor to implement the motion trajectory generation method of any one of the technical solutions.

[0064] In the embodiment of the present application, the trajectory generation model can be used to obtain the motion trajectory of the target vehicle. First, the historical motion trajectory data of the target vehicle and the reference vehicles around the target vehicle are obtained, and then the trajectory generation model is used to determine the social information vector corresponding to the target vehicle according to the historical motion trajectory data of all the reference vehicles. Next, the first historical trajectory feature vector corresponding to the target vehicle is determined according to the historical motion trajectory data of the target vehicle. Finally, the trajectory generation model is used to generate the motion planning trajectory of the target vehicle according to the social information vector corresponding to the target vehicle and the trajectory feature vector corresponding to the target vehicle. In this way, when the test vehicle is tested, the motion trajectories of the multiple traffic participants around the test vehicle can be quickly obtained according to the trajectory generation model. In this way, the motion states of the multiple traffic participants in the simulation test scene can be changed to efficiently and simply increase the complexity of the test scene. In this way, the problem of complex scene generation in the loop test system can be solved to perform more complete function tests on high-level autonomous vehicles. BRIEF DESCRIPTION OF DRAWINGS

[0065] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description are only some embodiments of the present application. Those skilled in the art can obtain other drawings according to these drawings without any creative effort.

[0066] Figure 1 A structural schematic diagram of an automatic driving test system provided by the embodiment of the present application;

[0067] Figure 2 A flowchart of a motion trajectory generation method provided by the embodiment of the present application;

[0068] Figure 3 A flowchart of a trajectory generation model training method provided by the embodiment of the present application;

[0069] Figure 4 A structural schematic diagram of a motion trajectory generation device provided by the embodiment of the present application;

[0070] Figure 5 A structural schematic diagram of an electronic device provided by the embodiment of the present application. DETAILED DESCRIPTION

[0071] The application provides a motion trajectory generation method and device, electronic equipment and a storage medium. The historical motion trajectory data of at least one reference vehicle and a target vehicle are analyzed and integrated by using a trajectory generation model, and then the motion trajectory of the target vehicle is generated according to the analysis result. In this way, the motion trajectories of multiple traffic participants can be generated by using the trajectory generation model, so that the complexity of the test scene can be increased by adding the multiple traffic participants to the test scene.

[0072] The technical solutions in the embodiments of the application will be clearly and completely described below with reference to the drawings in the embodiments of the application. Obviously, the described embodiments are only part of the embodiments of the application, rather than all the embodiments of the application. Based on the embodiments in the application, all other embodiments obtained by a person of ordinary skill in the art without creative labor fall within the protection scope of the application.

[0073] It should be noted that the terms 'include' and 'have' and any variations thereof in the embodiments of the application and the drawings are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device including a series of steps or units is not limited to the listed steps or units, but can optionally further include steps or units not listed or can optionally further include other steps or units inherent to the process, method, product or device.

[0074] With the rapid development of automatic driving technology, high-level automatic driving cars have gradually entered people's lives. Automatic driving cars with ADAS (Advanced Driving Assistance System) function, L1 (Level 1) and L2, have been put into mass production. At the same time, many car companies have begun to plan and promote the marketization process of L3 and L4 automatic driving cars. For the testing process of automatic driving cars, it is estimated that automatic driving cars need to be tested for an average of 1.3 million miles to ensure the safety of the system. Therefore, traditional field testing and road testing cannot meet the testing needs of automatic driving cars. This is because, using traditional road testing requires a large number of supporting equipment and experienced technical personnel, which will cause huge consumption of manpower, material resources and financial resources during the testing process. At the same time, due to the experience and site reasons of human beings, traditional field testing and road testing will result in poor repeatability of the tested functions and cannot test complex and dangerous scenes.

[0075] In view of the above difficulties, the test of the autonomous vehicle in the development and verification stage of the autonomous vehicle by using the loop test mode can not only greatly save time and cost, but also can carry out some dangerous test scenarios that cannot be realized on the actual site or road test. Therefore, the loop test has become an indispensable link for the test and verification of the autonomous vehicle. At present, there are various methods for loop test, from software-in-the-loop (SiL), model-in-the-loop (MiL), hardware-in-the-loop (HiL), to vehicle-in-the-loop (VIL). Different methods are aimed at different stages of autonomous driving research and test.

[0076] The test of the autonomous driving function at the vehicle level is usually carried out in the laboratory by using hardware and simulation software. The existing laboratory test scheme is based on the joint test of the shaft coupling dynamometer and the simulation software. The disadvantage of this method is that an additional steering mechanism needs to be added to realize the steering function of the wheels to achieve the purpose of high-level autonomous driving function test when testing the autonomous driving function. At the same time, the previous system does not have a complex scene generation system that can support high-level autonomous driving function test, and it is difficult to perform complete function test on high-level autonomous vehicles. Therefore, how to generate a complex test scene under the loop test scene has become a problem to be solved.

[0077] Based on the above problems, the embodiment of the present application provides a motion trajectory generation method and device, electronic equipment and storage medium. The trajectory generation model is used to analyze and integrate the historical motion trajectory data of at least one reference vehicle and a target vehicle, and then generate the motion trajectory of the target vehicle according to the analysis result. In this way, the trajectory generation model can be used to generate the motion trajectories of multiple traffic participants, so that the complexity of the test scene can be increased by adding multiple traffic participants to the test scene. The scheme provided by the embodiment of the present application will be described in detail in combination with specific embodiments.

[0078] Before introducing the detection method provided by the embodiment of the present application, the application scenario of the embodiment of the present application will be introduced. Figure 1 The structure schematic diagram of the autonomous driving test system provided by the embodiment of the present application is shown in FIG. 1. Figure 1 As shown in the figure, the autonomous driving test system includes the following modules:

[0079] The traffic flow generation module, the simulation module, the injection device, the intelligent test bench, and the interactive display module. The intelligent test bench corresponds to the intelligent test bench control unit and the vehicle dynamics module.

[0080] The traffic flow generation module is connected to one end of the simulation module, and the other end of the simulation module is connected to the tested vehicle through the injection device. The tested vehicle is placed on the intelligent test bench.

[0081] The traffic flow generation module is configured to provide complex driving scenarios and generate large-scale real traffic flow test scenarios. The traffic flow generation module can provide the simulation module with motion trajectories of multiple reference objects. The reference objects are surrounding moving objects that the test vehicle can encounter during actual driving, such as surrounding motor vehicles, roadside pedestrians, and surrounding non-motor vehicles. The reference objects need to interact with the test vehicle, and the test vehicle needs to control its own motion state and plan its own motion trajectory according to the motion of the surrounding moving objects, such as left turn, right turn, brake, acceleration, and deceleration. Therefore, the number of reference objects and the motion behavior of each reference object in the traffic flow generation module can be controlled to control the complexity of the test case. It can be understood that the more reference objects and the more complex motion behavior of each reference object (the more complex motion trajectory), the more dangerous the test scenario. When the safety performance of the test vehicle in a complex scenario needs to be tested, more dynamic interactive trajectory data sets can be constructed using the traffic flow generation module, and new reference object trajectories for dangerous interaction can be added continuously.

[0082] It can be understood that the motion trajectory generation method provided by the embodiments of the present application is applied to the traffic flow generation module, and the motion trajectory generation method is used to generate the motion trajectories of multiple reference objects, thereby increasing the simulation test scenario to generate various complex test scenarios.

[0083] The simulation module communicates with the traffic flow generation module to obtain the motion trajectory data of the reference objects in real time, and also communicates with the intelligent test bench to obtain the real-time motion data of the test vehicle. Then, the test scenario is simulated and restored according to the motion trajectory data of the reference objects and the real-time motion data of the test vehicle to obtain the simulation test scenario of the autonomous vehicle.

[0084] The interaction display module is connected with the simulation module and can display the simulation scenario in real time. In this way, the tester can observe the motion of the test vehicle in the simulation test scenario through the interaction display module to determine the safety of the test vehicle.

[0085] Then, the simulation module needs to inject the simulation test scene into the vehicle under test through the injection device, and the vehicle under test can plan and control the motion of the vehicle under test according to the simulation test scene. Among them, the injection device includes a video injection device and a radar injection device. The video injection device is used to inject the video data generated by the simulation scene into the camera module of the vehicle under test. The radar injection device injects the information of the reference target in the simulation scene into the radar processing module of the vehicle under test. In this way, the vehicle under test can determine the road conditions and the surrounding driving environment according to the video data and the radar data, so as to determine the subsequent driving behavior and plan the subsequent driving trajectory.

[0086] For example, the simulation module and the radar module are connected through a network switch and the like to inject the simulated traffic participants (reference targets) into the radar module. The radar module simulates the physical waveform of the reflected waveform of the radar after encountering an object, achieving the purpose of testing the radar.

[0087] For example, the simulation module and the GNSS (Global Navigation Satellite System) are connected through a network switch and the like to send the positioning information of the vehicle under test in the simulation test process to the GNSS system. The vehicle positioning device obtains the current vehicle position from the GNSS system to achieve the positioning purpose. The simulation module can also be electrically connected with the V2X (vehicle to everything) system, and the traffic participants and other information in the real-time simulation process are sent to the vehicle under test through the V2X in the form of 5G communication, etc. for sensing the data transmitted by the roadside facilities.

[0088] When the injection device injects the simulation test scene into the vehicle under test, the vehicle under test needs to plan and control the motion of the vehicle under test according to the simulation test scene. For example, the vehicle under test can include an intelligent driving perception unit and an intelligent driving control unit. The intelligent driving perception unit is electrically connected with the injection device to perceive various data corresponding to the simulation test scene. The intelligent driving control unit directly controls the acceleration, deceleration, turning, etc. of the vehicle.

[0089] The vehicle under test is directly placed on the intelligent test bench. The intelligent test bench is provided with a rotating drum to support the driving of the vehicle. The front ends of the two rollers of the rotating drum are in direct contact with the front wheels of the vehicle under test. When the vehicle under test drives, the two rollers in contact with the front wheels rotate. The two rollers are in a follow-up state, so they can support the follow-up steering motion of the wheels. The entity tray below the two rollers supports the rollers and the vehicle. A plurality of balls are placed below the entity tray to ensure the steering motion of the entity tray, thereby realizing the steering function of the front wheels of the vehicle.

[0090] The intelligent test bench control unit is electrically connected with the intelligent test bench, and performs rotation control, data acquisition and other operations on the intelligent test bench. That is, the intelligent test bench control unit directly tests the rotation data of the built-in rotating drum, and transmits the rotation data of the built-in rotating drum to the vehicle dynamics module. Then the vehicle dynamics module is used to determine the real-time motion data of the tested vehicle according to the rotation data of the built-in rotating drum, and transmits the real-time motion data of the tested vehicle to the simulation module. Specifically, the vehicle dynamics module calculates the speed, steering, roll angle, pitch angle, acceleration and other data according to the rotation speed and steering data of the rotating drum, and sends the data to the simulation module for updating the motion trajectory of the tested vehicle to realize closed loop test.

[0091] Based on the above automatic driving test system, the whole test process of the tested vehicle is as follows: first, the simulation module needs to build a model of the tested vehicle and traffic participants. The modeling of the tested vehicle can be based on the physical parameters and sensor configuration of the real tested vehicle. The physical model of the traffic participants (pedestrians, motor vehicles, non-motor vehicles) is also configured according to the real traffic conditions, including different types of pedestrians, different types of motor vehicles and different types of non-motor vehicles. Then the traffic flow generation module generates the motion trajectory of the pedestrian, motor vehicle and non-motor vehicle model close to the real traffic environment based on the deep learning neural network and motion trajectory generation method provided in the embodiments of the present application. At the same time, the density of the traffic flow can be modified according to the test requirements. That is, the motion behavior (motion trajectory) of a single traffic participant can be changed; at the same time, the number and proportion of pedestrians, motor vehicles and non-motor vehicles can be configured.

[0092] Then the traffic flow generation module inputs the motion trajectory of each traffic participant into the simulation module to change the motion state of the traffic participant in the simulation environment. Then the simulation module also needs to update the motion state of the tested vehicle in the simulation environment through the real-time motion data of the tested vehicle sent by the vehicle dynamics module. Finally, based on the updated tested vehicle and traffic participants, a new test environment is generated, and the updated test environment data is injected into the tested vehicle through the injection device, so that the tested vehicle controls the action, determines the subsequent motion behavior and motion trajectory.

[0093] In combination with the above application scenarios, the traffic flow generation module will be described in detail below. The traffic flow generation module can generate the motion trajectory of a plurality of reference targets according to a trajectory generation model. The trajectory generation model can include an encoder layer (Encoder), a convolutional pooling layer (Convolutional Social Pooling) and a decoder layer (Decoder) from left to right.

[0094] The encoder layer is used to provide motion features corresponding to the motion trajectory. For example, if the reference target is divided into a host vehicle and a surrounding vehicle, the historical trajectories of the host vehicle and the surrounding vehicle need to be input into an LSTM encoder unit for trajectory encoding to extract their respective historical motion features in the past period of time. LSTM (Long Short-Term Memory) is a special structure of recurrent neural network used to extract spatiotemporal sequence features. For example, in the present system, the LSTM parameters of each vehicle can be shared, and the number of hidden states is 128. After encoding, a 64-dimensional trajectory feature vector is output.

[0095] The convolutional pooling layer is used to interact and fuse the multiple relatively independent trajectory feature vectors generated by the encoder layer across agents. Specifically, the trajectory feature vectors of the surrounding vehicles will be filled into the corresponding virtual three-dimensional grid centered on the host vehicle according to their spatial distribution around the host vehicle, thereby forming a social information vector (Social Tensor) of the host vehicle. Subsequently, the social information vector will be sequentially processed by 3x3 convolution, 3x1 convolution, and 2x1 convolution max-pooling operations to further compress the features, and finally obtain a social context vector (social context). On the other hand, the trajectory feature vector obtained by encoding the motion trajectory of the host vehicle by the encoder layer will be directly processed by a fully connected neural network layer (fc 32) to obtain a trajectory feature vector (vehicle dynamics) containing the dynamics features of the host vehicle. Then, the trajectory feature vector of the host vehicle and the social context vector are concatenated to output a total trajectory encoding vector (Trajectory Encoding, TE) to the decoder module.

[0096] The decoder receives the TE vector output by the convolutional pooling layer, first calculates the lateral and longitudinal behavior classification prediction results of the host vehicle by two Softmax network layers, and then further encodes the behavior classification prediction results by one-hot category encoding. The encoded behavior classification prediction results are combined with the TE vector and input into an LSTM decoder unit for cyclic decoding in time sequence. The LSTM outputs a 5-dimensional vector at each time, which corresponds to 5 parameters of a two-dimensional Gaussian distribution, describing the motion trajectory of the host vehicle or the surrounding vehicles in the future period of time, i.e., the two-dimensional spatial distribution of the trajectory points generated at each time.

[0097] In combination with the above trajectory generation model, the motion trajectory generation method provided by the embodiments of the present application will be described in detail below. Figure 2 A flowchart of a motion trajectory generation method provided by an embodiment of the present application is shown in FIG. 2. As shown in FIG. 2, the motion trajectory generation method provided by the embodiment of the present application includes the following steps. Figure 2As shown, the generation method includes the following steps:

[0098] 301. Obtain historical motion trajectory data of at least one reference vehicle and a target vehicle.

[0099] When testing the test vehicle, first obtain the historical motion trajectory data of the test vehicle in a period of time, and the historical motion trajectory data of the surrounding vehicles around the test vehicle in the test scene. Since the test vehicle needs a complex test scene, the motion trajectory of the surrounding vehicles needs to be updated by using the generation method, so as to provide more driving adjustment for the test vehicle, so as to better detect the system safety of the test vehicle. Therefore, in this scene, the motion planning trajectory of each surrounding vehicle needs to be output one by one according to the trajectory generation model. It can be understood that the trajectory generation model can obtain the planning motion trajectory of the target vehicle according to the historical motion trajectory of the reference vehicle and the target vehicle, or obtain the planning motion trajectory of the surrounding vehicle according to the historical motion trajectory of the reference vehicle and the target vehicle. Only the output structure of the trajectory generation model needs to be changed. This is because the surrounding vehicles can be regarded as a "target vehicle", and the test vehicle and other surrounding vehicles can be regarded as "reference vehicles".

[0100] 302. Obtain the first historical trajectory feature vector of the target vehicle and the second historical trajectory feature vector corresponding to the reference vehicle by using the encoder layer.

[0101] Specifically, the encoder layer is used to extract spatio-temporal sequence features, and the historical motion trajectory data of the target vehicle and the reference vehicle need to be respectively input into an LSTM unit for trajectory encoding to extract their respective historical motion features in the past period of time. LSTM, i.e. long short-term memory network, is a special structure of recurrent neural network used to extract spatio-temporal sequence features. For example, in the present system, the LSTM parameters of each vehicle can be shared, and the number of hidden states is 128. After encoding processing, a 64-dimensional trajectory feature vector is respectively output.

[0102] 303. Input all the second historical trajectory feature vectors into the convolutional pooling layer, and use the convolutional pooling layer to perform feature fusion on all the second historical trajectory feature vectors to obtain the social information vector of the target vehicle.

[0103] The convolution pooling layer is used to fuse the multiple relatively independent track feature vectors generated by the encoder layer processing to realize cross-agent information interaction. Specifically, the second historical track feature vectors of the surrounding vehicles will be filled into the corresponding virtual three-dimensional grid centered on the target vehicle according to their spatial distribution around the target vehicle, thereby forming a social information vector of the target vehicle. Subsequently, the social information vector will be sequentially processed by 3x3 convolution, 3x1 convolution, and 2x1 convolution max-pooling operation to further compress the features, and finally obtain the compressed social information vector.

[0104] 304、The social information vector and the first historical track feature vector are spliced to obtain a track encoding vector.

[0105] On the other hand, the track feature vector obtained by the encoder layer encoding the host vehicle motion track will be directly processed by a fully connected neural network layer to obtain a track feature vector containing the host vehicle dynamics features. Then, the track feature vector of the host vehicle is spliced with the compressed social information vector to output a total track encoding vector to the decoder module.

[0106] 305、The track encoding vector is input into the decoder layer to obtain the motion planning track of the target vehicle.

[0107] The decoder layer receives the track encoding vector output by the convolution pooling layer, and first calculates the host vehicle lateral and longitudinal behavior classification prediction results through two Softmax network layers. The behavior classification prediction results are used to describe the motion planning track of the target vehicle in the future period of time, i.e., the two-dimensional spatial distribution of the track points generated at each time. It can be understood that changing the structure of the decoder layer can output the motion planning track of the surrounding vehicles in the future period of time. In this way, the simulation system can change the simulation environment according to the motion planning track of the surrounding vehicles in the future period of time to generate more complex simulation environments for testing.

[0108] In the embodiment of the present application, the trajectory generation model can be used to obtain the motion trajectory of the target vehicle. First, the historical motion trajectory data of the target vehicle and the reference vehicles around the target vehicle are obtained, and then the trajectory generation model is used to determine the social information vector corresponding to the target vehicle according to the historical motion trajectory data of all the reference vehicles. Then, the first historical trajectory feature vector corresponding to the target vehicle is determined according to the historical motion trajectory data of the target vehicle. Finally, the trajectory generation model is used to generate the motion planning trajectory of the target vehicle according to the social information vector corresponding to the target vehicle and the trajectory feature vector corresponding to the target vehicle. In this way, when the test vehicle is tested, the motion trajectories of the multiple traffic participants around the test vehicle can be quickly obtained according to the trajectory generation model. In this way, the motion states of the multiple traffic participants in the simulation test scene can be changed to efficiently and simply increase the complexity of the test scene. In this way, the problem of complex scene generation in the loop test system can be solved to perform more complete function tests on high-level autonomous vehicles.

[0109] The training process of the trajectory generation model will be briefly introduced below. Figure 3 The flowchart of a training method of a trajectory generation model provided by the embodiment of the present application is shown in FIG. 4. Figure 3 The training method includes the following steps:

[0110] 401. Obtain vehicle driving sample data.

[0111] The vehicle driving sample data includes the historical motion trajectory data of the first vehicle, the historical motion trajectory data of the second vehicle, and the current motion trajectory data of the second vehicle. It can be understood that the trajectory generation model needs to predict the future motion trajectory of the second vehicle according to the historical motion trajectory data of the first vehicle and the historical motion trajectory data of the second vehicle to obtain the motion planning trajectory of the second vehicle. Then, the obtained motion planning trajectory of the second vehicle needs to be compared with the current motion trajectory of the second vehicle to determine whether the prediction result of the trajectory generation model is accurate, and then the parameters of the trajectory generation model are adjusted to make the output result (prediction result) of the trajectory generation model more accurate.

[0112] 402. Input the historical motion trajectory data of the first vehicle and the historical motion trajectory data of the second vehicle into the trajectory generation model to obtain the motion trajectory prediction result of the second vehicle through the trajectory generation model.

[0113] The input of the trajectory generation model is the historical trajectory X = [x (t-h) ,…,x (t-1) ,x (t), t represents the time, t-1 represents the time before t, t-h represents the h time before t, x (t) represents the input historical position of the vehicle at time t, x (t-1) represents the input historical position of the vehicle at time t-1, x (t-h) represents the input historical position of the vehicle at time t-h; the output is the spatial probability distribution prediction of the trajectory points Y = [y (t +1) ,…,y (t+f) ] generated by the second vehicle in a future period of time, t+1 represents the time after t, t+f represents the f time after t, y （t+1） represents the predicted position of the second vehicle at time t+1, y （t+f） represents the predicted position of the second vehicle at time t+f, and the specific formula is:

[0114]

[0115] wherein, indicates the parameters of the predicted trajectory distribution, i is an integer, indicates the behavior category of the host vehicle.

[0116] 403. According to the motion trajectory prediction result and the current motion trajectory data, a loss value is determined.

[0117] For example, the target loss function of model training can be selected as the negative log maximum likelihood of the trajectory distribution generated by the second vehicle, that is:

[0118]

[0119] In order to make the loss function value described above steadily decrease in the training process, Adam algorithm can be selected as the gradient descent algorithm of model training.

[0120] 404. According to the loss value, the model parameters of the trajectory generation model are adjusted.

[0121] 405. When the training condition is reached, the training of the trajectory generation model is stopped.

[0122] Wherein, the training condition can be that the loss value is less than a preset threshold or the training times reach a preset number.

[0123] Figure 4 A structure diagram of a motion trajectory generation device provided by an embodiment of the present application is shown in FIG. 1, which comprises: Figure 4

[0124] ​The acquisition unit 501 is configured to acquire historical motion trajectory data of at least one reference vehicle and a target vehicle, the at least one reference vehicle being a surrounding vehicle of the target vehicle in a driving scene.

[0125] The determination unit 502 is configured to determine a social information vector corresponding to the target vehicle according to the historical motion trajectory data of all the reference vehicles by using a trajectory generation model.

[0126] The determination unit 502 is further configured to determine a first historical trajectory feature vector corresponding to the target vehicle according to the historical motion trajectory data of the target vehicle by using the trajectory generation model.

[0127] The generation unit 503 is configured to generate a motion planning trajectory of the target vehicle according to the social information vector corresponding to the target vehicle and the first historical trajectory feature vector corresponding to the target vehicle by using the trajectory generation model.

[0128] In an optional implementation, the trajectory generation model comprises an encoder layer configured to extract spatio-temporal sequence features. The generation apparatus further comprises an input-output unit 504.

[0129] The input-output unit 504 is configured to input the historical motion trajectory data of the at least one reference vehicle to the encoder layer.

[0130] The acquisition unit 501 is further configured to obtain a second historical trajectory feature vector corresponding to each reference vehicle through the encoder layer.

[0131] The input-output unit 504 is further configured to input the historical motion trajectory data of the target vehicle to the encoder layer.

[0132] The determination unit 502 is specifically configured to obtain the first historical trajectory feature vector corresponding to the target vehicle through the encoder layer.

[0133] In an optional implementation, the trajectory generation model comprises a convolutional pooling layer.

[0134] The input-output unit 504 is further configured to input all the second historical trajectory feature vectors to the convolutional pooling layer.

[0135] The determination unit 502 is specifically configured to perform feature fusion on all the second historical trajectory feature vectors by using the convolutional pooling layer to obtain the social information vector.

[0136] In an optional implementation, the determination unit 502 is further configured to perform pooling processing on the social information vector.

[0137] In an optional implementation, the trajectory generation model comprises a decoder layer.

[0138] The generating unit 503 is specifically configured to splice the social information vector after the pooling processing and the first historical trajectory feature vector to obtain a trajectory encoding vector.

[0139] The input and output unit 504 is further configured to input the trajectory encoding vector into the decoder layer.

[0140] The generating unit 503 is specifically configured to obtain a motion planning trajectory of the target vehicle. The motion planning trajectory of the target vehicle is used to describe the positions of the predicted trajectory points corresponding to each time point within a preset time period.

[0141] In an optional implementation, the generating apparatus further includes a training unit 505.

[0142] The training unit 505 is specifically configured to train the trajectory generation model. The training process of the trajectory generation model is as follows.

[0143] Obtain vehicle driving sample data, the vehicle driving sample data including historical motion trajectory data corresponding to a first vehicle, historical motion trajectory data corresponding to a second vehicle, and current motion trajectory data corresponding to the second vehicle.

[0144] Input the historical motion trajectory data corresponding to the first vehicle and the historical motion trajectory data corresponding to the second vehicle into the trajectory generation model, and obtain a motion trajectory prediction result corresponding to the second vehicle through the trajectory generation model.

[0145] Determine a loss value according to the motion trajectory prediction result and the current motion trajectory data.

[0146] Adjust model parameters of the trajectory generation model according to the loss value.

[0147] When a training condition is reached, stop the training process of the trajectory generation model.

[0148] In an optional implementation, the training condition is that the loss value is less than a preset threshold or the number of training times reaches a preset number.

[0149] In this embodiment, a trajectory generation model can be used to obtain the motion trajectory of the target vehicle. First, historical motion trajectory data of the target vehicle and surrounding reference vehicles are acquired. Then, the trajectory generation model is used to determine the social information vector corresponding to the target vehicle based on the historical motion trajectory data of all reference vehicles. Next, a first historical trajectory feature vector corresponding to the target vehicle is determined based on the historical motion trajectory data. Finally, the trajectory generation model is used to generate the target vehicle's planned motion trajectory based on the social information vector and the trajectory feature vector. Thus, when testing the vehicle under test, the motion trajectories of multiple traffic participants around the vehicle can be quickly obtained based on the trajectory generation model. This allows for changing the motion states of multiple traffic participants in the simulation test scenario, efficiently and easily increasing the complexity of the test scenario. This solves the problem of generating complex scenarios in a loop-in-the-loop testing system, enabling more comprehensive functional testing of high-level autonomous vehicles.

[0150] The following describes an electronic device provided by an embodiment of this application. Please refer to [link / reference]. Figure 5 , Figure 5 This is a schematic diagram of an electronic device provided in an embodiment of this application. The electronic device 800 can specifically be a virtual reality (VR) device, a mobile phone, a tablet, a laptop, a smart wearable device, a monitoring data processing device, or a radar data processing device, etc., and is not limited thereto. The electronic device 800 may be equipped with... Figure 4 The motion trajectory generation device described in the corresponding embodiment is used to implement Figures 1 to 3 The functions correspond to those in the embodiments. Specifically, the electronic device 800 includes: a receiver 801, a transmitter 802, a processor 803, and a memory 804 (wherein the number of processors 803 in the execution device 800 can be one or more). Figure 5 (Taking a processor as an example), the processor 803 may include an application processor 8031 ​​and a communication processor 8032. In some embodiments of this application, the receiver 801, transmitter 802, processor 803, and memory 804 may be connected via a bus or other means.

[0151] Memory 804 may include read-only memory and random access memory, and provides instructions and data to processor 803. A portion of memory 804 may also include non-volatile random access memory (NVRAM). Memory 804 stores processor and operation instructions, executable modules, or data structures, or subsets thereof, or extended sets thereof, wherein the operation instructions may include various operation instructions for implementing various operations.

[0152] The processor 803 controls the operation of the execution device. In a specific application, various components of the execution device are coupled together by a bus system, which can include a data bus, a power bus, a control bus, and a state signal bus, etc. However, for the sake of clarity, various buses are referred to as a bus system in the figure.

[0153] The method disclosed in the embodiments of the present application can be applied to the processor 803 or implemented by the processor 803. The processor 803 can be an integrated circuit chip with a signal processing capability. In the implementation process, the steps of various methods disclosed above can be completed by an integrated logic circuit of hardware in the processor 803 or an instruction in the form of software. The processor 803 described above can be a general-purpose processor, a digital signal processor (DSP), a microprocessor, or a microcontroller. The processor 803 can further include an application specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. The processor 803 can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of the present application. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor. The steps of the methods disclosed in conjunction with the embodiments of the present application can be directly embodied in hardware code of the processor, or a combination of hardware and software modules in the processor. The software module can be located in a storage medium such as random access memory (RAM), flash memory, read-only memory (ROM), programmable read-only memory (PROM), electrically programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), registers, or other mature storage media in the art. The storage medium is located in the memory 804, and the processor 803 reads information in the memory 804 and combines the hardware to complete the steps of the above methods.

[0154] The receiver 801 can be used to receive input digital or character information, and generate signal input related to the relevant settings and function control of the execution device. The transmitter 802 can be used to output digital or character information through the first interface; the transmitter 802 can also be used to send instructions to the disk group through the first interface to modify the data in the disk group; the transmitter 802 can also include a display device such as a display screen.

[0155] In the embodiments of the present application, the application processor 8031 in the processor 803 is configured to execute the method in the corresponding embodiments. It should be noted that the specific manner in which the application processor 8031 executes each step is the same as the method disclosed in the embodiments of the present application. Figures 1 to 3 Figures 1 to 3 ​Corresponding various method embodiments are based on the same concept, and the technical effects brought by the same are the same as the technical effects of the method embodiments of the present application Figures 1 to 3 Corresponding various method embodiments are the same, and specific contents can be referred to the description in the method embodiments shown in the foregoing of the present application, and will not be described here.

[0156] The present application also provides a computer readable storage medium, which includes computer instructions, and the computer instructions are used to implement the technical solution of any one of the motion trajectory generation methods in the embodiments of the present application when executed by a processor.

[0157] Those skilled in the art can understand that the drawings are only schematic diagrams of an embodiment, and the modules or flows in the drawings are not necessarily necessary for implementing the present application.

[0158] Those skilled in the art can understand that the modules in the device in the embodiments can be distributed in the device in the embodiments according to the embodiment description, or can be changed and located in one or more devices different from the present embodiment. The modules in the above embodiments can be combined as one module, or can be further split into multiple sub-modules.

[0159] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, and not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement for part of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A method of generating a motion trajectory, characterized by, The generation method is applied to a traffic flow generation module of an automatic driving test system, and the automatic driving test system further comprises a simulation module, a radar module, an injection device and an intelligent test bench; The traffic flow generation module is connected with an input end of the simulation module; an output end of the simulation module is connected with a tested vehicle through the injection device; an output end of the intelligent test bench is connected with an input end of the simulation module; the simulation module is used for generating a simulation scene about automatic driving according to a motion trajectory of a target vehicle transmitted by the traffic flow generation module and real-time motion data of the tested vehicle transmitted by the intelligent test bench; the injection device is used for injecting the simulation scene into the tested vehicle, so that the tested vehicle plans a motion state of the tested vehicle according to the simulation scene; the radar module is used for simulating a waveform reflected back by a radar after encountering an object in a physical waveform mode, so as to achieve a test purpose of a vehicle-mounted radar; and the intelligent test bench is used for perceiving and testing the motion state of the tested vehicle, so as to obtain the real-time motion data of the tested vehicle; The generation method comprises: acquiring historical motion trajectory data of at least one reference vehicle and a target vehicle; the at least one reference vehicle is a surrounding vehicle of the target vehicle in a driving scene; determining a social information vector corresponding to the target vehicle according to the historical motion trajectory data of all reference vehicles by using a trajectory generation model; the trajectory generation model comprises an encoder layer and a convolutional pooling layer; the encoder layer is used for extracting spatiotemporal sequence features; the determination of the social information vector corresponding to the target vehicle according to the historical motion trajectory data of all reference vehicles by using the trajectory generation model comprises: inputting the historical motion trajectory data of the at least one reference vehicle into the encoder layer; obtaining second historical trajectory feature vectors corresponding to the reference vehicles respectively through the encoder layer; inputting all second historical trajectory feature vectors into the convolutional pooling layer, and performing feature fusion on all second historical trajectory feature vectors by using the convolutional pooling layer to obtain the social information vector; determining a first historical trajectory feature vector corresponding to the target vehicle according to the historical motion trajectory data of the target vehicle by using the trajectory generation model; generating a motion planning trajectory of the target vehicle according to the social information vector corresponding to the target vehicle and the first historical trajectory feature vector corresponding to the target vehicle by using the trajectory generation model.

2. The generation method of claim 1, wherein, The determination of the first historical trajectory feature vector corresponding to the target vehicle according to the historical motion trajectory data of the target vehicle by using the trajectory generation model comprises: inputting the historical motion trajectory data of the target vehicle into the encoder layer; obtaining the first historical trajectory feature vector corresponding to the target vehicle through the encoder layer.

3. The generation method of claim 1, wherein, The method further comprises: performing pooling processing on the social information vector.

4. The generation method of claim 3, wherein, The trajectory generation model comprises a decoder layer; The generating, by the trajectory generation model, of the motion planning trajectory of the target vehicle from the social information vector corresponding to the target vehicle and the first historical trajectory feature vector corresponding to the target vehicle comprises: concatenating the social information vector and the first historical trajectory feature vector after the pooling processing to obtain a trajectory encoding vector; inputting the trajectory encoding vector into the decoder layer to obtain the motion planning trajectory of the target vehicle; the motion planning trajectory of the target vehicle is used to describe the positions of the predicted trajectory points corresponding to each time point within a preset time period.

5. The method of generating according to any one of claims 1 to 4, characterized in that, The method further comprises: training the trajectory generation model; wherein the training process of the trajectory generation model is as follows: obtaining vehicle driving sample data; the vehicle driving sample data comprises historical motion trajectory data of a first vehicle, historical motion trajectory data of a second vehicle, and current motion trajectory data of the second vehicle; inputting the historical motion trajectory data of the first vehicle and the historical motion trajectory data of the second vehicle into the trajectory generation model to obtain a motion trajectory prediction result of the second vehicle by the trajectory generation model; determining a loss value according to the motion trajectory prediction result and the current motion trajectory data; adjusting model parameters of the trajectory generation model according to the loss value; stopping the training process of the trajectory generation model when a training condition is reached.

6. The generation method of claim 5, wherein, The training condition is that the loss value is less than a preset threshold, or the number of training times reaches a preset number.

7. An apparatus for generating a motion trajectory, characterized by The generation device is applied to a traffic flow generation module of an automatic driving test system, and the automatic driving test system further comprises a simulation module, a radar module, an injection device, and an intelligent test bench. The traffic flow generation module is connected to an input end of the simulation module; an output end of the simulation module is connected to a tested vehicle through the injection device; an output end of the intelligent test bench is connected to an input end of the simulation module; the simulation module is used to generate a simulation scene related to automatic driving according to a motion trajectory of a target vehicle transmitted by the traffic flow generation module and real-time motion data of the tested vehicle transmitted by the intelligent test bench; the injection device is used to inject the simulation scene into the tested vehicle, so that the tested vehicle plans a motion state of the tested vehicle according to the simulation scene; the radar module is used to simulate a waveform reflected back by a radar after encountering an object in a physical waveform manner, so as to achieve a test purpose of a vehicle-mounted radar; the intelligent test bench is used to perceive and test the motion state of the tested vehicle to obtain the real-time motion data of the tested vehicle. The generation device comprises: an acquisition unit configured to acquire historical motion trajectory data of at least one reference vehicle and a target vehicle; the at least one reference vehicle is a surrounding vehicle of the target vehicle in a driving scene. The determination unit is configured to determine a social information vector corresponding to the target vehicle according to historical motion trajectory data of all reference vehicles by using a trajectory generation model; the trajectory generation model comprises an encoder layer and a convolutional pooling layer; the encoder layer is configured to extract spatiotemporal sequence features; the determination unit is specifically configured to: input the historical motion trajectory data of the at least one reference vehicle into the encoder layer; obtain second historical trajectory feature vectors corresponding to the reference vehicles respectively by the encoder layer; and input all the second historical trajectory feature vectors into the convolutional pooling layer, and perform feature fusion on all the second historical trajectory feature vectors by using the convolutional pooling layer to obtain the social information vector; The determination unit is also configured to determine a first historical trajectory feature vector corresponding to the target vehicle according to historical motion trajectory data of the target vehicle by using the trajectory generation model; The generation unit is configured to generate a motion planning trajectory of the target vehicle according to the social information vector corresponding to the target vehicle and the first historical trajectory feature vector corresponding to the target vehicle by using the trajectory generation model.

8. An electronic device, comprising: Comprise: A memory and a processor, the memory and the processor are coupled; The memory is configured to store one or more computer instructions; The processor is configured to execute the one or more computer instructions to implement the motion trajectory generation method according to any one of claims 1-6.

9. A computer-readable storage medium having stored thereon one or more computer instructions, wherein, The instructions are executed by the processor to implement the motion trajectory generation method according to any one of claims 1-6.

Images