An end-to-end vehicle trajectory prediction method based on machine vision

Abstract

The application discloses an end-to-end vehicle trajectory prediction method based on machine vision, comprising: 1. a feature extraction network, which extracts features of a plurality of monocular camera images by using a convolutional neural network; and 2. a motion prediction network, which predicts the motion trajectory of a surrounding vehicle at a future time by introducing two GRU models. The application can solve the problems of feature loss and error accumulation in the motion trajectory of the surrounding vehicle at the future time in a short time in automatic driving, thereby improving the prediction accuracy and guaranteeing the driving safety.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of automatic driving, in particular to the field of vehicle trajectory prediction, and provides an end-to-end vehicle trajectory prediction method based on machine vision. BACKGROUND

[0002] Current automatic driving technology routes are divided into two categories according to whether the perception module and the prediction module are separated. The first category separately designs the two modules and uses artificially selected features for communication between the two modules, such as predicting target trajectory, predicting target type, predicting target size, etc. However, this design of separating the perception and prediction modules has some problems. The prediction module can only receive part of the artificial information from the perception module and loses the information implied in the original sensor, such as tail lights and brake lights in driving prediction, and head and body posture in pedestrian prediction. And worse, errors from the perception module can be propagated and accumulated, thus adversely affecting the prediction results. Specifically, the historical trajectory used by the prediction module comes from its upstream perception module, and the perception module inevitably has errors, thus leading to a decline in prediction performance. At present, it is a relatively difficult task to design a trajectory prediction module that is robust to the output error of the upstream perception module. The advantage of the traditional modular design scheme is low coupling and high interpretability, but the optimality is often lost. SUMMARY

[0003] The present application is to solve the above-mentioned problems of the prior art, and provides an end-to-end vehicle trajectory prediction method based on machine vision, which aims to solve the problems of feature loss and error accumulation in the short-term future trajectory of surrounding vehicles in automatic driving, and improve the prediction accuracy, so as to ensure driving safety.

[0004] To achieve the above-mentioned application purposes, the technical scheme adopted by the present application is as follows:

[0005] The end-to-end vehicle trajectory prediction method based on machine vision has the characteristics of the following steps:

[0006] Step 1, data acquisition:

[0007] K monocular cameras are used to capture a frame of image data {O t k |k=1,2,…,K} of the current time t during vehicle driving, wherein O t k represents the frame of image data captured by the kth monocular camera at the current time t;

[0008] Extracting the position difference a of the previous frame of vehicle position relative to the current time t of the vehicle position during the vehicle driving process by using the CAN bus of the vehicle t ;

[0009] Step 2, building a feature extraction network based on a convolutional neural network, including: an encoder network, a feature projection and space conversion network, and a time sequence network;

[0010] Step 2.1, the encoder network is based on an Efficientnet-b4 network and sequentially includes eight convolutional blocks and an output block; wherein the output block is composed of an average pooling layer and a fully connected layer;

[0011] Inputting the image data {O t-i k |i=0,1,2,…,I} containing the current time t into the encoder network for processing, thereby obtaining the feature set {f t-i k |i=0,1,2,…,I} of the image data of the kth monocular camera; wherein O t-i k represents the image data of the kth monocular camera at time t-i, f t-i k represents the image data feature of O t-i k ;

[0012] Separating f t-i k in the channel dimension to obtain feature tensor and depth tensor and obtaining the outer product of the two to obtain the encoding feature of O t-i k ; thereby obtaining the encoding feature set {u t-i k |i=0,1,2,…,I};

[0013] Step 2.2, the feature projection and space conversion network processes the encoding feature set {u t-i k |i=0,1,2,…,I} to obtain the space conversion feature set {y t-i |i=0,1,2,…,I}, wherein y t-i represents the space conversion feature at time t-i;

[0014] Step 2.3, the time sequence network is based on a ResNet network and sequentially includes two three-dimensional convolutional layers, a pyramid pooling layer, and an output layer, wherein the output layer is composed of a three-dimensional convolutional layer;

[0015] each spatial transformation feature in the spatial transformation feature set {y t-i | i = 0, 1, 2, …, I} is sequentially processed by two three-dimensional convolution layers and a pyramid pooling layer, and the intermediate feature generated by each layer is spliced and input into the output layer, and the output layer adds the input and the output through a jump connection to finally obtain the spatio-temporal information feature set {s t-i | i = 0, 1, 2, …, I}, wherein s t-i represents the spatio-temporal information feature at time t-i.

[0016] Step 3, constructing a motion prediction network, including: an uncertainty modeling network, a two-way prediction model network and a decoder network;

[0017] Step 3.1, the uncertainty modeling network sequentially consists of n residual blocks, an average pooling layer and an output layer;

[0018] the spatio-temporal information feature s t at the current time t is input into the uncertainty modeling network and sequentially processed by n residual blocks and an average pooling layer to obtain the hidden state h t at the current time t.

[0019] the hidden state h t is input into an output layer composed of a two-dimensional convolution for processing to obtain the mean μ t and the variance σ t of the Gaussian distribution at the current time t. 2 , and sampling the normal distribution N(μ t , σ t 2 at time t to finally obtain the uncertainty state η t at the current time t.

[0020] Step 3.2, the two-way prediction model network consists of two GRU networks and a residual network, wherein the first GRU network consists of H GRU units, and the second GRU network consists of t+H GRU units.

[0021] When h = 1, the spatio-temporal information feature s t-I at time t-I is taken as the initial hidden state and input into the hth GRU unit of the first GRU network for processing, and the intermediate state p t+h 1 of the hth GRU unit of the first GRU network is output.

[0022] When h = 1, the spatio-temporal information feature s tas the hidden state of the h-th GRU unit of the first GRU network; and the predicted future state q t is input into the h-th GRU unit of the second GRU network for processing, and an intermediate state p t+h 2 ;

[0023] When h = 1, p t+h 1 is input into the residual network for processing, and a corresponding output p t+h 2 is obtained. t+h 1 is input into the residual network for processing, and a corresponding output p t+h 2 is obtained. t+h 1 is weighted processed according to the two weights, to obtain a predicted future state q t+h 2 . t+h ;

[0024] When h = 2, 3, …, H, the spatiotemporal information feature s t-I is input into the h-th GRU unit of the first GRU network for processing, and an intermediate state p t+h-1 is output. t+h 1 ;

[0025] When h = 2, 3, …, H, the predicted future state q t+h-1 is input into the h-th GRU unit of the second GRU network for processing, and an intermediate state p t is output. t+h 2 ;

[0026] When h = 2, 3, …, H, p t+h 1 is input into the residual network for processing, and a corresponding output p t+h 2 is obtained. t+h 1 is input into the residual network for processing, and a corresponding output p t+h 2 is obtained. t+h 1 is weighted processed according to the two weights, to obtain a predicted future state q t+h2 The weighted processing is performed to obtain the predicted future state q t+h ; so as to recursively predict the future state set {q t+i |i=1, 2,..., H} from t+1 to t+H.

[0027] Step 3.2, the decoder network is based on a ResNet-18 network, which is alternately composed of three residual blocks and three up-sampling layers in turn, and an output layer composed of two-dimensional convolution is connected after the third up-sampling layer;

[0028] The future state set {q t+i |i=1, 2,..., H} from t+1 to t+H is sent into the decoder network for processing to obtain the predicted vehicle motion trajectory result set {w t+i |i=1, 2,..., H}, wherein w t+i represents the position of the vehicle at t+i.

[0029] Step 4, a loss function is constructed with the average absolute error between the vehicle motion trajectory result set {w t+i |i=1, 2,..., H} and the real trajectory data as the loss; and the feature extraction and motion prediction network are trained through the Adam optimizer, and the loss function is constantly optimized to adjust the network parameters, so as to obtain a vehicle trajectory predictor for realizing the trajectory prediction result of the surrounding vehicle.

[0030] The feature projection and space conversion network in step 2.2 of the end-to-end vehicle trajectory prediction method based on machine vision also has the characteristics that the feature projection and space conversion network comprises a feature projection network and a space conversion network.

[0031] Step 2.2.1, the feature projection network projects the encoded feature x in the vertical direction using the sum pooling method to obtain the bird's eye view feature x t-i at t-i in the top view, so as to obtain the bird's eye view feature set {x t-i |i=0, 1, 2,..., I}.

[0032] Step 2.2.2, the space conversion network uses rotation and translation to transform {x t-i |i=0, 1, 2,..., I} to the vehicle body coordinate system at the current time t according to the position difference {a t-i |i=0, 1, 2,..., I}, so as to obtain the space conversion feature set {y t-i |i=0, 1, 2,..., I}.

[0033] The electronic device comprises a memory and a processor, and the memory is used for storing a program supporting the processor to execute the end-to-end vehicle trajectory prediction method, and the processor is configured to execute the program stored in the memory.

[0034] The computer readable storage medium stores a computer program, and when the computer program is run by a processor, the steps of the end-to-end vehicle trajectory prediction method are executed.

[0035] Compared with the prior art, the beneficial effects of the present application are reflected in the following aspects:

[0036] 1. The method of the present application uses the original image as input and utilizes the rich visual information therein for trajectory prediction. Compared with traditional trajectory prediction schemes, the method can avoid the problems of feature loss and error accumulation. This method can directly utilize the information of the original image for prediction without the need for additional feature extraction and conversion, and improves the accuracy and robustness of trajectory prediction.

[0037] 2. The method of the present application designs a double-path prediction model that explicitly considers past motion changes and infers the probabilistic nature of past and future motion by introducing an additional temporal model with a fusion unit. This model can effectively combine historical motion information and future motion trends to more accurately predict the trajectory of the target object. BRIEF DESCRIPTION OF DRAWINGS

[0038] Figure 1 A flowchart of an end-to-end vehicle trajectory prediction method based on machine vision is disclosed for the embodiments of the present application.

[0039] Figure 2 An encoder flowchart is disclosed.

[0040] Figure 3 A structure diagram of a double-path prediction model in motion prediction is disclosed. DETAILED DESCRIPTION

[0041] In this embodiment, for the specific needs of the autonomous driving system in the dynamic driving scene, an end-to-end vehicle trajectory prediction method based on machine vision is proposed, which includes: first, the feature extraction module uses a convolutional neural network to predict the depth probability distribution of monocular camera images, and completes the feature extraction task by designing a time series model. In addition, in order to improve the accuracy of trajectory prediction, a double-channel prediction model is designed in the motion prediction module to explicitly consider the past motion changes, and two GRU models are introduced to infer the probabilistic properties of past and future motion. Finally, a large number of experimental results on the NuScenes dataset show that this method is superior to traditional prediction schemes and previous end-to-end models, and can meet the trajectory prediction task in different driving environments, and has high application value. Specifically, as shown in Figure 1 the following steps are included:

[0042] Step 1, data acquisition:

[0043] K=6 monocular cameras are used to capture a frame of image data {O t k |k=1,2,…,6} of the vehicle at the current time t during driving, where O t k represents the frame of image data captured by the kth monocular camera at the current time t;

[0044] The position difference a t between the previous frame of vehicle position and the current time t during vehicle driving is extracted from the vehicle CAN bus;

[0045] Step 2, build a feature extraction network based on a convolutional neural network, including: an encoder network, a feature projection and spatial conversion network, and a time series network;

[0046] Step 2.1, the encoder network is based on the Efficientnet-b4 network, the structure of which is as shown in Figure 2 and sequentially includes eight convolutional blocks and an output block; wherein the output block is composed of an average pooling layer and a fully connected layer;

[0047] The first four frames of image data {O t-i k |i=0,1,2,3} containing the current time t are input into the encoder network for processing, thereby obtaining the feature set {f t-i k |i=0,1,2,3} of the image data captured by the kth monocular camera; wherein O t-i k represents a frame of image data captured by the kth monocular camera at time t-i, and f t-i k represents Ot-i k Image data characteristics:

[0048]

[0049] Where C is the number of feature channels, D is the number of discrete depths, and H... e With W e The feature space size is given. Each pixel is discretized into D segments corresponding to a ray, and each segment corresponds to a probability distribution α. i (i∈[0,D-1]), thus allowing each pixel to obtain a predicted depth over a series of discrete depth values ​​D. D is equal to the minimum depth value (Di). min =2m) and maximum depth (D max The number of equally spaced depth slices between 50m and 60m, and the slice distance D. size Set to 1 meter. Since the output stride of the backbone network being used is 8, He = 224 / 8 = 28, We = 480 / 8 = 60.

[0050] f t-i k After separation along the channel dimension, the feature tensor is obtained. and depth tensor Their dimensions are C×H respectively. e ×W e and D×H e ×W. Taking the outer product of the two tensors yields O. t-i k coding features Thus, the encoded feature set {u} is obtained. t-i k |i=0,1,2,4}:

[0051]

[0052] Step 2.2, Feature Projection and Spatial Transformation Network, including: Feature Projection Network and Spatial Transformation Network;

[0053] Step 2.2.1: Since the perspectives of multiple cameras are different, it is necessary to encode the feature set {u}. t-i k The parameters |i=0,1,2,3} are captured in the same top-view image using the camera's intrinsic and extrinsic parameters. The feature projection network encodes features based on the monocular camera's intrinsic and extrinsic parameters. Projecting the top view onto the vertical plane using sum-pooling, we obtain the bird's-eye view features x at time ti. t-i Thus, the feature set {x} of the bird's-eye view is obtained. t-i |i=0,1,2,4};

[0054] Step 2.2.2, since the coordinate systems of multi-frame images are different, it is necessary to convert the bird's eye view feature set {x t-i |i=0,1,2,4} into the same coordinate system. The spatial conversion network uses rotation and translation to transform {x t-i |i=0,1,2,4} into the vehicle body coordinate system at the current time t, thereby obtaining the spatial conversion feature set {y t-i |i=0,1,2,4}, wherein y t-i |i=0,1,2,4} represents the spatial conversion feature at time t-i. t-i ;

[0055] Step 2.3, the time sequence network is based on the ResNet network, and sequentially includes two three-dimensional convolution layers, a pyramid pooling layer and an output layer, wherein the convolution kernel sizes of the two three-dimensional convolution layers are (2, 3, 3) and (1, 3, 3) respectively, and the output layer is composed of a three-dimensional convolution layer with a convolution kernel size of (1, 1, 1);

[0056] Each spatial conversion feature in the spatial conversion feature set {y t-i |i=0,1,2,4} is sequentially processed by two three-dimensional convolution layers and a pyramid pooling layer, and after the intermediate feature generated by each layer is spliced, it is input into the output layer. The output layer adds the input and the output through the jump connection mode, and finally obtains the space-time information feature set {s t-i |i=0,1,2,4}, wherein s t-i represents the space-time information feature at time t-i.

[0057] Step 3, constructing a motion prediction network, including an uncertainty modeling network, a double-path prediction model network and a decoder network;

[0058] Step 3.1, the uncertainty modeling network is sequentially composed of 3 residual blocks, 1 average pooling layer and an output layer;

[0059] The space-time information feature s t at the current time t is input into the uncertainty modeling network, and after being processed by n residual blocks and an average pooling layer, the hidden state h t at the current time t is obtained.

[0060] The hidden state h t is input into an output layer composed of a two-dimensional convolution for processing, and the mean μ t and variance σ t of the Gaussian distribution at the current time t are obtained. 2 t ,σt 2 ) sampling at time t, finally obtaining the uncertainty state ηt of the current time t t ;

[0061] Step 3.2, in order to improve the accuracy of future prediction, it is necessary to more explicitly consider the past motion changes, so the study introduces a gated recurrent unit model for reasoning the probabilistic properties of past and future motion, corresponding to two GRU models in the two-way prediction model respectively. The two-way prediction model network is composed of two GRU networks and a residual network, wherein the first GRU network is composed of H GRU units, and the second GRU network is composed of t+H GRU units; specifically, as shown in Figure 3 , the steps include:

[0062] When h = 1, the spatiotemporal information feature s t-I at time t-I is taken as the initial hidden state and input into the hth GRU unit of the first GRU network for processing, and the intermediate state p t+h 1 ;

[0063] When h = 1, the spatiotemporal information feature s t at time t is taken as the initial hidden state, and the uncertainty state η t at time t is input into the hth GRU unit of the second GRU network for processing, and the intermediate state p t+h 2 ;

[0064] When h = 1, p t+h 1 and p t+h 2 are input into the residual network for processing, and the corresponding outputs p t+h 1 and p t+h 2 are obtained; the weights of p t+h 1 and p t+h 2 are obtained, so that p t+h ;

[0065] When h = 2, 3, …, H, the spatiotemporal information feature s t-I at time t-I is taken as the hidden state of the hth GRU unit of the first GRU network; the future state q t+h-1The input is processed in the hth GRU unit of the first GRU network, and the intermediate state p of the hth GRU unit of the first GRU network is output t+h 1 ;

[0066] When h = 2, 3, …, H, the future state q t+h-1 is obtained by inputting the uncertainty state η t of the current time t into the hth GRU unit of the second GRU network as the hidden state of the hth GRU unit of the second GRU network, and the intermediate state p of the hth GRU unit of the second GRU network is output t+h 2 ;

[0067] When h = 2, 3, …, H, the future state q t+h 1 is obtained by inputting p t+h 2 into the residual network for processing, and the corresponding output p t+h 1 is obtained. t+h 2 The weights of p t+h 1 and p t+h 2 are weighted to obtain the predicted future state q t+h at time t+h; thus, the future state set {q t+i |i = 1, 2, …, H} from time t+1 to t+H is recursively predicted.

[0068] Step 3.2, the decoder network is based on a ResNet-18 network, which is composed of three residual blocks with a convolution kernel of 3 and three up-sampling layers with a sampling factor of 2 in turn, and an output layer composed of two-dimensional convolution is connected after the third up-sampling layer.

[0069] The future state set {q t+i |i = 1, 2, …, H} from time t+1 to t+H is input into the decoder network for processing, and the predicted vehicle motion trajectory result set {w t+i |i = 1, 2, …, H} is obtained, wherein w t+i represents the position of the vehicle at time t+i.

[0070] Step 4, a vehicle motion trajectory result set {w t+iThe average absolute error between the predicted trajectory and the real trajectory data is a loss function of the loss function; and the feature extraction and motion prediction network are trained by an Adam optimizer, and the loss function is continuously optimized to adjust the network parameters, so as to obtain a vehicle trajectory predictor for realizing the trajectory prediction result of the surrounding vehicle.

[0071] In the embodiment, an electronic device includes a memory for storing a program supporting a processor to execute the above method, and the processor is configured to execute the program stored in the memory.

[0072] In the embodiment, a computer readable storage medium stores a computer program on the computer readable storage medium, and the computer program is run by a processor to execute the steps of the above method.

[0073] In the embodiment, the working process of the end-to-end vehicle trajectory prediction method based on machine vision is as follows:

[0074] Step a: using Efficientnet-b4 as the encoder E of the backbone network at each time step, so as to extract image features and predict the probability distribution of depth in each camera;

[0075] Step b: after the encoder, the generated three-dimensional information is projected onto the same bird's eye view plane according to the internal and external parameters of the camera to obtain the BEV feature. In order to fuse the time sequence information of multiple frames, the spatial alignment of the front and rear frames is needed by using the ego motion trajectory, so as to convert the previous BEV feature into the current frame;

[0076] Step c: all BEV features are sent into a convolutional neural network to obtain features with spatio-temporal information.

[0077] Step d: a Gaussian distribution is used to represent the probability distribution, and the Gaussian distribution is sampled.

[0078] Step e: the sampling result is input into a two-way GRU prediction model together with the features extracted from the encoder, so as to obtain the predicted state of the future frame;

[0079] Step f: a designed decoder is used to reconstruct the segmentation image and the predicted future motion trajectory from the predicted state, so as to realize the prediction of the future behavior of the vehicle.

Claims

1. A machine vision based end-to-end vehicle trajectory prediction method, characterized in that The steps are as follows: Step 1, data collection: K monocular cameras are used to capture a frame of image data {O t k |k=1,2,…,K} at a current time t, where O t k represents a frame of image data captured by the kth monocular camera at the current time t. extracting a position difference a of a previous frame of vehicle position relative to a current time t of the vehicle position during vehicle driving by using a CAN bus of the vehicle t ; Step 2, build a feature extraction network based on convolutional neural network, including: encoder network, feature projection and spatial conversion network, time series network; Step 2.1, the encoder network is based on Efficientnet-b4 network, and includes eight convolutional blocks and an output block in turn; Wherein, the output block is composed of an average pooling layer and a fully connected layer; image data {O t-i k | i = 0, 1, 2, …, I} are input into the encoder network for processing, thereby obtaining a feature set {f t-i k | i = 0, 1, 2, …, I}; wherein, O t-i k represents one frame of image data captured by the kth monocular camera at time t-i, f t-i k represents the image data feature of O t-i k . f t-i k After separation in the channel dimension, the feature tensor is obtained and the depth tensor After the outer product of the two, O t-i k The encoded features Thus, the encoded feature set {u t-i k |i=0,1,2,…,I} is obtained; Step 2.2, the feature projection and space conversion network processes the encoded feature set {u t-i k | i = 0, 1, 2, …, I} into a space conversion feature set {y t-i | i = 0, 1, 2, …, I}, wherein y t-i represents the space conversion feature at time t-i. Step 2.3, the time series network is based on ResNet network, including two three-dimensional convolution layers, a pyramid pooling layer and an output layer in turn, wherein the output layer is composed of a three-dimensional convolution layer; each of the spatial conversion feature set {y t-i Each of the spatial conversion features in the set {y t-i s t-i represents the space-time information feature at time t-i. Step 3, construct a motion prediction network, including: uncertainty modeling network, double path prediction model network and decoder network; Step 3.1, the uncertainty modeling network is composed of n residual blocks, an average pooling layer and an output layer in turn; spatiotemporal information feature s of the current time t is obtained t The hidden state h of the current time t is obtained after the uncertainty modeling network is input and sequentially processed by n residual blocks and an average pooling layer t ; The hidden state h t After processing the input into an output layer consisting of two-dimensional convolutions, the mean μ of the Gaussian distribution at the current time t is obtained. t and variance σ t 2 And for the normal distribution N(μ) t ,σ t 2 By sampling at time t, the uncertainty state η at the current time t is finally obtained. t ; Step 3.2, the double path prediction model network is composed of two GRU networks and a residual network, wherein the first GRU network is composed of H GRU units, and the second GRU network is composed of t+H GRU units; When h = 1, the space-time information feature s at the t-1 moment t-I As the initial hidden state and input into the hth GRU unit of the first GRU network for processing, the intermediate state p of the hth GRU unit of the first GRU network is output t+h 1 ; When h = 1, the space-time information feature s of the current time t is taken as the initial hidden state of the first GRU network, and the uncertainty state η of the current time t is taken as the initial hidden state of the second GRU network. t As the initial hidden state, and the uncertainty state η of the current time t is taken as the initial hidden state of the second GRU network. t The input is processed in the hth GRU unit of the second GRU network, and the intermediate state p of the hth GRU unit of the second GRU network is output. t+h 2 ; When h = 1, p t+h 1 and p t+h 2 into the residual network for processing, and a corresponding output p t+h 1 and p t+h 2 are weighted according to the two weights, so that p t+h 1 and p t+h 2 are weighted and processed to obtain the predicted future state q t+h ; When h = 2, 3, …, H, the space-time information feature s of the t-I moment t-I The hidden state of the hth GRU unit of the first GRU network as the first GRU network predicts the future state q at the t+h-1 moment t+h-1 The input is processed in the hth GRU unit of the first GRU network, and the intermediate state p of the hth GRU unit of the first GRU network is output t+h 1 ; When h = 2, 3, …, H, the future state q at time t+h-1 is predicted t+h-1 As the hidden state of the hth GRU unit of the second GRU network, the uncertainty state η of the current time t is input t The current time t is input into the hth GRU unit of the second GRU network for processing, and the intermediate state p of the hth GRU unit of the second GRU network is output t+h 2 ; When h = 2, 3, …, H, p t+h 1 With p t+h 2 into the residual network for processing, and the corresponding output p t+h 1 With p t+h 2 weight, so that p t+h 1 With p t+h 2 weighted processing, to obtain the predicted future state q t+h ; so as to recursively predict the future state set {q t+i |i = 1, 2, …, H} from t + 1 to t + H. Step 3.2, the decoder network is based on ResNet-18 network, which is composed of three residual blocks and three up-sampling layers alternately, and an output layer composed of two-dimensional convolution is connected after the third up-sampling layer; a future state set {q t+i |i = 1, 2, …, H} into the decoder network for processing to obtain a predicted vehicle motion trajectory result set {w t+i |i = 1, 2, …, H}, wherein w t+i represents the position of the vehicle at time t+i. Step 4, constructing a vehicle motion trajectory result set {w t+i The average absolute error between the feature extraction and motion prediction network and the true trajectory data is the loss function of the loss function; and the feature extraction and motion prediction network is trained by the Adam optimizer, and the loss function is continuously optimized to adjust the network parameters, so as to obtain a vehicle trajectory predictor for realizing the trajectory prediction result of the surrounding vehicle.

2. The end-to-end vehicle trajectory prediction method based on machine vision according to claim 1, characterized in that, The feature projection and spatial conversion network in step 2.2 includes: feature projection network and spatial conversion network; Step 2.2.1, the feature projection network projects the encoded features In the vertical direction, the sum pooling method is used to project in the top view to obtain the bird's eye view feature x t-i , thereby obtaining a set of bird's eye view features {x t-i |i = 0, 1, 2, …, I}. Step 2.2.2: The spatial transformation network uses the position difference {a} from the previous I-frame... t-i |i=0,1,2,…,I} use rotation and translation to transform {x t-i The coordinates of |i=0,1,2,…,I} are transformed into the vehicle coordinate system at the current time t, thus obtaining the spatial transformation feature set {y t-i |i=0,1,2,…,I}.

3. An electronic device comprising a memory and a processor, characterized in that The memory is used to store the program supporting the processor to execute the end-to-end vehicle trajectory prediction method of claim 1 or 2, and the processor is configured to execute the program stored in the memory.

4. A computer-readable storage medium having stored thereon a computer program, characterized in that The computer program is executed by the processor to execute the steps of the end-to-end vehicle trajectory prediction method of claim 1 or 2.

Images