Train trajectory prediction method based on recurrent convolutional neural network

Abstract

This invention discloses a method for vehicle behavior recognition and trajectory prediction based on a recurrent convolutional neural network, belonging to the field of intelligent driving. It includes the following steps: Step 1: Collect vehicle-related data for the autonomous vehicle's driving environment; Step 2: Based on the collected dataset, model vehicle behavior, road environment, and pedestrian / cyclist behavior, constructing a vector-representation-based human-vehicle-road coupling relationship model; Step 3: Based on the human-vehicle-road coupling relationship model, use a long short-term memory network to perform behavior pattern recognition on the target vehicle; Step 4: Combining the human-vehicle-road coupling relationship model and the vehicle behavior recognition results, use a convolutional neural network to predict the target vehicle's trajectory. Compared with existing technologies, the positive effects of this invention are: This invention effectively improves the accuracy of autonomous vehicles in recognizing the behavior of surrounding vehicles and predicting their trajectories, enhancing the efficiency and safety of autonomous vehicle operation.

Description

Technical Field

[0001] This invention relates to the field of intelligent driving, and in particular to a method for vehicle behavior recognition and trajectory prediction based on a recurrent convolutional neural network. Background Technology

[0002] For autonomous vehicles, safety is paramount. To detect potential hazards on the road early, a certain level of awareness of future changes in the surrounding road environment is required—that is, a strong predictive ability. Therefore, vehicle trajectory prediction is a significant and worthwhile technical challenge. In particular, the ability of autonomous vehicles to predict the behavior of surrounding vehicles in advance has a crucial impact on their driving safety and efficiency. How to accurately and reliably predict the future trajectories of surrounding vehicles has become one of the key issues in autonomous vehicle research.

[0003] For example, if another vehicle suddenly cuts in front of an autonomous vehicle, the autonomous vehicle can only avoid or mitigate the harm by emergency braking; otherwise, the risk of a collision is very high. If an autonomous vehicle can predict the future trajectories of surrounding vehicles, it can smoothly decelerate in advance to avoid a collision while ensuring the comfort of the occupants. Therefore, predicting the future trajectories of surrounding vehicles can improve the safety of autonomous vehicles. Summary of the Invention

[0004] The purpose of this invention is to overcome the shortcomings or defects of the existing technology, so as to design a vehicle behavior recognition and trajectory prediction method based on recurrent convolutional neural networks under urban congestion conditions and considering the actual situation of road and traffic flow constraints, thereby improving the safety and efficiency of autonomous vehicle driving.

[0005] To achieve the above objectives, this invention provides a method for vehicle behavior recognition and trajectory prediction based on a recurrent convolutional neural network, comprising the following steps:

[0006] Step 1: Collect relevant vehicle data based on the driving environment of the autonomous vehicle;

[0007] Step 2: Based on the collected dataset, model vehicle behavior, road environment, and pedestrian / cyclist behavior, and construct a vector-representation-based human-vehicle-road coupling relationship model;

[0008] Step 3: Based on the human-vehicle-road coupling relationship model, use a long short-term memory network to identify the behavior patterns of the target vehicle;

[0009] Step 4: Combining the human-vehicle-road coupling relationship model and vehicle behavior recognition results, the trajectory prediction of the target vehicle is achieved based on a convolutional neural network.

[0010] Preferably, step 2 includes:

[0011] Vehicle behavior includes vehicle trajectory information; road environment includes lane line positions, zebra crossing positions, and other information; pedestrian / cyclist behavior includes pedestrian / cyclist trajectory information.

[0012] Using a vector representation method, different vector nodes are established for vehicle behavior, road environment, and pedestrian / cyclist behavior according to different categories. Based on graph neural networks, a node subgraph is established for each category, and then a global interaction graph between different categories is established to obtain the vectorized interaction relationships between vehicles and roads, vehicles and people, and vehicles and vehicles, and to construct a human-vehicle-road coupling relationship model based on vector representation.

[0013] Preferably, step 3 includes:

[0014] Vehicle behavior patterns include going straight, turning left, turning right, changing lanes to the left, and changing lanes to the right.

[0015] Based on the target vehicle state information and combined with the global interaction feature information of the human-vehicle-road coupling relationship model, a multi-layer long short-term memory network structure is adopted, and only the last node is output as the classification result, that is, the recognition results of 5 behavior patterns.

[0016] Preferably, step 4 includes:

[0017] Convolutional neural networks and multilayer perceptrons are used to predict the long-term multimodal trajectory of the target vehicle, listing all possible predicted trajectories and selecting the most likely predicted trajectory.

[0018] This invention takes autonomous vehicles as the research object, considers the actual situation of urban traffic congestion and road and traffic flow constraints, and performs long-term multimodal trajectory prediction of vehicles around (target) autonomous vehicles based on the understanding of the urban road environment.

[0019] In summary, this invention proposes a novel scheme that integrates multiple existing models to improve the accuracy of autonomous vehicles in recognizing the behavior of surrounding vehicles and predicting their trajectories. Specifically, for vehicle behavior pattern recognition, based on the existing method of using an LSTM model for recognition, a human-vehicle-road coupling relationship model (the formula of this model is also existing technology) is integrated. Therefore, it can better identify vehicle behavior, that is, accurately identify five behavior patterns: straight driving, lane changing, and turning. For vehicle trajectory prediction, the method first uses the results of vehicle behavior pattern recognition, then integrates the results of the human-vehicle-road coupling relationship model, and finally uses a CNN model + multilayer perceptron (existing methods use a CNN model for trajectory prediction) to perform long-term multimodal trajectory prediction of the target vehicle, thereby improving the accuracy of trajectory prediction.

[0020] Compared with the prior art, the present invention has the following technical advantages:

[0021] 1) This invention proposes a method for vehicle behavior recognition and trajectory prediction based on recurrent convolutional neural networks, which effectively improves the accuracy of autonomous vehicles in recognizing the behavior of surrounding vehicles and predicting their trajectories, thereby enhancing the safety and efficiency of autonomous vehicles.

[0022] 2) This invention establishes a vehicle behavior pattern recognition model based on long short-term memory network. Based on the human-vehicle-road coupling relationship model, the long short-term memory network is used to recognize the behavior patterns of target vehicles, including five behavior patterns: straight driving, changing lanes left and right, and turning left and right, which improves the accuracy of behavior pattern recognition.

[0023] 3) This invention establishes a vehicle trajectory prediction model based on convolutional neural networks and multilayer perceptrons. Based on the human-vehicle-road coupling relationship model and combined with the behavior pattern recognition results, the convolutional neural network and multilayer perceptron are used to perform long-term multimodal trajectory prediction of the target vehicle, which reduces the offset error of vehicle trajectory prediction and improves the prediction accuracy. Attached Figure Description

[0024] The specific embodiments of the present invention will now be described in detail with reference to the accompanying drawings:

[0025] Figure 1 This is a flowchart illustrating a method for vehicle behavior recognition and trajectory prediction based on a recurrent convolutional neural network.

[0026] Figure 2 This is a schematic diagram of a vector-based human-vehicle-road coupling relationship model.

[0027] Figure 3 This is a schematic diagram of a vehicle behavior pattern recognition model based on a long short-term memory network.

[0028] Figure 4 This is a schematic diagram of a vehicle trajectory prediction model based on convolutional neural networks and multilayer perceptrons. Detailed Implementation

[0029] To further illustrate the features of the present invention, please refer to the following detailed description and accompanying drawings. The drawings are for reference and illustration only and are not intended to limit the scope of protection of the present invention.

[0030] like Figure 1 As shown, this embodiment discloses a method for vehicle behavior recognition and trajectory prediction based on a recurrent convolutional neural network, including the following steps S1 to S4:

[0031] S1. Collect vehicle-related data based on the driving environment of autonomous vehicles;

[0032] S2. Based on the collected dataset, model vehicle behavior, road environment, and pedestrian / cyclist behavior, and construct a human-vehicle-road coupling relationship model based on vector representation;

[0033] S3. Based on the human-vehicle-road coupling relationship model, use long short-term memory network to identify the behavior pattern of the target vehicle;

[0034] S4. Combining the human-vehicle-road coupling relationship model and vehicle behavior recognition results, the trajectory prediction of the target vehicle is achieved based on a convolutional neural network.

[0035] Preferably, step S1 includes:

[0036] By utilizing various onboard sensors, such as cameras and LiDAR, map data of the environment in which the autonomous vehicle is located, as well as the position and movement data of all surrounding vehicles, pedestrians, and cyclists, a dataset is created. However, due to practical limitations, relevant open-source datasets in the field of intelligent driving can be used, such as the Waymo dataset, the KITTI dataset, and the NuScenes dataset.

[0037] It should be noted that this description uses the Waymo motion dataset, which contains 574 hours of data, 103,354 map-based data segments, and three categories of labels: cars, pedestrians, and bicycles. Each object has a 2D bounding box annotation. The dataset mines behaviors and scenarios for behavioral prediction research, including turning, merging, changing lanes, and merging. Locations include San Francisco, Phoenix, Mountain View, Los Angeles, Detroit, and Seattle.

[0038] Preferably, step S2, as follows: Figure 2 As shown, it includes:

[0039] Vehicle behavior includes vehicle trajectory information; road environment includes lane line positions, zebra crossing positions, and other information; pedestrian / cyclist behavior includes pedestrian / cyclist trajectory information.

[0040] Using a vector representation method, different vector nodes are established for vehicle behavior, road environment, and pedestrian / cyclist behavior according to different categories. For each category, a subgraph of interconnected nodes is established, that is, different polyline subgraphs containing different numbers of vector nodes are established. Then, based on a graph neural network, an attention mechanism is used to capture the interaction between lanes and agents, and the interaction between polyline subgraphs is modeled, that is, a global interaction graph between different categories is constructed to obtain the vectorized interaction relationships between vehicles and roads, vehicles and people, vehicles and vehicles, etc., and to build a human-vehicle-road coupling relationship model based on vector representation.

[0041] It should be noted that the input information is rasterized and vectorized map information, specifically vector information such as vehicle, pedestrian, and cyclist trajectory features, lane features, and traffic light features from the Waymo motion dataset. The final representation is vector node feature information, and the output information is global interaction feature information.

[0042] It needs further clarification that rasterized and vectorized map information refers to the process of rasterizing the driving environment of autonomous vehicles and then vectorizing it to obtain the final vectorized map information. Raster maps are map image data created by scanning color maps, followed by digital transformation and image processing. The basic unit of composition is the raster (pixel), and the data consists of a matrix of points. Vector maps, on the other hand, encode and define the attributes of all content on the map according to relevant specifications and standards, determining the category, level, and characteristics of map elements, and representing them with corresponding coordinate positions. The data format consists of points and lines.

[0043] It is necessary to further explain the feature information of vector node i. Including the starting coordinates of the vector End point coordinates Corresponding attribute feature a i Examples of attributes include object type, trajectory timestamp, road feature type, or lane speed limit, as well as integer IDs corresponding to different categories j. All vector nodes of the same category are joined together (directly connected) to form P. j Therefore, V i ∈P j Where i, j = {1, 2, ..., P}, and P represents the maximum number of vector nodes of the same category.

[0044] Multiple nodes of the same category but different feature information are used to create multiple internal relational node subgraphs (polyline subgraphs) through relational operations such as aggregation and splicing. The formula is as follows:

[0045]

[0046] in, This represents the node feature information of the l-th layer node subgraph, where l = {0, 1, ..., P}. It is the input feature information V i , This involves aggregating information about all neighboring nodes connected to node i, and then performing a max-pooling operation. This represents the relationship between node i and its neighbor node j; selecting the concatenation operation will suffice. This represents the node feature information of the (l+1)th layer node subgraph.

[0047] Multiple node subgraphs of different categories are pooled to obtain multiple different node subgraph feature information, and the formula is as follows:

[0048]

[0049] in, Represents vector node V i The total set of feature information for P layers, P j This provides feature information for different categories of node subgraphs.

[0050] Furthermore, a global interaction graph is established based on a graph neural network, with the following formula:

[0051]

[0052] in, GNN(·) represents the set of feature information of node subgraphs of different categories, and GNN(·) is the corresponding one-layer graph neural network. This represents a set of global interaction graph feature information for different categories.

[0053] More specifically, this global interaction graph employs a self-attention mechanism, with the following formula:

[0054]

[0055] Where P represents the node feature matrix, P Q ,P K ,P V It is its corresponding linear projection.

[0056] Step S3, as follows Figure 3 As shown, it includes:

[0057] Vehicle behavior patterns include going straight, turning left, turning right, changing lanes to the left, and changing lanes to the right.

[0058] Based on the target vehicle state information and combined with the global interaction feature information of the human-vehicle-road coupling relationship model, a multi-layer long short-term memory network structure is adopted, and only the last node is output as the classification result, that is, the recognition results of 5 behavior patterns.

[0059] It should be noted that the input information consists of global interactive feature information and vehicle status information, and the output information consists of the vehicle behavior pattern recognition results for going straight, turning left, turning right, changing lanes left and changing lanes right.

[0060] It should be further explained that the vehicle status information includes vehicle ID, vehicle horizontal and vertical coordinates, vehicle speed, vehicle acceleration, and vehicle heading angle.

[0061] It should be noted that Long Short-Term Memory (LSTM) networks are a type of recurrent neural network suitable for processing and predicting important events with very long intervals and delays in time series. LSTMs primarily achieve selective forgetting and remembering of information through forget gates, input gates, and output gates. Furthermore, the orbits (called cell states) only involve simple linear operations (i.e., multiplication and addition), thus achieving long-term memory and effectively suppressing the gradient dissipation problem of long-term memory. The specific formula is as follows:

[0062] f t =σ(W f ·[h t-1 x t ]+b f )

[0063] i t =σ(W i ·[h t-1 x t ]+b i )

[0064]

[0065]

[0066] 0 t =σ(W o ·[h t-1 x t ]+b o )

[0067] h t =o t *tanh(C t )

[0068] Among them, f t This refers to the output value of the forget gate, σ(·) is the Sigmoid function, and W f It is the weight matrix of the forget gate, h t -1 and x t These are the output value of the LSTM at the previous time step and the input value of the network at the current time step, respectively. t-1 x t The symbol ] represents concatenating two vectors into a longer vector, b f It is the bias term of the forget gate, i t W refers to the output value of the input gate. i It is the weight matrix of the forget gate, b i It is the bias term of the forgetting gate. This refers to the current memory location, tanh(-) is the tanh function, and W C It is the weight matrix in the current memory, Ct This refers to the current state of the cell, C. t-1 It refers to long-term memory, o t This refers to the output value of the output gate, W. o It is the weight matrix of the output gate, b o It is the bias term of the output gate, h t This is the output value of the LSTM at the current time.

[0069] This specification employs a two-layer LSTM structure. The input layer contains global interaction feature information and vehicle state information. After passing through two LSTM layers, it passes through a fully connected layer, and only the output of the last node is used as the classification result, i.e., the recognition result of five behavior patterns. More specifically, the output result is the target vehicle's most likely behavior pattern within the prediction time. The prediction time can be selected as 3 seconds, 5 seconds, or 8 seconds. Furthermore, the cross-entropy loss function can be used.

[0070] It should be noted that the data format for the input layer is (batch size, feature length, embedding). The batch size can be set to 100, the number of features in the input data (number of LSTM units) can be set to 25, and the corresponding vector length (embedding) of the input LSTM units is 1. The data format for the two LSTM layers is (number of hidden layer nodes), so the number of hidden layer nodes can be chosen as 256. The data format for the fully connected layer is (input size, output size). The input size of the fully connected layer is the same as the number of hidden layer nodes, 256, and the output size is 5. The data format for the output layer is (batch size, feature length, output size). The batch size can be set to 100, the number of features in the input data (number of LSTM units) is 25, and the output size is 5.

[0071] Step S4, as follows Figure 4 As shown, it includes:

[0072] Convolutional neural networks and multilayer perceptrons are used to predict the long-term multimodal trajectory of the target vehicle, listing all possible predicted trajectories and selecting the most likely predicted trajectory.

[0073] It should be noted that the input information includes global interactive feature information, behavior pattern recognition results, and rasterized and vectorized map information. A vehicle trajectory prediction model is trained using a convolutional neural network model and a multilayer perceptron. The output information includes K predicted trajectories of the target vehicle within the prediction time and their corresponding confidence scores.

[0074] It should be further explained that Convolutional Neural Networks (CNNs) are a type of feedforward neural network that includes convolutional computations and has a deep structure. Their formula is as follows:

[0075]

[0076] in, This represents the convolution output of the i-th neuron in the l-th layer. Let be the weight of the j-th neuron in the (l+1)-th layer relative to the i-th neuron. Let f(·) be the deviation of the j-th neuron in the (l+1)-th layer, and f(·) be the activation function. This represents the convolution output of the j-th neuron in the (l+1)-th layer.

[0077] Classic convolutional neural network models include Inception, Xception, SeNet, and ResNext. This description may use Inception or Xception models.

[0078] It should be further explained that the Multilayer Perceptron (MLP) is a feedforward artificial neural network model, and its formula is as follows:

[0079]

[0080] Layer l contains m neurons. The convolution output of the k-th neuron in the l-th layer. Let f(·) be the weight of the k-th neuron in layer l to the j-th neuron in layer l+1, and let f(·) be the activation function. This is the convolution output of the j-th neuron in the (l+1)-th layer.

[0081] The multilayer perceptron architecture described in this paper includes normalization layers, fully connected layers, and non-linear activation function layers. Layer normalization refers to processing the data simultaneously during training and testing. By summing the input data of the same layer and calculating the mean and variance, the input data of each layer is normalized, thereby accelerating the convergence speed of the network. The ReLU function can be selected as the non-linear activation function.

[0082] It should be further clarified that in the K predicted trajectories and their corresponding confidence levels, K=6, meaning that six future trajectories of the target vehicle need to be predicted within the prediction time, and the confidence level C corresponding to each trajectory needs to be given. i = i / ∑ K e K Let i = {1, 2, ...,}, and e be the natural constant (Euler number). The future trajectory with the highest confidence level (highest probability) is selected as the prediction result. The prediction time can be selected as 3 seconds, 5 seconds, or 8 seconds.

[0083] In summary, this invention proposes a novel approach that integrates multiple existing models, which can improve the accuracy of autonomous vehicles in recognizing the behavior of surrounding vehicles and predicting their trajectories.

[0084] Specifically, for vehicle behavior pattern recognition, based on the existing method of using the LSTM model for recognition, a human-vehicle-road coupling relationship model (the formula of this model is also an existing technology) is integrated, thus enabling better recognition of vehicle behavior, that is, accurately identifying five behavior patterns: straight driving, lane changing, and turning left and right. For vehicle trajectory prediction, the method first uses the vehicle behavior pattern recognition results, then integrates the results of the human-vehicle-road coupling relationship model, and finally uses a CNN model + multilayer perceptron (existing methods use the CNN model for trajectory prediction) to perform long-term multimodal trajectory prediction of the target vehicle, which can improve the accuracy of trajectory prediction.

[0085] In summary, this invention proposes a method for vehicle behavior recognition and trajectory prediction based on recurrent convolutional neural networks, using autonomous vehicles as a platform. Based on a human-vehicle-road coupling relationship model, it combines five vehicle behavior pattern recognition models based on long short-term memory networks and utilizes a vehicle trajectory prediction model based on convolutional neural networks and multilayer perceptrons to perform long-term multimodal trajectory prediction of the target vehicle. It also studies an end-to-end behavior prediction algorithm for surrounding vehicles in the long-term domain, effectively improving the accuracy of autonomous vehicles in recognizing and predicting the behavior of surrounding vehicles, and enhancing the safety and efficiency of autonomous vehicle operation.

[0086] In another aspect, the present invention also discloses a computer-readable storage medium storing a computer program, which, when executed by a processor, causes the processor to perform the steps of any of the methods described above.

[0087] In another aspect, the present invention also discloses a computer device, including a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor performs the steps of any of the methods described above.

[0088] In another embodiment provided in this application, a computer program product containing instructions is also provided, which, when run on a computer, causes the computer to perform the steps of any of the methods described in the above embodiments.

[0089] It is understood that the system provided in the embodiments of the present invention corresponds to the method provided in the embodiments of the present invention, and the explanation, examples and beneficial effects of the relevant content can be referred to the corresponding parts of the above methods.

[0090] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments described above. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and RAMbus dynamic RAM (RDRAM), etc.

[0091] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for vehicle behavior recognition and trajectory prediction based on a recurrent convolutional neural network, characterized in that, Includes the following steps: Step 1: Collect relevant vehicle data based on the driving environment of the autonomous vehicle; Step 2: Based on the collected dataset, model vehicle behavior, road environment, and pedestrian / cyclist behavior, and construct a vector-representation-based human-vehicle-road coupling relationship model; Step 3: Based on the human-vehicle-road coupling relationship model, use a long short-term memory network to identify the behavior patterns of the target vehicle; Step 4: Combining the human-vehicle-road coupling relationship model and vehicle behavior recognition results, the trajectory prediction of the target vehicle is achieved based on a convolutional neural network; Step 2 includes: (1) Vehicle behavior includes vehicle trajectory information, road environment includes lane line position, zebra crossing position and other information, and pedestrian / cyclist behavior includes pedestrian / cyclist trajectory information; (2) Using a vector representation method, different vector nodes are established for vehicle behavior, road environment, and pedestrian / cyclist behavior according to different categories, specifically: Including vector nodes Feature information This includes the starting coordinates of the vector. End point coordinates Corresponding attribute features This includes object type, trajectory timestamp, road feature type or lane speed limit, and corresponding different categories. An integer ID, composed of all vector nodes of the same category, is formed by concatenation. , ;in, , This represents the maximum number of vector nodes of the same category; Multiple subgraphs with internal relationships are created by aggregating and concatenating the feature information of different nodes of the same category. The formula is as follows: in, Indicates the first Node feature information of the layer node subgraph. , Input feature information , It is an aggregation node Information on all connected neighbor nodes is used to select the maximum pooling operation. This indicates a node. Its neighboring nodes To determine the relationship between the elements, simply select the splicing operation. Indicates the first Node feature information of the layer node subgraph; Multiple node subgraphs of different categories are pooled to obtain multiple different node subgraph feature information, and the formula is as follows: in, Represents vector nodes Total The set of feature information of the layer Feature information of node subgraphs for different categories (3) Based on graph neural networks, a subgraph of nodes is built for each category, and then a global interaction graph between different categories is built, as shown in the following formula: in, This represents a set of feature information for node subgraphs of different categories. It corresponds to a single layer of a graph neural network. A set of global interaction graph feature information representing different categories; Ultimately, the vectorized relationships between vehicles and roads, vehicles and people, and vehicles and other vehicles are obtained, and a human-vehicle-road coupling relationship model based on vector representation is constructed.

2. The method for vehicle behavior recognition and trajectory prediction based on a recurrent convolutional neural network according to claim 1, characterized in that, Step 3 includes: Vehicle behavior patterns include going straight, turning left, turning right, changing lanes to the left, and changing lanes to the right; Based on the target vehicle state information and combined with the global interaction feature information of the human-vehicle-road coupling relationship model, a multi-layer long short-term memory network structure is adopted, and only the last node is output as the classification result, that is, the recognition results of the above 5 behavior patterns.

3. The method for vehicle behavior recognition and trajectory prediction based on a recurrent convolutional neural network according to claim 2, characterized in that, Step 3 adopts a 2-layer LSTM structure. The input layer is global interactive feature information and vehicle state information. After passing through 2 LSTM layers, it passes through a fully connected layer, and only the output of the last node is used as the classification result, that is, the recognition result of 5 behavior patterns. More specifically, the output result is the target vehicle is most likely to be in one of the 5 behavior patterns within the prediction time.

4. The method for vehicle behavior recognition and trajectory prediction based on a recurrent convolutional neural network according to claim 3, characterized in that: The LSTM structure achieves selective forgetting and remembering of information through forget gates, input gates, and output gates, as shown in the following formula: in, This refers to the output value of the forget gate. It is the Sigmoid function. It is the weight matrix of the forget gate. and These are the output value of the LSTM at the previous time step and the input value of the network at the current time step, respectively. This means concatenating two vectors into a longer vector. It is the bias term of the forgetting gate. This refers to the output value of the input gate. It is the weight matrix of the forget gate. It is the bias term of the forgetting gate. This refers to the current memory. It is the tanh function. It is the weight matrix of the current memory. It refers to the cell state at the current moment. It refers to long-term memory. This refers to the output value of the output gate. It is the weight matrix of the output gate. It is the bias term of the output gate. This is the output value of the LSTM at the current time.

5. A computer-readable storage medium storing a computer program that, when executed by a processor, causes the processor to perform the steps of the method as claimed in any one of claims 1 to 4.

Images