Vehicle trajectory prediction method, device and equipment and storage medium

Abstract

The application provides a vehicle trajectory prediction method and device, equipment and a storage medium. The method comprises: obtaining initial trajectory prediction associated data and stop-prohibited information corresponding to a target vehicle from at least one data platform; preprocessing the initial trajectory prediction associated data to obtain trajectory prediction processing data; inputting the trajectory prediction processing data into a first-layer neural network model, and using the first-layer neural network model to perform first-layer feature extraction, first-layer interaction relationship calculation and multi-modal prediction on the trajectory prediction processing data to obtain first-layer prediction data corresponding to the target vehicle; inputting the first-layer prediction data into a second-layer neural network model, and using the second-layer neural network model to perform second-layer feature extraction, second-layer interaction relationship calculation and single-modal prediction on the first-layer prediction data and the stop-prohibited information to obtain a single-modal predicted trajectory corresponding to the target vehicle.

Description

Technical Field

[0001] This application relates to data processing technology, and more particularly to a method, apparatus, device, and storage medium for predicting vehicle trajectories. Background Technology

[0002] With the rapid development of society, we have entered the era of big data in our daily lives, and how to utilize big data to achieve convenience has become a hot topic. In recent years, researchers have focused on processing large amounts of data to predict the trajectories of vehicles on the road, thereby facilitating vehicle tracking.

[0003] In existing technologies, prediction devices use a single initial data point to predict the trajectory of a target vehicle. This single initial data point can be initial state information or a portion of that initial state information. Alternatively, multiple initial data points are input into a single model to predict the trajectory of the target vehicle. The single model calculates the multiple initial data points in different forms to obtain the predicted trajectory. These multiple initial data points include initial state information and initial traffic information, among others.

[0004] Therefore, it is evident that a single initial data set is limited in scope and cannot encompass all the current data related to trajectory prediction for the target vehicle. Furthermore, using a single model to predict multiple initial data sets is problematic because the multiple initial data sets are in different forms, making it impossible to reasonably and harmoniously integrate them. Additionally, both single and multiple initial data sets contain limited data, leading to inaccurate prediction results. Summary of the Invention

[0005] This application provides a vehicle trajectory prediction method, apparatus, device, and storage medium to solve the problem of inaccurate prediction results.

[0006] In a first aspect, this application provides a vehicle trajectory prediction method, including:

[0007] The initial trajectory prediction association data and parking restriction information corresponding to the target vehicle are obtained from at least one data platform; wherein, the initial trajectory prediction association data includes initial state information, initial traffic information, initial geographic information and initial weather information;

[0008] The initial trajectory prediction associated data is preprocessed to obtain trajectory prediction processed data;

[0009] The trajectory prediction processing data is input into the first layer neural network model, and the first layer neural network model is used to perform first layer feature extraction, first layer interaction relationship calculation and multimodal prediction on the trajectory prediction processing data to obtain the first layer prediction data corresponding to the target vehicle.

[0010] The first layer prediction data is input into the second layer neural network model, and the second layer neural network model is used to perform second layer feature extraction, second layer interaction relationship calculation and single-modal prediction on the first layer prediction data and the no-parking information to obtain the single-modal prediction trajectory corresponding to the target vehicle.

[0011] In one embodiment, the first layer of the neural network model includes a first layer of feature extraction, a first layer of interaction relationship, and a multimodal prediction layer;

[0012] The step of using the first-layer neural network model to perform first-layer feature extraction, first-layer interaction relationship calculation, and multimodal prediction on the trajectory prediction processing data to obtain the first-layer prediction data corresponding to the target vehicle includes:

[0013] The trajectory prediction processing data is subjected to the first layer feature extraction layer to extract the first layer features, so as to obtain the first layer feature data corresponding to the target vehicle; the first layer feature data includes the first layer temporal features, the first layer spatial features and the first layer environmental features.

[0014] The interaction relationship is calculated on the first layer feature data using the first layer interaction relationship layer to obtain the first layer interaction features corresponding to the target vehicle.

[0015] The multimodal prediction layer is used to perform multimodal prediction on the first layer interaction features and the first layer feature data to obtain the first layer multimodal data corresponding to the target vehicle.

[0016] In one embodiment, the first feature extraction layer includes a first-layer Long Short-Term Memory (LSTM) encoder, a Convolutional Neural Network (CNN) sub-network layer, and a Shallow Neural Network (SNN) sub-network layer; the trajectory prediction processing data includes state time series data, traffic time series data, geographic time series data, geographic image data, and weather normalized data.

[0017] The step of using the first feature extraction layer to perform first-layer feature extraction on the trajectory prediction processing data to obtain first-layer feature data corresponding to the target vehicle includes:

[0018] The state time series data, the traffic time series data, and the geographic time series data are input into the first layer LSTM encoder, and the first layer LSTM encoder is used to extract the first layer time features of the state time series data, the traffic time series data, and the geographic time series data to obtain the first layer time features corresponding to the target vehicle.

[0019] The geographic image data is input into the CNN sub-network layer, and the CNN sub-network layer is used to extract spatial features from the geographic image data to obtain the first layer of spatial features corresponding to the target vehicle.

[0020] The normalized weather data is input into the SNN sub-network layer, and the SNN sub-network layer is used to extract environmental features from the normalized weather data to obtain the first layer of environmental features corresponding to the target vehicle.

[0021] In one embodiment, the first interaction layer includes a first-layer graph attention network (GAT); the first-layer GAT considers the interactions between the target vehicle and surrounding vehicles.

[0022] The step of using the first layer of interaction relationship to calculate the interaction relationship of the first layer of feature data to obtain the first layer of interaction features corresponding to the target vehicle includes:

[0023] The first-layer temporal features and the first-layer spatial features are input into the first-layer GAT. The first-layer GAT is used to calculate the interaction relationship between the first-layer temporal features and the first-layer spatial features to obtain the first-layer interaction features corresponding to the target vehicle.

[0024] In one embodiment, the multimodal prediction layer includes a first layer of Long Short-Term Memory (LSTM) network decoder and a multilayer perceptron (MLP) sub-network layer; the first layer of multimodal data includes multimodal predicted trajectories and driving lane probabilities.

[0025] The step of using the multimodal prediction layer to perform multimodal prediction on the first layer interaction features and the first layer feature data to obtain the first layer multimodal data corresponding to the target vehicle includes:

[0026] The first layer interaction features, the first layer time features, and the first layer environment features in the first layer feature data are input into the first layer LSTM decoder, and the first layer LSTM decoder is used to perform multimodal prediction on the first layer time features, the first layer interaction features, and the first layer environment features to obtain the multimodal predicted trajectory corresponding to the target vehicle.

[0027] The first layer time features, the first layer interaction features, and the first layer environment features are input into the MLP sub-network layer, and the MLP sub-network layer is used to predict the driving lane probability of the first layer time features, the first layer interaction features, and the first layer environment features to obtain the driving lane probability corresponding to the target vehicle.

[0028] In one embodiment, the second-layer neural network model includes a second-layer feature extraction layer, a second-layer interaction relationship layer, and a single-modal prediction layer; the first-layer prediction data includes the multimodal predicted trajectory, the driving lane probability, the first-layer spatial features, and the first-layer environmental features.

[0029] The second-layer neural network model is used to perform second-layer feature extraction, second-layer interaction relationship calculation, and single-modal prediction on the first-layer prediction data and the no-parking information to obtain the single-modal predicted trajectory corresponding to the target vehicle, including:

[0030] The second feature extraction layer is used to extract the second time feature from the first prediction data to obtain the second time feature of the target vehicle.

[0031] The second-layer time features, the first-layer spatial features, and the first-layer environmental features are input into the second-layer interaction relationship layer, and the second-layer interaction relationship layer is used to calculate the second-layer interaction relationship of the second-layer time features and the first-layer spatial features to obtain the second-layer interaction features corresponding to the target vehicle.

[0032] The second-layer time features, the second-layer interaction features, the first-layer environmental features, and the no-stopping information are input into the single-modal prediction layer, and the single-modal prediction layer is used to perform single-modal prediction on the second-layer time features, the second-layer interaction features, the first-layer environmental features, and the no-stopping information to obtain the single-modal predicted trajectory corresponding to the target vehicle.

[0033] In one embodiment, the second feature extraction layer includes a second long short-term memory (LSTM) encoder.

[0034] The step of using the second feature extraction layer to extract second-layer temporal features from the first-layer prediction data includes:

[0035] The multimodal predicted trajectory and the driving lane probability in the first layer prediction data are input into the second layer LSTM encoder, and the second layer LSTM encoder is used to extract the second layer temporal features of the multimodal predicted trajectory and the driving lane probability.

[0036] The second layer of interaction relationship includes a second layer graph attention network (GAT); when the second layer GAT predicts that the target vehicle will collide with surrounding vehicles, it considers the high degree of interaction between the target vehicle and surrounding vehicles.

[0037] The step of using the second layer of interaction relationships to calculate the second layer of interaction relationships between the second layer of temporal features and the first layer of spatial features includes:

[0038] The second-layer GAT is used to calculate the second-layer interaction relationship between the second-layer temporal features and the first-layer spatial features;

[0039] The single-modal prediction layer includes a second-layer Long Short-Term Memory (LSTM) decoder.

[0040] The step of using the single-modal prediction layer to perform single-modal prediction on the second-layer temporal features, the second-layer interaction features, the first-layer environmental features, and the no-stopping information includes:

[0041] The second-layer LSTM decoder is used to perform single-modal prediction on the second-layer time features, the second-layer interaction features, the first-layer environmental features, and the no-stop information.

[0042] Secondly, this application provides a vehicle trajectory prediction device, the device comprising:

[0043] The acquisition module is used to acquire initial trajectory prediction association data and no-parking information corresponding to the target vehicle from at least one data platform; wherein, the initial trajectory prediction association data includes initial state information, initial traffic information, initial geographic information, and initial weather information;

[0044] The classification module is used to preprocess the initial trajectory prediction associated data to obtain trajectory prediction processed data;

[0045] The first input module is used to input the trajectory prediction processing data into the first layer neural network model. The first layer neural network module is used to perform first layer feature extraction, first layer interaction relationship calculation and multimodal prediction on the trajectory prediction processing data using the first layer neural network model to obtain the first layer prediction data corresponding to the target vehicle.

[0046] The second input module is used to input the first layer prediction data into the second layer neural network model. The second layer neural network module is used to perform second layer feature extraction, second layer interaction relationship calculation and single-modal prediction on the first layer prediction data and the no-parking information using the second layer neural network model, so as to obtain the single-modal prediction trajectory corresponding to the target vehicle.

[0047] Thirdly, this application provides an electronic device, including: a processor, and a memory communicatively connected to the processor;

[0048] The memory stores computer-executed instructions;

[0049] The processor executes computer execution instructions stored in the memory to implement the method as described in the first aspect or any of the above.

[0050] Fourthly, this application provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement the method described in the first aspect or any of the above embodiments.

[0051] This application provides a vehicle trajectory prediction method, apparatus, device, and storage medium, comprising: acquiring initial trajectory prediction association data and no-parking information corresponding to a target vehicle from at least one data platform; wherein, the initial trajectory prediction association data includes initial state information, initial traffic information, initial geographic information, and initial weather information; preprocessing the initial trajectory prediction association data to obtain trajectory prediction processing data; inputting the trajectory prediction processing data into a first-layer neural network model, and using the first-layer neural network model to perform first-layer feature extraction, first-layer interaction relationship calculation, and multimodal prediction on the trajectory prediction processing data to obtain first-layer prediction data corresponding to the target vehicle; inputting the first-layer prediction data into a second-layer neural network model, and using the second-layer neural network model to perform second-layer feature extraction, second-layer interaction relationship calculation, and single-modal prediction on the first-layer prediction data and no-parking information to obtain a single-modal predicted trajectory corresponding to the target vehicle.The vehicle trajectory prediction device (hereinafter referred to as the prediction device) in this application can obtain initial prediction association data and no-stopping information corresponding to the target vehicle from at least one data platform. This allows the application to obtain diverse data, including initial trajectory prediction association data such as initial state information, initial traffic information, initial geographical information, and initial weather information. These data all influence the target vehicle's trajectory. Therefore, the data in this application is comprehensive and can more fully and accurately reflect the target vehicle's driving state information and surrounding environmental information, thus providing a more accurate trajectory and improving the user experience. Next, the prediction device preprocesses the initial trajectory prediction association data to obtain trajectory prediction processing data. Further, the prediction device inputs the trajectory prediction processing data into a first-layer neural network model to obtain first-layer prediction data. Then, the first-layer prediction data is used as input to a second-layer neural network model. The second-layer neural network model predicts the first-layer prediction data and no-stopping information to obtain a single-modal predicted trajectory. The second-layer neural network... The model incorporates no-stopping information, which can influence the trajectory of the target vehicle over a future period. This inclusion makes the data more comprehensive and more closely reflects actual lane conditions, helping to optimize the target vehicle's path, select appropriate lanes, save travel time and fuel, and thus improve accuracy and reliability. The application uses two neural network models to calculate interaction relationships, considering the interaction between the target vehicle and surrounding vehicles. Furthermore, a hierarchical neural network model is used to perform layered calculations of the interaction effects between vehicles, enabling accurate prediction of lane-changing behaviors during the target vehicle's journey, resulting in more accurate trajectory prediction. The hierarchical neural network design first predicts the first layer of data using a first-layer neural network model. This data is then further refined and input into a second-layer neural network model for second-layer feature extraction, second-layer interaction relationship calculation, and single-modal prediction. This ensures that the final single-modal predicted trajectory closely matches the actual situation, thereby improving the accuracy of the trajectory prediction results. Attached Figure Description

[0052] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0053] Figure 1 An application scenario diagram of the vehicle trajectory prediction method provided in this application;

[0054] Figure 2 A schematic diagram of a vehicle trajectory prediction device provided in this application;

[0055] Figure 3This is a schematic flowchart of a vehicle trajectory prediction method provided in Embodiment 1 of this application;

[0056] Figure 4 This is a schematic flowchart of a vehicle trajectory prediction method provided in Embodiment 2 of this application;

[0057] Figure 5 This is a schematic flowchart of a vehicle trajectory prediction method provided in Embodiment 3 of this application;

[0058] Figure 6 This is a schematic diagram of a first-layer neural network model structure provided in Embodiment 3 of this application;

[0059] Figure 7 This is a schematic diagram of a vehicle in a predicted lane area provided in Embodiment 3 of this application;

[0060] Figure 8 This is a schematic flowchart of a vehicle trajectory prediction method provided in Embodiment 4 of this application;

[0061] Figure 9 This is a schematic diagram of a second-layer neural network model structure provided in Embodiment 4 of this application;

[0062] Figure 10 A collision diagram provided for embodiment five of the application;

[0063] Figure 11 This is a schematic diagram of a hierarchical neural network model provided in Embodiment Six of this application;

[0064] Figure 12 This application provides a schematic diagram of a vehicle trajectory prediction device according to Embodiment Seven.

[0065] Figure 13 This is a schematic diagram of the structure of an electronic device provided in Embodiment 8 of this application.

[0066] The accompanying drawings illustrate specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concepts of this application to those skilled in the art through reference to particular embodiments. Detailed Implementation

[0067] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0068] First, let me explain the terms used in this application:

[0069] Long Short-Term Memory (LSTM) networks are a type of neural network model commonly used for processing time-series data. This application includes an LSTM encoder and an LSTM decoder. The input to the LSTM encoder is a sequence, and the output of the LSTM encoder is a fixed-dimensional vector. The input to the LSTM decoder is the encoded vector and the previous target element, and the output of the LSTM decoder is a sequence.

[0070] Convolutional Neural Network (CNN): A type of deep learning neural network primarily used for computer vision tasks such as image recognition, image classification, and object detection.

[0071] Shallow Neural Network (SNN): This is a simple neural network model containing only one or two layers, used to extract environmental features from weather information.

[0072] Graph Attention Network (GAT): A graph neural network model based on attention mechanisms, used to handle tasks involving graph data structures, such as node classification, graph classification, and link prediction.

[0073] Multilayer Perceptron (MLP): A type of feedforward neural network consisting of multiple fully connected layers. Each fully connected layer consists of multiple neurons, and each neuron is connected to all neurons in the previous layer. The input signal is weighted and then passed through an activation function to obtain the output.

[0074] Existing predictive technologies often use a single set of initial data to predict the trajectory of a target vehicle. For example, they may use GPS data combined with radar or video image data. This single initial data can be initial state information or a portion of it. Alternatively, existing technologies may combine portions of two or three types of initial data as the basis for trajectory prediction. Another approach is to use multiple initial data points input into a single model to predict the target vehicle's trajectory. This single model calculates the trajectory based on these multiple initial data points in different formats. The single model can capture certain temporal characteristics and nonlinear relationships to obtain a predicted trajectory. These multiple initial data points include initial state information and initial traffic information, among others.

[0075] Therefore, it is evident that a single initial data set is limited in scope and cannot encompass all the current data related to trajectory prediction for the target vehicle. Furthermore, using a single model to predict multiple initial data sets is problematic because these sets are in different formats and a single model cannot handle data of different types and dimensions. Consequently, it cannot consider the mutual influence and fusion between different data sets. Thus, using a single model cannot reasonably and harmoniously integrate multiple initial data sets. At the same time, both single and multiple initial data sets contain limited data, so both of these factors contribute to inaccurate predictions of the target vehicle's trajectory.

[0076] To address the shortcomings of existing technologies, the inventors of this solution have creatively designed a new approach. This solution provides a vehicle trajectory prediction method. To solve the problem of inaccurate prediction results for target vehicle trajectories, the prediction device in this solution first acquires initial trajectory prediction correlation data and no-stopping information corresponding to the target vehicle from at least one data platform. The initial trajectory prediction correlation data includes initial state information, initial traffic information, initial geographic information, and initial weather information. Therefore, the initial prediction correlation data in this solution is comprehensive, considering data that may affect the target vehicle's trajectory from various aspects. Next, the prediction device preprocesses the initial trajectory prediction correlation data to obtain trajectory prediction processing data. Furthermore, a first-layer neural network model is used to perform first-layer feature extraction, first-layer interaction relationship calculation, and multimodal prediction on the trajectory prediction processing data to obtain first-layer prediction data corresponding to the target vehicle. The proposed solution first employs a first-layer neural network model for prediction. Then, a second-layer neural network model is used to perform second-layer feature extraction, second-layer interaction calculation, and single-modal prediction on the first-layer prediction data and no-stopping information to obtain the single-modal predicted trajectory of the target vehicle. Since no-stopping information is included, this solution further considers more data influencing the target vehicle's trajectory. The second-layer neural network model performs various calculations on the first-layer prediction data and no-stopping information to obtain the single-modal predicted trajectory of the target vehicle. Therefore, this solution leverages the feature extraction, interaction calculation, multimodal prediction, and single-modal prediction of both first-layer and second-layer neural networks to ultimately obtain the single-modal predicted trajectory. Thus, using a hierarchical neural network for trajectory prediction ensures the accuracy of the target vehicle's trajectory prediction. Furthermore, the use of a neural network model for interaction calculation considers the interaction between the target vehicle and surrounding vehicles, as well as the influence of surrounding vehicles on the target vehicle, further improving the accuracy of the target vehicle's trajectory prediction.

[0077] The following describes the application scenarios of the vehicle trajectory prediction method, device, equipment, and storage medium provided in this application.

[0078] Figure 1 This diagram illustrates an application scenario for the vehicle trajectory prediction method provided in this application. Figure 1 As shown in the diagram, the application scenario includes a large data platform 101 and an electronic device 102.

[0079] The data platform 101 can be a server or a mobile device, etc., without limitation. It should be noted that the data platform 101 includes at least one data platform, each of which can provide different data. These data platforms may include transportation bureau platforms, meteorological bureau platforms, geological bureau platforms, vehicle-mounted sensors, and open internet data platforms, etc.

[0080] The electronic device 102 includes a prediction device 103. The data platform 101 is communicatively connected to the electronic device 102, and the connection can be wired or wireless. The prediction device 103 includes a first-layer neural network model 104 and a second-layer neural network model 105.

[0081] Specifically, the first vehicle traveling in the lane of the prediction area is the target vehicle. The data platform 101 stores the initial trajectory prediction association data and no-stopping information corresponding to the target vehicle, and sends the initial trajectory prediction association data and no-stopping information to the electronic device 102. The electronic device 102 obtains the initial trajectory prediction association data and no-stopping information and sends it to the prediction device 103. The prediction device 103 preprocesses the initial trajectory prediction association data to obtain trajectory prediction processing data. Then, the prediction device 103 inputs the trajectory prediction processing data into the first-layer neural network model 104. The prediction device 103 uses the first-layer neural network model 104 to perform first-layer feature extraction, first-layer interaction relationship calculation and multimodal prediction on the trajectory prediction processing data, and finally obtains the first-layer prediction data corresponding to the target vehicle.

[0082] Furthermore, the prediction device 103 inputs the first-layer prediction data into the second-layer neural network model 105. The prediction device 103 uses the second-layer neural network model 105 to perform second-layer feature extraction, second-layer interaction relationship calculation and single-modal prediction on the first-layer prediction data and the no-stop information, thereby obtaining the single-modal prediction trajectory corresponding to the target vehicle, and thus successfully predicting the trajectory of the target vehicle.

[0083] It should be noted that the prediction device 103 also includes a data processing module, which is used to preprocess the initial trajectory prediction associated data to obtain trajectory prediction processed data.

[0084] It should be noted that the initial trajectory prediction associated data includes initial state information, initial traffic information, initial geographic information, and initial weather information. Initial state information can be obtained from the transportation bureau platform and / or vehicle-mounted sensors; initial traffic information can also be obtained from the transportation bureau platform; initial geographic information can be obtained from the geological bureau platform; and initial weather information can be obtained from the meteorological bureau platform. It should also be noted that some of this information can be obtained from open data platforms on the internet; there are no restrictions on this.

[0085] Figure 2 This is a schematic diagram of the structure of a vehicle trajectory prediction device provided in this application. Figure 2 As shown, the data processing module 201, the first-layer neural network model 202, and the second-layer neural network model 203 are included. The data processing module 201 is connected to the first-layer neural network model 202. The first-layer neural network model includes a first-layer feature extraction layer 2021, a first-layer interaction relationship layer 2022, and a multimodal prediction layer 2023. The second-layer neural network model includes a second-layer feature extraction layer 2031, a second-layer interaction relationship layer 2032, and a single-modal prediction layer 2033. The single-modal predicted trajectory corresponding to the target vehicle is output from the second-layer neural network model.

[0086] The data processing module 201 is connected to the first-layer neural network model 202, and the first-layer neural network model 202 is connected to the second-layer neural network model 203.

[0087] The first feature extraction layer 2021 is connected to the first interaction layer 2022; both the first feature extraction layer 2021 and the first interaction layer 2022 are connected to the multimodal prediction layer 2023. The second feature extraction layer 2031 is connected to the second interaction layer 2032; both the second feature extraction layer 2031 and the second interaction layer 2032 are connected to the single-modal prediction layer 2033.

[0088] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use and processing of the relevant data must comply with relevant laws, regulations and standards, and corresponding operation entry points are provided for users to choose to authorize or refuse.

[0089] This application provides a vehicle trajectory prediction method, which aims to solve the above-mentioned technical problems of the prior art.

[0090] The technical solution of this application and how the technical solution of this application solves the above-mentioned technical problems are described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will now be described with reference to the accompanying drawings.

[0091] Example 1

[0092] Figure 3 This is a schematic flowchart illustrating a vehicle trajectory prediction method provided in Embodiment 1 of this application. The executing entity of this application is a prediction device, which can be deployed in an electronic device. For example... Figure 3 As shown, the specific steps are as follows.

[0093] S301, Obtain initial trajectory prediction associated data and no-parking information corresponding to the target vehicle from at least one data platform; wherein, the initial trajectory prediction associated data includes initial state information, initial traffic information, initial geographic information and initial weather information.

[0094] Initial state information refers to at least one of the following: position, speed, acceleration, and direction of the target vehicle and surrounding vehicles.

[0095] Initial traffic information refers to traffic lights and all traffic-related information, specifically including traffic light and traffic flow information. Traffic light information may include traffic light status and countdown timers. Traffic flow information may include historical traffic flow and congestion data, which can be used to describe congestion conditions in different areas.

[0096] Initial geographic information refers to geographic information such as high-rise buildings, structures, and speed limits on roads.

[0097] Initial weather information refers to the weather conditions corresponding to the target vehicle, including discrete and continuous information. Discrete information can be weather such as rain or snow, while continuous information can be specific temperature and humidity, etc. There are no restrictions here.

[0098] Among them, no-parking information refers to areas on the road where vehicles are not allowed to park or drive. For example, bus lanes where private vehicles are not allowed to drive, and parking spaces where outside vehicles are not allowed to park in addition to private vehicles.

[0099] The data platform may include transportation bureau platforms, meteorological bureau platforms, geological bureau platforms, vehicle-mounted sensors, and open internet data platforms.

[0100] It should be noted that initial traffic information can be obtained from the transportation bureau's platform or from an open data platform on the Internet. This makes the sources of initial traffic information more diverse, comprehensive, and complete.

[0101] It should be noted that the predictive device may obtain initial trajectory prediction associated data and parking restriction information from at least one data platform in different ways. For open data platforms on the Internet and vehicle sensors, the acquisition can be achieved by successively confirming the data format, importing data, cleaning data, converting data, and analyzing data.

[0102] It should be noted that the initial trajectory prediction correlation data and no-stopping information can be obtained through processes such as confirming the data format, importing data, data cleaning, data conversion, and data analysis, or other feasible methods; no restrictions are imposed here. The methods for obtaining initial trajectory prediction correlation data can be the same or different for different scenarios.

[0103] S302, preprocess the initial trajectory prediction associated data to obtain trajectory prediction processed data.

[0104] Furthermore, the prediction device preprocesses the initial trajectory prediction associated data. Specifically, the prediction device includes a data processing module that preprocesses the initial trajectory prediction associated data into time-series data, image data, and normalized data, and determines the resulting processed data as the trajectory prediction processing data. This trajectory prediction processing data may include state time-series data, traffic time-series data, geographic time-series data, geographic image data, weather normalized data, and other time-series or image data.

[0105] It should be noted that the initial trajectory prediction associated data, as initial data, may be missing some data. For example, the initial state information may only include velocity information and not information such as position, acceleration, and direction; or the initial trajectory prediction associated data may not have been standardized; or the prediction device may not obtain the initial trajectory prediction associated data from at least one data platform in an orderly manner according to categories, but may obtain it scattered together. For example, multiple pieces of information in the initial state information may be scattered in the initial traffic information.

[0106] Specifically, the prediction device will supplement the missing initial state information to obtain complete state information. Then, the prediction device will process the complete state information into state time-series data. The initial state information and initial traffic information change over time. Further, the prediction device can standardize the initial traffic information, classify it, and process it into traffic time-series data. For the initial geographic information, most of which consists of buildings with fixed spatial structures and relationships, the prediction device will process it into geographic image data. A small portion of the geographic information, lane speed limits, is time-related data and will be processed into geographic time-series data. For the initial weather information, it will be processed into discrete weather information and continuous normalized data. For example, one-hot encoding (OHE) can be used to process discrete information into discrete weather information; simultaneously, normalization processing can be used to normalize continuous information, ultimately obtaining continuous normalized data. The data obtained through these two processing methods will ultimately yield normalized weather data. Other feasible methods can be used in this embodiment, and no limitations are imposed here. In this embodiment, the initial weather information can be processed by unique thermal encoding and / or normalization to obtain normalized weather data.

[0107] It should be noted that preprocessing involves supplementing the initial trajectory prediction data with calculations, classifications, and standardization to make the data more complete.

[0108] In one approach, this method uses the preprocessing of initial state information as an example. The preprocessing flow is as follows: determining the data type, standardizing and / or normalizing, and outputting the state time series data. Other initial trajectory-related data can be processed using corresponding preprocessing methods, which will not be elaborated here.

[0109] S303, the trajectory prediction processing data is input into the first-layer neural network model, and the first-layer neural network model is used to perform first-layer feature extraction, first-layer interaction relationship calculation and multimodal prediction on the trajectory prediction processing data to obtain the first-layer prediction data corresponding to the target vehicle.

[0110] The first-layer feature extraction refers to the extraction of time, space, environment, and other features in the first-layer neural network model.

[0111] The first-layer interaction relationship calculation refers to the calculation of the interaction and influence between the target vehicle and surrounding vehicles in the first-layer neural network model.

[0112] Multimodal prediction refers to the multimodal prediction of lane occupancy and the target vehicle's trajectory in the first layer of the neural network model. A multimodal predicted trajectory involves predicting multiple trajectories for the target vehicle, including at least one predicted trajectory. A single-modal predicted trajectory involves predicting only one trajectory for the target vehicle. This single predicted trajectory has the highest probability of occurrence and is the most likely trajectory for the target vehicle to move according to in the future.

[0113] The first layer of prediction data includes any data output from the first layer of the neural network model; there are no restrictions here.

[0114] It should be noted that in this embodiment, the prediction device can use the previous output results as the input results of the next according to the calculation order, so as to perform the first layer feature extraction, the first layer interaction relationship calculation and multimodal prediction in sequence.

[0115] Specifically, the prediction device uses a first-layer neural network model to perform first-layer feature extraction on the trajectory prediction processing data to obtain first-layer feature data. Then, it performs first-layer interaction relationship calculation on the first-layer feature data to obtain first-layer interaction features. Finally, it performs multimodal prediction on the first-layer feature data and the first-layer interaction features to obtain first-layer prediction data.

[0116] S304, the first-layer prediction data is input into the second-layer neural network model, and the second-layer neural network model is used to perform second-layer feature extraction, second-layer interaction relationship calculation and single-modal prediction on the first-layer prediction data and no-stopping information to obtain the single-modal prediction trajectory corresponding to the target vehicle.

[0117] It should be noted that this step is similar to S303. The previous output result can also be used as the input result for the next step. Then, the second layer of feature extraction, the second layer of interaction relationship calculation and single-modal prediction can be performed in sequence to finally obtain the single-modal predicted trajectory corresponding to the target vehicle.

[0118] It should be noted that the trajectory in this application can be understood as location information. This application predicts the location changes of a target vehicle within a certain period of time in the future. For example, if the current location of the target vehicle is (1, 1), the prediction device predicts the next location of the target vehicle in the next few seconds or tens of seconds. Assuming that the prediction device predicts that the location of the target vehicle will be (1, 2) in ten seconds, then the target vehicle moving from location (1, 1) to location (1, 2) is the predicted trajectory in this application.

[0119] This embodiment provides a vehicle trajectory prediction method, including: acquiring initial trajectory prediction association data and no-parking information corresponding to a target vehicle from at least one data platform; wherein, the initial trajectory prediction association data includes initial state information, initial traffic information, initial geographic information, and initial weather information; preprocessing the initial trajectory prediction association data to obtain trajectory prediction processing data; inputting the trajectory prediction processing data into a first-layer neural network model, and using the first-layer neural network model to perform first-layer feature extraction, first-layer interaction relationship calculation, and multimodal prediction on the trajectory prediction processing data to obtain first-layer prediction data corresponding to the target vehicle; inputting the first-layer prediction data into a second-layer neural network model, and using the second-layer neural network model to perform second-layer feature extraction, second-layer interaction relationship calculation, and single-modal prediction on the first-layer prediction data and no-parking information to obtain a single-modal predicted trajectory corresponding to the target vehicle.In this embodiment, the prediction device can obtain initial prediction association data and no-stopping information corresponding to the target vehicle from at least one data platform. This allows the device to acquire diverse data. The initial trajectory prediction association data includes initial state information, initial traffic information, initial geographic information, and initial weather information. These data all affect the target vehicle's trajectory. Therefore, the data in this application is comprehensive and can more fully and accurately reflect the target vehicle's driving status information and surrounding environmental information, thus providing a more accurate trajectory and improving the user experience. Next, the prediction device preprocesses the initial trajectory prediction association data to obtain trajectory prediction processing data. Further, the prediction device inputs the trajectory prediction processing data into a first-layer neural network model to obtain first-layer prediction data. Then, the first-layer prediction data is used as input to a second-layer neural network model. The second-layer neural network model predicts the first-layer prediction data and no-stopping information to obtain a single-modal predicted trajectory. The no-stopping information is added to the second-layer neural network model. Since no-stopping information can affect the trajectory of a target vehicle over a period of time, this embodiment considers no-stopping information, making the data more comprehensive and more closely aligned with actual lane conditions. This helps optimize the target vehicle's path, select appropriate lanes, save driving time and fuel consumption, and thus improve accuracy and reliability. This embodiment uses two neural network models to calculate interaction relationships, thus considering the interaction between the target vehicle and surrounding vehicles. Furthermore, this application uses a hierarchical neural network model to perform hierarchical calculations of the interaction effects between vehicles, thereby accurately predicting the target vehicle's lane-changing behaviors during driving, resulting in more accurate trajectory prediction. This embodiment employs a hierarchical neural network design, first predicting the first layer of data through the first-layer neural network model, then further refining the data and inputting it into the second-layer neural network model to further achieve second-layer feature extraction, second-layer interaction relationship calculation, and single-modal prediction. This ensures that the final single-modal predicted trajectory closely matches the actual situation, thereby improving the accuracy of the trajectory prediction results.

[0120] Example 2

[0121] This embodiment is a further refinement of the first embodiment described above. In this embodiment, the first layer of the neural network model includes a first layer of feature extraction, a first layer of interaction relationship, and a multimodal prediction layer.

[0122] The first feature extraction layer refers to the network layer that extracts feature data.

[0123] The first layer, the interaction relationship layer, refers to the network layer that calculates the interaction relationship and degree of influence between the target vehicle and surrounding vehicles.

[0124] The multimodal prediction layer refers to the network layer that calculates lane occupancy and the multimodal predicted trajectory of the target vehicle.

[0125] Figure 4 This is a schematic flowchart of a vehicle trajectory prediction method provided in Embodiment 2 of this application. This embodiment is an optional approach that uses a first-layer neural network model to perform first-layer feature extraction, first-layer interaction relationship calculation, and multimodal prediction on the trajectory prediction processing data to obtain the first-layer prediction data corresponding to the target vehicle. Figure 4 As shown, the specific steps are as follows.

[0126] S401, the first layer feature extraction layer is used to extract the first layer features from the trajectory prediction processing data to obtain the first layer feature data corresponding to the target vehicle; the first layer feature data includes the first layer time features, the first layer spatial features and the first layer environmental features.

[0127] The first layer feature data is the output of the first feature extraction layer. The first layer feature data can exist in vector form.

[0128] S402, the first layer of interaction relationship is used to calculate the interaction relationship of the first layer of feature data to obtain the first layer of interaction features corresponding to the target vehicle.

[0129] The first layer of interaction features refers to the interaction relationship and degree of influence between the target vehicle and surrounding vehicles in the first layer of the neural network model.

[0130] Specifically, the prediction device inputs at least one type of data from the first layer of feature data into the first layer of interaction relationship layer and outputs the first layer of interaction features.

[0131] S403 employs a multimodal prediction layer to perform multimodal prediction on the first-layer interaction features and the first-layer feature data to obtain the first-layer multimodal data corresponding to the target vehicle.

[0132] The first layer of multimodal data refers to the lane where the target vehicle is located and the occupancy of surrounding lanes, as well as multimodal predicted trajectory data. The lane occupancy can be characterized by the probability of the vehicle traveling in a lane.

[0133] This embodiment provides a vehicle trajectory prediction method. The first-layer neural network model in this embodiment includes a first-layer feature extraction layer, a first-layer interaction relationship layer, and a multimodal prediction layer. When using the first-layer neural network model to perform first-layer feature extraction, first-layer interaction relationship calculation, and multimodal prediction on the trajectory prediction processing data to obtain first-layer prediction data corresponding to the target vehicle, the method specifically includes: using the first-layer feature extraction layer to perform first-layer feature extraction on the trajectory prediction processing data to obtain first-layer feature data corresponding to the target vehicle; the first-layer feature data includes first-layer temporal features, first-layer spatial features, and first-layer environmental features; using the first-layer interaction relationship layer to perform interaction relationship calculation on the first-layer feature data to obtain first-layer interaction features corresponding to the target vehicle; and using the multimodal prediction layer to perform multimodal prediction on the first-layer interaction features and the first-layer feature data to obtain first-layer multimodal data corresponding to the target vehicle. In this embodiment, the first layer of feature extraction is performed first to obtain the first layer of feature data. Then, the first layer of interaction relationship is used to calculate the interaction relationship of the first layer of feature data to obtain the first layer of interaction features. Finally, the multimodal prediction layer is used to perform multimodal prediction on the first layer of interaction features and the first layer of feature data, and finally obtain the first layer of multimodal data. In this embodiment, the first layer of feature extraction and the first layer of interaction relationship calculation are performed before the multimodal prediction. Therefore, this embodiment performs multiple calculations on the trajectory prediction data to make the first layer of multimodal data more accurate and comprehensive.

[0134] Example 3

[0135] This embodiment is a further refinement of any of the above embodiments. In this embodiment, the first feature extraction layer includes a first-layer long short-term memory network (LSTM) encoder, a convolutional neural network (CNN) sub-network layer, and a shallow neural network (SNN) sub-network layer. The trajectory prediction processing data includes state time series data, traffic time series data, geographic time series data, geographic image data, and weather normalized data.

[0136] Figure 5 This is a schematic flowchart of a vehicle trajectory prediction method provided in Embodiment 3 of this application. This embodiment is an optional method that uses a first-layer feature extraction layer to extract first-layer features from the trajectory prediction processing data to obtain the first-layer feature data corresponding to the target vehicle. Figure 5 As shown, the specific steps are as follows.

[0137] S501, input the state time series data, traffic time series data and geographic time series data into the first layer LSTM encoder, and use the first layer LSTM encoder to extract the first layer time features of the state time series data, traffic time series data and geographic time series data to obtain the first layer time features corresponding to the target vehicle.

[0138] Specifically, the first layer of LSTM obtains time series data, which includes state time series data, traffic time series data, and geographic time series data. Based on the time series data, the output of the previous time moment is used as the input of the current time moment, thereby capturing the time dependencies in the time series data and extracting the first layer of time features.

[0139] S502, input the geographic image data into the CNN sub-network layer, and use the CNN sub-network layer to extract spatial features from the geographic image data to obtain the first layer of spatial features corresponding to the target vehicle.

[0140] It should be noted that the CNN sub-network layers include a CNN input layer, convolutional layers, pooling layers, and fully connected layers. The CNN input layer receives geographic image data. The convolutional layer is mainly used to extract features from the geographic image data. The convolutional layer performs a convolution operation on the CNN input layer by sliding a convolutional kernel, thus obtaining a new geographic image feature map. The convolutional kernel can extract a specific image feature from the geographic image feature map, such as edges or textures. The pooling layer can reduce the dimensionality of the geographic image data, reducing complexity by averaging or maximizing each small region in the geographic image feature map, resulting in a new, smaller geographic image feature map, which is then output to the fully connected layer. The fully connected layer maps the smaller geographic image feature map output by the pooling layer to specific categories to complete the task.

[0141] It should be noted that the convolutional and pooling layers of the CNN sub-network can effectively capture the local features and spatial structure of geographic image data, while the fully connected layers can convert these features into the final output. Thus, the prediction device will extract spatial features from the geographic image data, thereby obtaining the first layer of spatial features.

[0142] S503 inputs the normalized weather data into the SNN sub-network layer and uses the SNN sub-network layer to extract environmental features from the normalized weather data to obtain the first layer of environmental features corresponding to the target vehicle.

[0143] It should be noted that the SNN sub-network layer includes an SNN input layer, an SNN hidden layer, and an SNN output layer. The SNN input layer is mainly used to input normalized weather data, which can be either a vector or a matrix. The hidden layer extracts and transforms features from the normalized weather data. The output layer outputs the first layer of environmental features. Figure 6 This is a schematic diagram of a first-layer neural network model structure provided in Embodiment 3 of this application. Figure 6 As shown, the first feature extraction layer includes a first-layer LSTM encoder, a CNN sub-network layer, and an SNN sub-network layer.

[0144] It should be noted that in this embodiment, the first LSTM encoder can effectively capture long-term temporal dependencies; the CNN sub-network layer can effectively capture local features and spatial structure of geographic images; and the SNN sub-network layer can effectively reduce model complexity.

[0145] This embodiment provides a vehicle trajectory prediction method. The first feature extraction layer in this embodiment includes a first-layer Long Short-Term Memory (LSTM) encoder, a Convolutional Neural Network (CNN) sub-network layer, and a Shallow Neural Network (SNN) sub-network layer. The trajectory prediction processing data includes state time-series data, traffic time-series data, geographic time-series data, geographic image data, and weather normalized data. Specifically, when using the first-layer feature extraction layer to extract first-layer features from the trajectory prediction processing data to obtain the first-layer feature data corresponding to the target vehicle, the method includes: inputting the state time-series data, traffic time-series data, and geographic time-series data into the first-layer LSTM encoder, and using the first-layer LSTM encoder to extract first-layer time features from the state time-series data, traffic time-series data, and geographic time-series data to obtain the first-layer time features corresponding to the target vehicle; inputting the geographic image data into the CNN sub-network layer, and using the CNN sub-network layer to extract spatial features from the geographic image data to obtain the first-layer spatial features corresponding to the target vehicle; and inputting the weather normalized data into the SNN sub-network layer, and using the SNN sub-network layer to extract environmental features from the weather normalized data to obtain the first-layer environmental features corresponding to the target vehicle. In this embodiment, LSTM is used for the first layer of temporal feature extraction to obtain the first layer of temporal features, CNN sub-network layers are used for spatial feature extraction to obtain the first layer of spatial features, and SNN is used for environmental feature extraction to obtain the first layer of environmental features. As can be seen, this embodiment uses different sub-network layers for corresponding feature extraction. The selected sub-network layers are in line with the actual situation. Therefore, when each sub-network layer extracts its own appropriate and compatible inputs, it will be more harmonious, and the final extraction result will be more accurate.

[0146] In one embodiment, the first interaction layer includes a first-layer graph attention network (GAT); the first-layer GAT considers the interaction between the target vehicle and surrounding vehicles. This embodiment is an optional method of using the first-layer interaction layer to calculate the interaction relationship of the first-layer feature data to obtain the first-layer interaction features corresponding to the target vehicle, as detailed below.

[0147] The first-layer temporal features and the first-layer spatial features are input into the first-layer GAT. The first-layer GAT is used to calculate the interaction relationship between the first-layer temporal features and the first-layer spatial features to obtain the first-layer interaction features corresponding to the target vehicle.

[0148] It should be noted that the first layer of GAT includes a GAT input layer, a GAT hidden layer, and a GAT output layer. The GAT input layer receives the first-layer temporal and spatial features. The GAT hidden layer extracts and transforms these features. The core of the GAT hidden layer is the attention mechanism, which calculates the first-layer attention score and, combined with the corresponding first-layer temporal and spatial features, calculates the first-layer weighted node feature vector. This weighted node feature vector is then used as the first-layer interaction feature. The first-layer weighted node feature vector and the first-layer attention score can also be used to characterize the influence of surrounding vehicles on the target vehicle.

[0149] like Figure 6 As shown, the first layer of interaction relationship includes the first layer of GAT, whose inputs are the first layer of time features and the first layer of spatial features, and whose output is the first layer of interaction features.

[0150] It should be noted that the first-layer temporal features and the first-layer spatial features describe the feature vector of each node.

[0151] It should be noted that considering the interaction between the target vehicle and surrounding vehicles in the first layer of GAT means that each vehicle can be regarded as a node, and then the interaction between the nodes is feature extracted, and the interaction between the nodes is learned using an attention mechanism. Here, the target vehicle is the target node, and the surrounding vehicles are the neighbor nodes.

[0152] Specifically, the first layer of GAT treats the first layer of temporal features and the first layer of spatial features as node feature matrices. Figure 7 This is a schematic diagram illustrating a vehicle prediction method in a lane within a specified area, as provided in Embodiment 3 of this application. Figure 7 Including (A) and (B), as shown in (A), the current prediction area has 3 lanes, namely lane A, lane B and lane C, and 3 vehicles, namely vehicle A, vehicle B and vehicle C, which are considered as 3 nodes. The corresponding node feature matrix can be represented as:

[0153]

[0154] Among them, y LSTM ,y CNN These represent the first-layer temporal features output by the first-layer LSTM encoder of vehicle A, and the first-layer spatial features output by the first-layer CNN sub-network layer, respectively. LSTM ,y CNN The node feature corresponding to vehicle A is identified as b. LSTM ,b CNN These represent the first-layer temporal features output by the first-layer LSTM encoder of vehicle B, and the first-layer spatial features output by the first-layer CNN sub-network layer, respectively.LSTM ,b CNN The node feature corresponding to vehicle B has been identified. LSTM ,o CNN These represent the first-layer temporal features output by the first-layer LSTM encoder of vehicle C, and the first-layer spatial features output by the CNN sub-network layer, respectively. LSTM ,o CNN The node features corresponding to vehicle C were identified.

[0155] Next, the first-layer GAT will use an attention mechanism to calculate the first-layer attention score of each neighbor node to the target node. This first-layer attention score can be calculated based on the first-layer temporal and spatial features, or it can be calculated based on the distance between each node; no restriction is placed here. Further, the first-layer GAT will calculate the first-layer weighted node feature vector based on the first-layer attention scores and corresponding node features of each surrounding vehicle, outputting the first-layer interaction features corresponding to the target vehicle.

[0156] For example, taking vehicle A as the target vehicle, for vehicle A, assuming that the first-layer attention score corresponding to vehicle B is 0.6 and the first-layer attention score corresponding to vehicle C is 0.4, the prediction device will calculate the product of the first-layer attention score of each surrounding vehicle and the corresponding node feature, sum the product results, calculate the first-layer weighted node feature vector, and determine the first-layer weighted node feature vector as the first-layer interaction feature of the target vehicle.

[0157] It should be noted that in this embodiment, the graph edge weight matrix, graph edge feature matrix, and adjacency matrix can be used to calculate the first layer of interaction features.

[0158] This embodiment provides a vehicle trajectory prediction method. The first interaction layer in this embodiment includes a first-layer graph attention network (GAT). The first-layer GAT considers the interaction between the target vehicle and surrounding vehicles. Specifically, when using the first-layer interaction layer to calculate the interaction relationships of the first-layer feature data to obtain the first-layer interaction features corresponding to the target vehicle, the method includes: inputting the first-layer temporal features and first-layer spatial features into the first-layer GAT; and using the first-layer GAT to calculate the interaction relationships of the first-layer temporal features and first-layer spatial features to obtain the first-layer interaction features corresponding to the target vehicle. In this embodiment, the first-layer GAT considers the interaction between the target vehicle and surrounding vehicles, thus obtaining the first-layer interaction features corresponding to the target vehicle. Since the interaction between the target vehicle and surrounding vehicles will affect the target vehicle's trajectory over a future period, considering the interaction and obtaining the first-layer interaction features in this embodiment makes the prediction more realistic and the final prediction result more accurate.

[0159] In one embodiment, the multimodal prediction layer includes a first layer of Long Short-Term Memory (LSTM) network decoder and a multilayer perceptron (MLP) sub-network layer; the first layer of multimodal data includes multimodal predicted trajectory and driving lane probability.

[0160] This embodiment is an optional method that uses a multimodal prediction layer to perform multimodal prediction on the first-layer interaction features and the first-layer feature data to obtain the first-layer multimodal data corresponding to the target vehicle. The specific content is as follows.

[0161] The first-layer interaction features, as well as the first-layer temporal features and first-layer environmental features from the first-layer feature data, are input into the first-layer LSTM decoder. The first-layer LSTM decoder is then used to perform multimodal prediction on the first-layer temporal features, first-layer interaction features, and first-layer environmental features to obtain the multimodal predicted trajectory corresponding to the target vehicle. The first-layer temporal features, first-layer interaction features, and first-layer environmental features are then input into the MLP sub-network layer. The MLP sub-network layer is then used to perform driving lane probability prediction on the first-layer temporal features, first-layer interaction features, and first-layer environmental features to obtain the driving lane probability corresponding to the target vehicle.

[0162] Among them, the driving lane probability refers to the probability that the target vehicle is driving in each lane. The higher the probability, the more likely the target vehicle is to drive in that lane.

[0163] like Figure 6As shown, the multimodal prediction layer includes a first-layer LSTM decoder and an MLP sub-network layer. The first-layer LSTM decoder takes as input the first-layer temporal features, first-layer interaction features, and first-layer environmental features, and outputs the multimodal predicted trajectory. As shown in (A), assuming the target vehicle is vehicle A, the three predicted trajectories for vehicle A are a1, a2, and a3, which are the multimodal predicted trajectories. a1 represents vehicle A changing lanes left from its current position to another position in the future; a2 represents vehicle A moving forward from its current position to another position in the future; and a3 represents vehicle A changing lanes right from its current position to another position in the future. The inputs to the MLP sub-network layer are the first-layer temporal features, the first-layer interaction features, and the first-layer environmental features. The output is the driving lane probability, as shown in (A). The predicted trajectory a1 corresponds to lane A, and its corresponding driving lane probability is 80%, which means that the probability of the target vehicle (vehicle A) driving on lane A in the future is 80%. This means that the probability of vehicle A continuing to drive along lane A is 80%, that is, the driving lane probability corresponding to the multimodal predicted trajectory a1 of the target vehicle is 80%. The predicted trajectory a2 corresponds to lane B, and its corresponding driving lane probability is 20%, which means that the probability of vehicle A changing lanes from lane A to lane B in the future is 20%, that is, the driving lane probability corresponding to the multimodal predicted trajectory a2 of the target vehicle is 20%. The predicted trajectory a3 corresponds to the road green belt, and its corresponding driving lane probability is 0%, which means that the probability of vehicle A changing lanes from lane A to the road green belt in the future is 0%, that is, the driving lane probability corresponding to the multimodal predicted trajectory a3 of the target vehicle is 0%.

[0164] It should be noted that the first-layer LSTM decoder can use parameters of a binary Gaussian distribution. The predicted trajectories of the target vehicle (vehicle A) in three directions in (A) are collectively referred to as multimodal predicted trajectories. It should be noted that the parameters of the binary Gaussian distribution can capture the dynamic characteristics of the vehicle at different times and accurately characterize the vehicle's position and speed.

[0165] like Figure 6 As shown, the SNN sub-network layer, the first-layer GAT, and the first-layer LSTM encoder are all connected to the first-layer LSTM decoder. The CNN sub-network layer and the first-layer LSTM encoder are all connected to the first-layer GAT. The SNN sub-network layer, the first-layer GAT, and the first-layer LSTM encoder are all connected to the MLP sub-network layer. In the first-layer neural network model, the CNN sub-network layer, the SNN sub-network layer, the second-layer LSTM encoder, and the MLP sub-network layer are all connected to the second-layer neural network model.

[0166] This embodiment provides a vehicle trajectory prediction method. The multimodal prediction layer includes a first-layer Long Short-Term Memory (LSTM) decoder and a multilayer perceptron (MLP) sub-network layer. The first-layer multimodal data includes the multimodal predicted trajectory and the probability of driving lanes. When using the multimodal prediction layer to perform multimodal prediction on the first-layer interaction features and the first-layer feature data to obtain the first-layer multimodal data corresponding to the target vehicle, the method specifically includes: inputting the first-layer time features and the first-layer environmental features from the first-layer interaction features and the first-layer feature data into the first-layer LSTM decoder, and using the first-layer LSTM decoder to perform multimodal prediction on the first-layer time features, the first-layer interaction features, and the first-layer environmental features to obtain the multimodal predicted trajectory corresponding to the target vehicle; inputting the first-layer time features, the first-layer interaction features, and the first-layer environmental features into the MLP sub-network layer, and using the MLP sub-network layer to predict the probability of driving lanes corresponding to the first-layer time features, the first-layer interaction features, and the first-layer environmental features to obtain the probability of driving lanes corresponding to the target vehicle. In this embodiment, the first-layer LSTM decoder performs multimodal prediction to obtain the multimodal predicted trajectory, and the MLP sub-network layer performs driving lane probability prediction to obtain the driving lane probability, thereby obtaining the first-layer multimodal data. Since this embodiment uses different network layers to obtain the multimodal predicted trajectory and driving lane probability respectively, and this embodiment considers the multimodal predicted trajectory and driving lane probability from multiple aspects, the output result is accurate.

[0167] Example 4

[0168] This embodiment is a further refinement of any of the above embodiments. In this embodiment, the second-layer neural network model includes a second-layer feature extraction layer, a second-layer interaction relationship layer, and a single-modal prediction layer; the first-layer prediction data includes multimodal prediction trajectory, driving lane probability, first-layer spatial features, and first-layer environmental features.

[0169] Figure 8 This is a schematic flowchart of a vehicle trajectory prediction method provided in Embodiment 4 of this application. This embodiment is an optional method that uses a second-layer neural network model to perform second-layer feature extraction, second-layer interaction relationship calculation, and single-modal prediction on the first-layer prediction data and no-parking information to obtain the single-modal predicted trajectory corresponding to the target vehicle. Figure 8 As shown, the specific steps are as follows.

[0170] S801 uses a second-layer feature extraction layer to extract second-layer time features from the first-layer prediction data to obtain the second-layer time features of the target vehicle.

[0171] The second-layer time feature is the time feature extracted from the second-layer neural network model.

[0172] In one approach, before inputting the first-layer prediction data into the second-layer neural network model in the prediction device, the first-layer prediction data can be pre-processed, for example, by standardization, to obtain pre-processed first-layer prediction data. Then, the pre-processed first-layer prediction data is input into the second-layer neural network model for second-layer feature extraction, second-layer interaction calculation, and single-modal prediction.

[0173] S802, the second-layer time features, the first-layer spatial features, and the first-layer environmental features are input into the second-layer interaction relationship layer, and the second-layer interaction relationship layer is used to calculate the second-layer interaction relationship of the second-layer time features and the first-layer spatial features to obtain the second-layer interaction features corresponding to the target vehicle.

[0174] The second-layer interaction features refer to the interaction relationships and influence between the target vehicle and surrounding vehicles in the second-layer neural network model. The second-layer interaction features are calculated using the second-layer interaction relationship layer.

[0175] S803 inputs the second-layer time features, the second-layer interaction features, the first-layer environmental features, and the no-stopping information into the single-modal prediction layer, and uses the single-modal prediction layer to perform single-modal prediction on the second-layer time features, the second-layer interaction features, the first-layer environmental features, and the no-stopping information to obtain the single-modal predicted trajectory corresponding to the target vehicle.

[0176] The single-modal predicted trajectory refers to the final predicted trajectory output from the second-layer neural network model. There is only one single-modal predicted trajectory for the target vehicle, which is the single trajectory formed by the possible positional changes of the target vehicle within a certain period of time.

[0177] As shown in (B), there are 3 vehicles in the current prediction area. Assuming the target vehicle is vehicle A, its corresponding single-mode prediction trajectory is a1, where a1 represents that the target vehicle (vehicle A) will travel on lane A in the future. Therefore, vehicle A will travel from its current position to another position along lane A in the future.

[0178] Figure 9 This is a schematic diagram of a second-layer neural network model structure provided in Embodiment 4 of this application. Figure 9 As shown, the second feature extraction layer includes, from left to right, the second feature extraction layer, the second interaction relationship layer, and the single-modal prediction layer.

[0179] This embodiment provides a vehicle trajectory prediction method. The second-layer neural network model in this embodiment includes a second-layer feature extraction layer, a second-layer interaction relationship layer, and a single-modal prediction layer. The first-layer prediction data includes multimodal predicted trajectories, driving lane probabilities, first-layer spatial features, and first-layer environmental features. When using the second-layer neural network model to perform second-layer feature extraction, second-layer interaction relationship calculation, and single-modal prediction on the first-layer prediction data and no-stopping information to obtain the single-modal predicted trajectory corresponding to the target vehicle, the method specifically includes: using the second-layer feature extraction layer to perform second-layer temporal feature extraction on the first-layer prediction data to obtain the target vehicle's trajectory. The second-layer temporal features of the target vehicle are input into the second-layer interaction relationship layer, along with the first-layer spatial and environmental features. The second-layer interaction relationship layer is then used to calculate the second-layer interaction relationship between the second-layer temporal features and the first-layer spatial features to obtain the second-layer interaction features corresponding to the target vehicle. The second-layer temporal features, second-layer interaction features, first-layer environmental features, and no-stopping information are then input into the single-modal prediction layer. The single-modal prediction layer is then used to perform single-modal prediction on the second-layer temporal features, second-layer interaction features, first-layer environmental features, and no-stopping information to obtain the single-modal predicted trajectory corresponding to the target vehicle. This embodiment obtains the second-layer temporal features through the second-layer feature extraction layer and the second-layer interaction features through the second-layer interaction relationship layer. Then, a single-modal prediction layer is used to perform single-modal prediction on the second-layer temporal features, the second-layer interaction features, the first-layer environmental features, and the no-stopping information, thereby finally obtaining the single-modal predicted trajectory. In this embodiment, the single-modal prediction layer considers the no-stopping information as a constraint, which can identify potential dangers and take avoidance measures. Thus, the single-modal prediction layer will predict the trajectory beyond the no-stopping information, predicting a more optimized trajectory, selecting a suitable and realistic lane for driving, ensuring safety, and complying with traffic regulations. Therefore, the accuracy of the single-modal predicted trajectory is ultimately improved.

[0180] Example 5

[0181] This embodiment is a further refinement of any of the above embodiments. In this embodiment, the second feature extraction layer includes a second-layer Long Short-Term Memory (LSTM) encoder. This embodiment is an optional method for using the second-layer feature extraction layer to extract the second-layer temporal features from the first-layer prediction data. The specific details are as follows.

[0182] The multimodal predicted trajectory and driving lane probability from the first layer prediction data are input into the second layer LSTM encoder, and the second layer LSTM encoder is used to extract the second layer temporal features of the multimodal predicted trajectory and driving lane probability.

[0183] like Figure 9As shown, the second feature extraction layer includes a second LSTM encoder. It should be noted that the model parameters in the second LSTM encoder are superior to those in the first LSTM encoder. The input to the second LSTM encoder is the multimodal predicted trajectory and the driving lane probability, and the output is the second-layer temporal feature. The output of the multimodal prediction layer is also the multimodal predicted trajectory and the driving lane probability; therefore, the output of the multimodal prediction layer serves as the input to the second LSTM encoder. It should be noted that the model parameters in the second neural network model can be superior to those in the first neural network model.

[0184] This embodiment provides a vehicle trajectory prediction method. The second feature extraction layer in this embodiment includes a second-layer Long Short-Term Memory (LSTM) encoder. When using the second feature extraction layer to perform second-layer temporal feature extraction on the first-layer prediction data, specifically, it includes: inputting the multimodal predicted trajectory and driving lane probability from the first-layer prediction data into the second-layer LSTM encoder, and then using the second-layer LSTM encoder to perform second-layer temporal feature extraction on the multimodal predicted trajectory and driving lane probability. In this embodiment, the second feature extraction layer includes an LSTM encoder, which performs temporal feature extraction on the multimodal predicted trajectory and driving lane probability again. Since this embodiment performs feature extraction again on the output of the first-layer neural network model, the result of the second-layer temporal feature extraction is more accurate. The multimodal predicted trajectory and driving lane probability also connect the first-layer neural network model and the second-layer neural network model, thus realizing a hierarchical neural network design, which helps improve the accuracy of the final single-modal prediction result.

[0185] In one embodiment, the second interaction layer includes a second graph attention network (GAT); when the second GAT predicts that a collision will occur between the target vehicle and surrounding vehicles, it considers the high degree of interaction between the target vehicle and surrounding vehicles.

[0186] It should be noted that the second-layer GAT has the function of predicting whether the target vehicle will collide with surrounding vehicles. Figure 10 This is a collision diagram provided for Embodiment 5 of the application. For example... Figure 10 As shown, there are three lanes: lane 1, lane 2, and lane 3. There are two vehicles, vehicle C and vehicle D. Figure 10Four collision scenarios are marked (as shown by the dashed rectangles in the image). When vehicle C changes lanes to the left into the first lane, and vehicle D continues to change lanes to the left, successively changing lanes to the second and first lanes, a collision is likely to occur in the future. When vehicle C is moving forward in the second lane, and vehicle D changes lanes to the left into the second lane, a collision is likely to occur in the future. When vehicle C changes lanes to the right into the third lane, and vehicle D changes lanes to the left into the second lane or continues to move forward in the third lane, a collision is likely to occur in the future. Using vehicle C as the target vehicle and vehicle D as surrounding vehicles, when the second-layer GAT predicts a collision between vehicle C and vehicle D, it considers the high-level interaction between vehicle C and vehicle D when calculating the second-layer interaction features, resulting in a more accurate calculation of the second-layer interaction relationship.

[0187] This embodiment is an optional method for calculating the second-layer interaction relationship between the second-layer temporal features and the first-layer spatial features using a second-layer interaction relationship layer. The specific content is as follows.

[0188] The second-layer GAT is used to calculate the second-layer interaction relationship between the second-layer temporal features and the first-layer spatial features.

[0189] like Figure 9 As shown, the second interaction layer includes a second-layer GAT. The input to the second-layer GAT is the second-layer temporal features and the first-layer spatial features; the output is the second-layer interaction features.

[0190] This embodiment provides a vehicle trajectory prediction method. In this embodiment, the second-layer interaction relationship layer includes a second-layer Gaussian Atlas (GAT). When the second-layer GAT predicts a collision between a target vehicle and surrounding vehicles, it considers the high-level interaction between the colliding target vehicle and the surrounding vehicles. Specifically, when calculating the second-layer interaction relationship using the second-layer interaction relationship layer on the second-layer temporal features and the first-layer spatial features, it includes: using the second-layer GAT to calculate the second-layer interaction relationship on the second-layer temporal features and the first-layer spatial features. In this embodiment, when the second-layer GAT predicts a collision between a target vehicle and surrounding vehicles, it considers the high-level interaction between the colliding target vehicle and the surrounding vehicles. Because it considers the high-level interaction, the result of calculating the second-layer interaction relationship in the second-layer GAT is more accurate and closer to the actual situation.

[0191] In one embodiment, the single-modal prediction layer includes a second-layer Long Short-Term Memory (LSTM) decoder. This embodiment is an optional approach to using a single-modal prediction layer to perform single-modal prediction of second-layer temporal features, second-layer interaction features, first-layer environmental features, and no-stop information, as detailed below.

[0192] A second-layer LSTM decoder is used to perform single-modal prediction of the second-layer time features, the second-layer interaction features, the first-layer environmental features, and the no-stop information.

[0193] In this embodiment, the input to the second-layer LSTM decoder is the second-layer time feature, the second-layer interaction feature, the first-layer environment feature, and the no-stop information. The first-layer environment feature is the output of the SNN sub-network layer in the first-layer neural network model.

[0194] like Figure 9 As shown, the single-modal prediction layer includes a second-layer LSTM decoder. The second-layer LSTM encoder is connected to the second-layer GAT and the second-layer LSTM decoder, and the second-layer GAT is connected to the second-layer LSTM decoder. The first-layer neural network model is connected to the second-layer LSTM encoder, second-layer GAT, and second-layer LSTM decoder within the second-layer neural network model.

[0195] This embodiment provides a vehicle trajectory prediction method. The single-modal prediction layer in this embodiment includes a second-layer Long Short-Term Memory (LSTM) decoder. Specifically, when using the single-modal prediction layer to perform single-modal prediction on the second-layer temporal features, second-layer interaction features, first-layer environmental features, and no-stopping information, the method involves using the second-layer LSTM decoder to perform single-modal prediction on these features. In this embodiment, the second-layer LSTM decoder uses the second-layer temporal features, second-layer interaction features, first-layer environmental features, and no-stopping information for single-modal prediction. Since the second-layer temporal features and second-layer interaction features are extracted twice, their accuracy is high. Simultaneously, this embodiment considers no-stopping information, thereby better predicting the trajectory, optimizing the path, and predicting a suitable trajectory. Therefore, the output of the second-layer LSTM decoder results in a more accurate single-modal predicted trajectory.

[0196] Example 6

[0197] Figure 11 This is a schematic diagram of a hierarchical neural network model provided in Embodiment Six of this application. Figure 11 As shown, the prediction device obtains the initial trajectory prediction association data and parking restriction information corresponding to the target vehicle from at least one data platform. The initial trajectory prediction association data includes initial state information, initial traffic information, initial geographic information, and initial weather information. The data is preprocessed by the data processing module in the prediction device to obtain trajectory prediction processing data, which includes state time series data, traffic time series data, geographic time series data, geographic image data, and weather normalized data.

[0198] Furthermore, the state time series data, traffic time series data, and geographic time series data are input into the first-layer LSTM encoder to extract the first-layer time features and output the first-layer time features.

[0199] Furthermore, the geographic image data is input into the CNN sub-network layer. The workflow of the CNN sub-network layer is as follows: the geographic image data passes through the CNN input layer, convolutional layer, pooling layer and fully connected layer in sequence, and finally the spatial features are extracted to output the first layer of spatial features.

[0200] Furthermore, the normalized weather data is input into the SNN sub-network layer. The workflow of the SNN sub-network layer is as follows: obtain the normalized weather data, extract environmental features, and output the first layer of environmental features.

[0201] Furthermore, the first-layer temporal features and the first-layer spatial features are input into the first-layer GAT. The workflow of the first-layer GAT is as follows: obtain the first-layer temporal features and the first-layer spatial features, determine them as the first-layer node features, represent the first-layer interaction relationships, calculate the first-layer attention score, calculate the first-layer weighted node feature vector based on the first-layer attention score and the first-layer node features, determine the first-layer weighted node feature vector as the first-layer interaction features, and output the first-layer interaction features.

[0202] Furthermore, the first-layer interaction features, the first-layer time features, and the first-layer environment features are input into the first-layer LSTM decoder. The first-layer LSTM decoder performs multimodal prediction and outputs the multimodal predicted trajectory.

[0203] Furthermore, the first-layer interaction features, the first-layer time features, and the first-layer environmental features are input into the MLP sub-network layer to predict the driving lane probability and output the driving lane probability.

[0204] Furthermore, the multimodal predicted trajectory and driving lane probability are input into the second-layer LSTM encoder to extract the second-layer temporal features and obtain the second-layer temporal features.

[0205] Furthermore, the second-layer temporal features and the first-layer spatial features are input into the second-layer GAT. The workflow of the second-layer GAT is as follows: obtain the second-layer temporal features and the first-layer spatial features, determine them as second-layer node features, predict collisions, represent second-layer interaction relationships, calculate the second-layer attention score, calculate the second-layer weighted node feature vector based on the second-layer attention score and the second-layer node feature matrix, determine the second-layer weighted node feature vector as the second-layer interaction features, and output the second-layer interaction features.

[0206] Furthermore, the second-layer time features, second-layer interaction features, first-layer environmental features, and no-stop information are input into the second-layer LSTM decoder. The second-layer LSTM decoder performs single-mode prediction and outputs the single-mode predicted trajectory.

[0207] It should be noted that the no-parking information can be preprocessed using a data processing module.

[0208] It should be noted that the node features corresponding to the target vehicle change as it moves, thus the node features corresponding to the target vehicle are continuously updated in the first-layer GAT. The second-layer GAT works similarly, and will not be elaborated here.

[0209] It should be noted that the interaction between the target vehicle and surrounding vehicles can be represented using graph edges, where each edge represents the distance between vehicles, and the edge weight is the reciprocal of the distance between vehicles. As the vehicle moves, its corresponding graph edges and edge weights are continuously updated.

[0210] It should be noted that this application uses different network layers to process different types of data, making better use of the advantages of each network layer, thereby improving the accuracy of prediction.

[0211] Based on the above description, an LSTM encoder is used to extract temporal features from state time series data, traffic time series data, and geographic time series data; a CNN sub-network layer is used to extract spatial features from geographic image data; and an SNN sub-network layer is used to extract environmental features from weather normalized data.

[0212] Example 7

[0213] The following is an embodiment of the device. Figure 12 This application provides a schematic diagram of a vehicle trajectory prediction device according to Embodiment Seven. The prediction device 120 includes the following modules:

[0214] The acquisition module 1201 is used to acquire initial trajectory prediction association data and no-parking information corresponding to the target vehicle from at least one data platform; wherein, the initial trajectory prediction association data includes initial state information, initial traffic information, initial geographic information and initial weather information;

[0215] The classification module 1202 is used to preprocess the initial trajectory prediction associated data to obtain trajectory prediction processed data;

[0216] The first input module 1203 is used to input trajectory prediction processing data into the first layer neural network model. The first layer neural network module 1204 is used to perform first layer feature extraction, first layer interaction relationship calculation and multimodal prediction on the trajectory prediction processing data using the first layer neural network model, so as to obtain the first layer prediction data corresponding to the target vehicle.

[0217] The second input module 1205 is used to input the first layer prediction data into the second layer neural network model. The second layer neural network module 1206 is used to perform second layer feature extraction, second layer interaction relationship calculation and single-modal prediction on the first layer prediction data and parking restriction information using the second layer neural network model, so as to obtain the single-modal prediction trajectory corresponding to the target vehicle.

[0218] In one embodiment, the first-layer neural network model includes a first-layer feature extraction layer, a first-layer interaction relationship layer, and a multimodal prediction layer; the first-layer neural network module 1204, when using the first-layer neural network model to perform first-layer feature extraction, first-layer interaction relationship calculation, and multimodal prediction on trajectory prediction processing data to obtain first-layer prediction data corresponding to the target vehicle, is specifically used for:

[0219] A first-layer feature extraction layer is used to extract first-layer features from the trajectory prediction data to obtain first-layer feature data corresponding to the target vehicle. The first-layer feature data includes first-layer temporal features, first-layer spatial features, and first-layer environmental features. A first-layer interaction relationship layer is used to calculate the interaction relationship of the first-layer feature data to obtain the first-layer interaction features corresponding to the target vehicle. A multimodal prediction layer is used to perform multimodal prediction on the first-layer interaction features and the first-layer feature data to obtain the first-layer multimodal data corresponding to the target vehicle.

[0220] In one approach, the first feature extraction layer includes a first-layer Long Short-Term Memory (LSTM) encoder, a Convolutional Neural Network (CNN) sub-network layer, and a Shallow Neural Network (SNN) sub-network layer; the trajectory prediction processing data includes state time-series data, traffic time-series data, geographic time-series data, geographic image data, and weather normalized data; the first-layer neural network module 1204, when performing first-layer feature extraction on the trajectory prediction processing data using the first-layer feature extraction layer to obtain the first-layer feature data corresponding to the target vehicle, is specifically used for:

[0221] State time series data, traffic time series data, and geographic time series data are input into the first-layer LSTM encoder, and the first-layer LSTM encoder is used to extract the first-layer time features of the state time series data, traffic time series data, and geographic time series data to obtain the first-layer time features corresponding to the target vehicle. Geographic image data is input into the CNN sub-network layer, and the CNN sub-network layer is used to extract the spatial features of the geographic image data to obtain the first-layer spatial features corresponding to the target vehicle. Weather normalized data is input into the SNN sub-network layer, and the SNN sub-network layer is used to extract the environmental features of the weather normalized data to obtain the first-layer environmental features corresponding to the target vehicle.

[0222] In one approach, the first interaction layer includes a first-layer graph attention network (GAT); the first-layer GAT considers the interactions between the target vehicle and surrounding vehicles; the first-layer neural network module 1204, when using the first-layer interaction layer to calculate the interaction relationships of the first-layer feature data to obtain the first-layer interaction features corresponding to the target vehicle, is specifically used for:

[0223] The first-layer temporal features and the first-layer spatial features are input into the first-layer GAT. The first-layer GAT is used to calculate the interaction relationship between the first-layer temporal features and the first-layer spatial features to obtain the first-layer interaction features corresponding to the target vehicle.

[0224] In one approach, the multimodal prediction layer includes a first-layer Long Short-Term Memory (LSTM) decoder and a multilayer perceptron (MLP) sub-network layer; the first-layer multimodal data includes multimodal predicted trajectories and driving lane probabilities; the first-layer neural network module 1204, when using the multimodal prediction layer to perform multimodal prediction on the first-layer interaction features and the first-layer feature data to obtain the first-layer multimodal data corresponding to the target vehicle, is specifically used for:

[0225] The first-layer interaction features, as well as the first-layer temporal features and first-layer environmental features from the first-layer feature data, are input into the first-layer LSTM decoder. The first-layer LSTM decoder is then used to perform multimodal prediction on the first-layer temporal features, first-layer interaction features, and first-layer environmental features to obtain the multimodal predicted trajectory corresponding to the target vehicle. The first-layer temporal features, first-layer interaction features, and first-layer environmental features are then input into the MLP sub-network layer. The MLP sub-network layer is then used to perform driving lane probability prediction on the first-layer temporal features, first-layer interaction features, and first-layer environmental features to obtain the driving lane probability corresponding to the target vehicle.

[0226] In one approach, the second-layer neural network model includes a second-layer feature extraction layer, a second-layer interaction relationship layer, and a single-modal prediction layer; the first-layer prediction data includes multimodal prediction trajectories, driving lane probabilities, first-layer spatial features, and first-layer environmental features; the second-layer neural network module 1206, when using the second-layer neural network model to perform second-layer feature extraction, second-layer interaction relationship calculation, and single-modal prediction on the first-layer prediction data and no-stopping information to obtain the single-modal prediction trajectory corresponding to the target vehicle, is specifically used for:

[0227] A second-layer feature extraction layer is used to extract second-layer temporal features from the first-layer prediction data to obtain the second-layer temporal features of the target vehicle. The second-layer temporal features, first-layer spatial features, and first-layer environmental features are input into the second-layer interaction relationship layer, and the second-layer interaction relationship layer is used to calculate the second-layer interaction relationship between the second-layer temporal features and the first-layer spatial features to obtain the second-layer interaction features corresponding to the target vehicle. The second-layer temporal features, second-layer interaction features, first-layer environmental features, and no-stopping information are input into the single-modal prediction layer, and the single-modal prediction layer is used to perform single-modal prediction on the second-layer temporal features, second-layer interaction features, first-layer environmental features, and no-stopping information to obtain the single-modal predicted trajectory corresponding to the target vehicle.

[0228] In one embodiment, the second feature extraction layer includes a second-layer Long Short-Term Memory (LSTM) encoder; the second-layer neural network module 1206, when performing second-layer temporal feature extraction on the first-layer prediction data using the second-layer feature extraction layer, is specifically used for:

[0229] The multimodal predicted trajectory and driving lane probability from the first layer prediction data are input into the second layer LSTM encoder, and the second layer LSTM encoder is used to extract the second layer temporal features of the multimodal predicted trajectory and driving lane probability.

[0230] The second interaction layer includes a second-layer graph attention network (GAT). When the second-layer GAT predicts a collision between the target vehicle and surrounding vehicles, it considers the high-level interaction between the target vehicle and the surrounding vehicles. The second-layer neural network module 1206, when using the second-layer interaction layer to calculate the second-layer interaction relationship based on the second-layer temporal features and the first-layer spatial features, is specifically used for:

[0231] A second-layer GAT is used to calculate the second-layer interaction relationship between the second-layer temporal features and the first-layer spatial features;

[0232] The single-modal prediction layer includes a second-layer Long Short-Term Memory (LSTM) decoder; the second-layer neural network module 1206, when using the single-modal prediction layer to perform single-modal prediction on the second-layer temporal features, second-layer interaction features, first-layer environmental features, and no-stop information, is specifically used for:

[0233] A second-layer LSTM decoder is used to perform single-modal prediction of the second-layer time features, the second-layer interaction features, the first-layer environmental features, and the no-stop information.

[0234] Example 8

[0235] Figure 13 This is a schematic diagram of the structure of an electronic device provided in Embodiment 8 of this application. Figure 13As shown, the electronic device 130 may include a processor 1301 and a memory 1302 communicatively connected to the processor 1301. The memory 1302 stores computer-executable instructions; the processor 1301 executes the computer-executable instructions stored in the memory 1302 to implement any one of the method embodiments 1 to 6 described above. The specific implementation methods and technical effects are similar and will not be repeated here.

[0236] In this embodiment, the memory 1302 and the processor 1301 are connected via a bus. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. The bus can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 13 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0237] Example 9

[0238] This application provides a computer-readable storage medium storing computer-executable instructions. When executed by a processor, the computer-executable instructions are used to implement any one of the method embodiments 1 to 6 above. The specific implementation methods and technical effects are similar, and will not be repeated here.

[0239] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this application are indicated by the following claims.

[0240] It should be understood that this application is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this application is limited only by the appended claims.

Claims

1. A vehicle trajectory prediction method, characterized in that, The method includes: The initial trajectory prediction association data and parking restriction information corresponding to the target vehicle are obtained from at least one data platform; wherein, the initial trajectory prediction association data includes initial state information, initial traffic information, initial geographic information and initial weather information; The initial trajectory prediction associated data is preprocessed to obtain trajectory prediction processed data; The trajectory prediction processing data is input into a first-layer neural network model, which includes a first-layer feature extraction layer, a first-layer interaction relationship layer, and a multimodal prediction layer. The trajectory prediction processing data is subjected to the first layer feature extraction layer to extract the first layer features, so as to obtain the first layer feature data corresponding to the target vehicle; the first layer feature data includes the first layer temporal features, the first layer spatial features and the first layer environmental features. The interaction relationship is calculated on the first layer feature data using the first layer interaction relationship layer to obtain the first layer interaction features corresponding to the target vehicle. The multimodal prediction layer is used to perform multimodal prediction on the first layer interaction features and the first layer feature data to obtain the first layer multimodal data corresponding to the target vehicle; the first layer multimodal data includes the multimodal predicted trajectory and the driving lane probability; the first layer prediction data includes the multimodal predicted trajectory, the driving lane probability, the first layer spatial features, and the first layer environmental features; The first layer prediction data is input into the second layer neural network model, which includes a second layer feature extraction layer, a second layer interaction relationship layer, and a single-modal prediction layer. The second feature extraction layer is used to extract the second time feature from the first prediction data to obtain the second time feature of the target vehicle. The second-layer time features, the first-layer spatial features, and the first-layer environmental features are input into the second-layer interaction relationship layer, and the second-layer interaction relationship layer is used to calculate the second-layer interaction relationship of the second-layer time features and the first-layer spatial features to obtain the second-layer interaction features corresponding to the target vehicle. The second-layer time features, the second-layer interaction features, the first-layer environmental features, and the no-stopping information are input into the single-modal prediction layer, and the single-modal prediction layer is used to perform single-modal prediction on the second-layer time features, the second-layer interaction features, the first-layer environmental features, and the no-stopping information to obtain the single-modal predicted trajectory corresponding to the target vehicle.

2. The method according to claim 1, characterized in that, The first feature extraction layer includes a first-layer Long Short-Term Memory (LSTM) encoder, a Convolutional Neural Network (CNN) sub-network layer, and a Shallow Neural Network (SNN) sub-network layer; the trajectory prediction processing data includes state time series data, traffic time series data, geographic time series data, geographic image data, and weather normalized data. The step of using the first feature extraction layer to perform first-layer feature extraction on the trajectory prediction processing data to obtain first-layer feature data corresponding to the target vehicle includes: The state time series data, the traffic time series data, and the geographic time series data are input into the first layer LSTM encoder, and the first layer LSTM encoder is used to extract the first layer time features of the state time series data, the traffic time series data, and the geographic time series data to obtain the first layer time features corresponding to the target vehicle. The geographic image data is input into the CNN sub-network layer, and the CNN sub-network layer is used to extract spatial features from the geographic image data to obtain the first layer of spatial features corresponding to the target vehicle. The normalized weather data is input into the SNN sub-network layer, and the SNN sub-network layer is used to extract environmental features from the normalized weather data to obtain the first layer of environmental features corresponding to the target vehicle.

3. The method according to claim 2, characterized in that, The first interaction layer includes a first-layer graph attention network (GAT); the first-layer GAT considers the interaction between the target vehicle and surrounding vehicles. The step of using the first layer of interaction relationship to calculate the interaction relationship of the first layer of feature data to obtain the first layer of interaction features corresponding to the target vehicle includes: The first-layer temporal features and the first-layer spatial features are input into the first-layer GAT. The first-layer GAT is used to calculate the interaction relationship between the first-layer temporal features and the first-layer spatial features to obtain the first-layer interaction features corresponding to the target vehicle.

4. The method according to claim 3, characterized in that, The multimodal prediction layer includes a first layer of Long Short-Term Memory (LSTM) network decoder and a multilayer perceptron (MLP) sub-network layer. The step of using the multimodal prediction layer to perform multimodal prediction on the first layer interaction features and the first layer feature data to obtain the first layer multimodal data corresponding to the target vehicle includes: The first layer interaction features, the first layer time features, and the first layer environment features in the first layer feature data are input into the first layer LSTM decoder, and the first layer LSTM decoder is used to perform multimodal prediction on the first layer time features, the first layer interaction features, and the first layer environment features to obtain the multimodal predicted trajectory corresponding to the target vehicle. The first layer time features, the first layer interaction features, and the first layer environment features are input into the MLP sub-network layer, and the MLP sub-network layer is used to predict the driving lane probability of the first layer time features, the first layer interaction features, and the first layer environment features to obtain the driving lane probability corresponding to the target vehicle.

5. The method according to claim 1, characterized in that, The second feature extraction layer includes a second-layer Long Short-Term Memory (LSTM) encoder. The step of using the second feature extraction layer to extract second-layer temporal features from the first-layer prediction data includes: The multimodal predicted trajectory and the driving lane probability in the first layer prediction data are input into the second layer LSTM encoder, and the second layer LSTM encoder is used to extract the second layer temporal features of the multimodal predicted trajectory and the driving lane probability. The second layer of interaction relationship includes a second layer graph attention network (GAT); when the second layer GAT predicts that the target vehicle will collide with surrounding vehicles, it considers the high degree of interaction between the target vehicle and surrounding vehicles. The step of using the second layer of interaction relationships to calculate the second layer of interaction relationships between the second layer of temporal features and the first layer of spatial features includes: The second-layer GAT is used to calculate the second-layer interaction relationship between the second-layer temporal features and the first-layer spatial features; The single-modal prediction layer includes a second-layer Long Short-Term Memory (LSTM) decoder. The step of using the single-modal prediction layer to perform single-modal prediction on the second-layer temporal features, the second-layer interaction features, the first-layer environmental features, and the no-stopping information includes: The second-layer LSTM decoder is used to perform single-modal prediction on the second-layer time features, the second-layer interaction features, the first-layer environmental features, and the no-stop information.

6. A vehicle trajectory prediction device, characterized in that, The device includes: The acquisition module is used to acquire initial trajectory prediction association data and no-parking information corresponding to the target vehicle from at least one data platform; wherein, the initial trajectory prediction association data includes initial state information, initial traffic information, initial geographic information, and initial weather information; The classification module is used to preprocess the initial trajectory prediction associated data to obtain trajectory prediction processed data; The first input module is used to input the trajectory prediction processing data into the first layer neural network model, which includes a first layer feature extraction layer, a first layer interaction relationship layer, and a multimodal prediction layer. The trajectory prediction processing data is subjected to the first layer feature extraction layer to extract the first layer features, so as to obtain the first layer feature data corresponding to the target vehicle; the first layer feature data includes the first layer temporal features, the first layer spatial features and the first layer environmental features. The interaction relationship is calculated on the first layer feature data using the first layer interaction relationship layer to obtain the first layer interaction features corresponding to the target vehicle. The multimodal prediction layer is used to perform multimodal prediction on the first layer interaction features and the first layer feature data to obtain the first layer multimodal data corresponding to the target vehicle; the first layer multimodal data includes the multimodal predicted trajectory and the driving lane probability; the first layer prediction data includes the multimodal predicted trajectory, the driving lane probability, the first layer spatial features, and the first layer environmental features; The second input module is used to input the first layer prediction data into the second layer neural network model, which includes a second layer feature extraction layer, a second layer interaction relationship layer, and a single-modal prediction layer. The second feature extraction layer is used to extract the second time feature from the first prediction data to obtain the second time feature of the target vehicle. The second-layer time features, the first-layer spatial features, and the first-layer environmental features are input into the second-layer interaction relationship layer, and the second-layer interaction relationship layer is used to calculate the second-layer interaction relationship of the second-layer time features and the first-layer spatial features to obtain the second-layer interaction features corresponding to the target vehicle. The second-layer time features, the second-layer interaction features, the first-layer environmental features, and the no-stopping information are input into the single-modal prediction layer, and the single-modal prediction layer is used to perform single-modal prediction on the second-layer time features, the second-layer interaction features, the first-layer environmental features, and the no-stopping information to obtain the single-modal predicted trajectory corresponding to the target vehicle.

7. An electronic device, comprising: A processor, and a memory communicatively connected to the processor; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory to implement the method as described in any one of claims 1-5.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the method as described in any one of claims 1-5.

Images