Multi-vehicle cooperative trajectory prediction method and system based on federated learning

Abstract

The invention discloses a multi-vehicle cooperative trajectory prediction method and system based on federated learning, and relates to the technical field of intelligent traffic and artificial intelligence crossing. The method comprises the steps that each vehicle collects and preprocesses historical trajectory data of the vehicle, a trajectory coding module is trained through comparison learning tasks, and semantic feature vectors with driving intention discrimination are generated; semantic feature vectors are exchanged among vehicles through communication, a plurality of relational graphs are constructed based on spatial distance and semantic similarity, message passing and feature aggregation are performed by using a multi-relational graph attention mechanism, and vehicle representation fused with context information is generated; inputting the vehicle representation into a trajectory decoding module, and outputting a future trajectory prediction result; and each vehicle uploads the local model parameters to a central server, and the server performs weighted aggregation according to the vehicle contribution degree, generates global model parameters, and issues and updates the global model parameters. According to the invention, on the premise of protecting data privacy, high-precision multi-vehicle cooperative trajectory prediction is realized.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of intelligent transportation and artificial intelligence, in particular to a multi-vehicle cooperative trajectory prediction method and system based on federated learning. BACKGROUND

[0002] In the field of intelligent transportation and autonomous driving, vehicle trajectory prediction is a key technology for realizing safe decision-making and cooperative control. Traditional methods mostly use centralized learning frameworks, relying on roadside units or cloud servers to collect raw trajectory data from each vehicle for unified modeling. However, this approach has a serious risk of privacy leakage and is highly dependent on communication bandwidth and centralized infrastructure, making it difficult to apply to large-scale, decentralized inter-vehicle cooperative scenarios.

[0003] In recent years, some research has attempted to introduce a federated learning framework for distributed model training to protect data privacy. However, existing federated trajectory prediction methods usually only train independently based on local data, lacking effective modeling of neighboring vehicle behavior, resulting in insufficient cooperative perception capabilities. In addition, most methods use simple spatial adjacency relationships to construct interaction graphs, making it difficult to capture complex semantic-level interactions between vehicles, and do not introduce a self-supervised mechanism in local training to improve feature representation quality, resulting in poor generalization ability of the model in data heterogeneous scenarios.

[0004] Publication No. CN116595478A, entitled "Interactive Vehicle Multi-modal Trajectory Prediction Method Based on GRU-GCN", proposes extracting vehicle dynamics features through a GRU encoder, modeling temporal and spatial interactions between vehicles using a graph convolution network (GCN), and combining a behavior recognition module to output multi-modal predicted trajectories. However, this method relies on a centralized server for model training, which poses a risk of privacy leakage, and does not introduce a federated learning mechanism, making it impossible to achieve multi-vehicle cooperative perception and model updating while protecting data privacy.

[0005] Publication No. CN118430256A, entitled "Vehicle Trajectory Prediction Method Based on Air Computing and Federated Learning", proposes using air computing technology to achieve concurrent uploading and aggregation of multi-vehicle model gradients through signal superposition effects, significantly reducing the communication latency of federated learning. However, this method does not consider real-time interactions between vehicles during local training, relying solely on a central base station for model aggregation, lacking deep perception and cooperative modeling of neighboring vehicle behavior intentions, resulting in insufficient cooperative reasoning capabilities of the prediction model in complex traffic scenarios.

[0006] Therefore, how to effectively integrate multi-vehicle semantic features and model diverse interaction relationships without sharing raw data, and improve the accuracy of cooperative trajectory prediction under the federated learning framework, remains a technical challenge that needs to be addressed. SUMMARY

[0007] To solve the above technical problems, one technical solution adopted by the present application is to provide a multi-vehicle cooperative trajectory prediction method based on federated learning, which comprises: S1: Collect and store the historical trajectory data of the target vehicle, including position, speed, acceleration, heading angle and timestamp information, normalize each dimension of the historical trajectory data by using the Z-score standardization method, and divide the standardized historical trajectory data into trajectory sequences of fixed time windows Each target vehicle locally maintains a cooperative prediction model comprising a trajectory encoding module, a graph interaction fusion module and a trajectory decoding module; S2: Design a contrastive learning task, input the trajectory sequence into the trajectory encoding module to obtain the semantic feature vector of the trajectory, and train the trajectory encoding module by minimizing the contrastive learning loss function; S3: The target vehicle communicates with the neighbor vehicles to obtain the semantic feature vectors of the neighbor vehicle trajectories, calculates the distance between each vehicle, defines three graphs through the distance and the semantic feature vector similarity relationship, and combines the multi-relation graph attention mechanism in the graph interaction fusion module to perform message passing and feature aggregation on the three graphs to generate vehicle representations; S4: Input the vehicle representation into the trajectory decoding module to obtain the trajectory prediction sequence of the target vehicle and the neighbor vehicles in the future Combine the prediction loss function and the contrastive learning loss function, and jointly update the cooperative prediction model parameters of the target vehicle through the back propagation algorithm; S5: The participating vehicles upload the locally trained cooperative prediction model parameters to the central server, the central server aggregates the global cooperative prediction model parameters according to the contribution of each participating vehicle, and replaces the local cooperative prediction model parameters of each participating vehicle with the global cooperative prediction model parameters; S6: Repeat S1-S5 until the global cooperative prediction model converges to obtain the final cooperative prediction model; S7: In the model deployment stage, the target vehicle obtains its real-time motion state through the vehicle-mounted sensor to form a trajectory sequence, extracts the semantic feature vector by using the trajectory encoding module of the cooperative prediction model, then obtains the semantic feature vectors of the neighbor vehicle trajectories, obtains the vehicle representation through the graph interaction fusion module, and finally outputs the trajectory prediction sequence of the target vehicle and the neighbor vehicles through the trajectory decoding module.

[0008] Further, the Z-score standardization method is used to normalize each dimension of the historical trajectory data, which comprises: The position is divided into two dimensions of longitude and latitude, the speed, acceleration and heading angle are one dimension respectively, and the five dimensions are processed respectively and represented as The normalization process thereof is: , wherein, is the standardized feature value of the historical trajectory data feature dimension , is the original feature value of the historical trajectory data feature dimension , is the mean value of the historical trajectory data feature dimension , is the standard deviation of the historical trajectory data feature dimension , represents the historical trajectory data feature dimension, .

[0009] Further, the contrastive learning task comprises: Taking a trajectory sequence of a target vehicle as an anchor sample, generating a positive sample of the anchor sample through time clipping, constructing a training batch from multiple target vehicle trajectory sequences, and taking other trajectory sequences in the batch except the anchor sample as negative samples of the anchor sample; By minimizing the contrastive learning loss function, the trajectory encoding module learns to obtain a semantic feature vector with driving intention discriminability.

[0010] Further, the time clipping comprises: Randomly selecting a starting time step and an ending time step from the anchor sample trajectory sequence , satisfying and , intercepting a subsequence as a clipped trajectory segment, is the length of the clipped trajectory segment; For each time step , , the length of the clipped trajectory segment is restored to a fixed time window through a padding mechanism: For , the original data is retained, for the missing frames in the front part of , a forward extension strategy is adopted to copy the starting frame to fill in, for the missing frames in the rear part of , the terminal frame is copied to fill in.

[0011] Further, the contrastive learning loss function comprises: The cosine similarity between the anchor sample and its positive sample is taken as a positive term, and the cosine similarity between the anchor sample and multiple negative samples is taken as a negative term; The positive term and the negative term are respectively subjected to exponential weighting with a natural constant as a base and a temperature parameter as an index of the denominator; All weighted terms are normalized by a normalized exponential function to obtain the relative probability of the anchor sample and the positive sample; The negative logarithm of the relative probability is minimized as a contrastive learning loss function to train the trajectory encoding module; The temperature parameter satisfies .

[0012] Further, the trajectory encoding module comprises: The trajectory encoding module comprises an encoder composed of two layers of stacked long short-term memory networks (LSTM) for processing anchor sample trajectory sequences and positive sample trajectory sequences , respectively, and a fixed time window, a number of feature dimensions of the trajectory sequence, . For each input sequence, the hidden state of the long short-term memory network at each time step is obtained to form a sequence of hidden vectors with a length of the fixed time window . The hidden state sequence is flattened into a one-dimensional vector in the time dimension, and is mapped into a semantic feature vector with a fixed dimension by a fully connected layer, wherein is a preset semantic feature vector length.

[0013] Further, the target vehicle communicates with neighbor vehicles, comprising: The target vehicle takes the position at the end time of the anchor sample trajectory sequence as a reference point to calculate the Euclidean distance with surrounding vehicles at the same time, and based on the Euclidean distance, candidate neighboring vehicles with a distance less than are selected within the set maximum perception and communication range , and the K closest vehicles are selected as neighbor vehicles, and if the number of candidate neighboring vehicles is less than K, a zero vector with the same dimension as the semantic feature vector is used to replace the missing vehicle feature input.

[0014] Further, the three graphs are defined by distance and semantic feature vector similarity, comprising: The first graph is a connection graph based on spatial proximity : calculate the reciprocal of the Euclidean distance between vehicles as the initial weight, and normalize the initial weight of each vehicle, if the normalized weight exceeds the preset threshold , a connection is established between the two vehicles, indicating a spatial connectivity relationship; The second graph is a connection graph based on semantic behavior similarity : calculate the cosine similarity of the semantic feature vectors between vehicles as the initial weight, and normalize it, if the normalized weight exceeds the preset threshold , a connection is established, indicating a semantic connectivity relationship; The third graph is a comprehensive connection graph based on space-semantic fusion : The corresponding normalized weights in the first two graphs are weighted and summed according to the preset proportion to obtain the fusion weight, and the fusion weight is normalized again; if the fusion weight exceeds the preset threshold , a connection is established, indicating a comprehensive interactive connectivity relationship.

[0015] Further, the multi-relation graph attention mechanism in the graph interaction fusion module comprises: The connection graph based on spatial proximity , the connection graph based on semantic behavior similarity , and the comprehensive connection graph based on space-semantic fusion , respectively, construct independent graph attention layers; In each graph attention layer, for any vehicle node in the graph , its semantic feature vector is used as the query vector after a learnable linear transformation, and the semantic feature vectors of all neighbor nodes connected to it in the relationship graph are used as the key vector and the value vector after a learnable linear transformation, respectively. The attention weight is calculated as follows: , wherein represents the type of relationship graph, is the attention weight of the vehicle node and its neighbor node in the relationship graph , and and are the semantic feature vectors of the vehicle node and the neighbor node , respectively, is the corresponding learnable projection matrix in the relationship graph , and is the vector dimension of , normalized along the neighbor node dimension For Relationship diagram with vehicle nodes The set of connected neighbor nodes; Based on the calculated attention weights, the semantic feature vectors of neighbor nodes are weighted and aggregated to obtain the first Vehicle nodes under the relationship Contextual representation of: , in, For Vehicle nodes in the relationship graph Context representation, For the relationship Vehicle nodes in the figure Its neighboring nodes The attention weight, For Relationship diagram with vehicle nodes The set of connected neighbor nodes, is the projection matrix of the value vector, Neighbor nodes Semantic feature vector of The context representation obtained under the three relationship diagrams Splice and generate vehicle nodes through an output projection layer Vehicle characterization : , in, Vehicle node Vehicle characterization, represents vector concatenation, is the output projection matrix, It is the context representation obtained under the three relationship diagrams.

[0016] Furthermore, the trajectory decoding module includes: The trajectory decoding module includes a feedforward neural network composed of multiple fully connected layers, which takes the vehicle representation of each vehicle node output in step S3 as input, processes it layer by layer through linear transformation and ReLU activation function, and finally outputs a elements of the prediction vector; In the prediction vector, each two consecutive elements represent the future The predicted longitude and latitude coordinates of the time steps are arranged in chronological order to form the future coordinates of the target vehicle. A trajectory sequence of time steps.

[0017] Furthermore, the combination of the prediction loss function and the contrastive learning loss function includes: define the prediction loss function for measuring the difference between the model-predicted trajectory and the true future trajectory , using the average of the squared Euclidean distance at each time step as the loss: , where is the prediction loss function, is the length of the time steps of the predicted trajectory, is the position coordinate of the true trajectory at the th time step, is the position coordinate of the predicted trajectory at the th time step, is the squared Euclidean distance between the predicted point and the true point; weight the prediction loss function and the contrastive learning loss function to construct a comprehensive loss function: , where is an adjustable hyperparameter for balancing the trajectory prediction accuracy and the quality of feature representation, is the comprehensive loss function, is the prediction loss function, is the contrastive learning loss function; based on the comprehensive loss function , the gradient is calculated by the backpropagation algorithm, and the Adam optimizer is used to jointly update all trainable parameters of the trajectory encoding module, the graph interaction fusion module, and the trajectory decoding module.

[0018] Further, the weighted aggregation according to the contribution degree of each participating vehicle includes: The central server takes the number of local historical trajectory data used by each participating vehicle in the current round of training as the measure of its contribution degree to calculate the aggregation weight; For the set of vehicles participating in the current round of training , the aggregation weight of vehicle is determined by the ratio of its local historical trajectory data quantity to the sum of the total data quantities of all participating vehicles: , where represents the number of trajectory sequences of vehicle used for global collaborative prediction model update in the current round of training, represents the number of trajectory sequences of vehicle used for global collaborative prediction model update in the current round of training, a vehicle aggregated weights used for global collaborative prediction model update in this round of training, a set of vehicles participating in this round of training; the central server calculates the aggregated weights based on the calculated weights , performs a weighted average on the local collaborative prediction model parameters uploaded by all participating vehicles to generate new global collaborative prediction model parameters : , wherein, to generate new global collaborative prediction model parameters, a vehicle aggregated weights used for global collaborative prediction model update in this round of training, a vehicle uploaded local collaborative prediction model parameters, a set of vehicles participating in this round of training.

[0019] Further, a multi-vehicle collaborative trajectory prediction system based on federated learning comprises: a data acquisition and preprocessing module: used for acquiring historical trajectory data of the vehicle itself, and adopting a Z-score standardization method for normalization processing, and dividing into trajectory sequences of fixed time windows; a trajectory encoding calculation module: used for locally performing a contrast learning task to encode the input trajectory sequence into a semantic feature vector with driving intention discriminability; a graph interaction fusion calculation module: used for obtaining semantic feature vectors of surrounding vehicles through inter-vehicle communication, constructing a multi-relation graph based on spatial distance and semantic similarity, and performing message passing and feature aggregation by using a multi-relation graph attention mechanism to generate vehicle representation; a trajectory decoding calculation module: used for decoding the fused vehicle representation into a position sequence of the target vehicle and neighbor vehicles in the future time steps; a model aggregation and update module: used for uploading the collaborative prediction model parameters after local training to a central server, and receiving global collaborative prediction model parameters issued by the central server to replace the local model for the next round of training; wherein, the central server is independently configured, used for receiving collaborative prediction model parameters uploaded by each participating vehicle, calculating aggregated weights according to the local data volume of each vehicle, and performing a weighted average on the collaborative prediction model parameters to generate updated global collaborative prediction model parameters and issue them to each participating vehicle. The beneficial effects of the present application are:

[0020] ​The application realizes high-precision prediction of the future trajectory of a target vehicle under the premise of not sharing original trajectory data by performing self-supervised training based on contrast learning on a local vehicle and combining a multi-relation graph attention mechanism to fuse context features, and balances data privacy protection and collaborative perception performance, and is suitable for a federated learning scene in a multi-vehicle collaborative environment. BRIEF DESCRIPTION OF DRAWINGS

[0021] Figure 1 A flowchart of a multi-vehicle collaborative trajectory prediction method based on federated learning provided by an embodiment of the application is shown.

[0022] Figure 2 An architectural diagram of a multi-vehicle collaborative trajectory prediction method based on federated learning provided by an embodiment of the application is shown.

[0023] Figure 3 A module diagram of a multi-vehicle collaborative trajectory prediction system based on federated learning provided by an embodiment of the application is shown. DETAILED DESCRIPTION

[0024] The preferred embodiments of the application are described in detail below with reference to the accompanying drawings, so that the advantages and features of the application can be more easily understood by those skilled in the art, and the protection scope of the application can be more clearly defined.

[0025] In the following description, many specific details are set forth in order to provide a thorough understanding of the application, but the application can also be practiced without the specific details described herein; obviously, the examples in the specification are only some of the embodiments of the application, not all.

[0026] Figure 1 is a flowchart of a multi-vehicle collaborative trajectory prediction method based on federated learning provided by an embodiment of the application, and the method comprises: S1: Collect and store historical trajectory data of a target vehicle, including position, speed, acceleration, heading angle and timestamp information, normalize each dimension of the historical trajectory data by using a Z-score standardization method, divide the standardized historical trajectory data into trajectory sequences of fixed time windows Each target vehicle locally maintains a collaborative prediction model comprising a trajectory encoding module, a graph interaction fusion module and a trajectory decoding module; S2: Design a contrast learning task, input the trajectory sequence into the trajectory encoding module to obtain a semantic feature vector of the trajectory, and train the trajectory encoding module by minimizing a contrast learning loss function; S3: The target vehicle communicates with neighboring vehicles to obtain the semantic feature vectors of their trajectories, calculate the distances between each vehicle, and define three graphs based on the similarity between distances and semantic feature vectors. Combined with the multi-relational graph attention mechanism in the graph interaction fusion module, message passing and feature aggregation are performed on the three graphs to generate a vehicle representation. S4: Input the vehicle representation into the trajectory decoding module to obtain the future trajectory of the target vehicle and neighboring vehicles. The trajectory prediction sequence of time steps is combined with the prediction loss function and the contrastive learning loss function to jointly update the target vehicle's collaborative prediction model parameters through the back propagation algorithm; S5: The participating vehicles upload the locally trained collaborative prediction model parameters to the central server. The central server performs weighted aggregation based on the contribution of each participating vehicle to generate a global collaborative prediction model. The global collaborative prediction model parameters are distributed to each participating vehicle, replacing their local collaborative prediction model parameters. S6: Repeat S1-S5 until the global collaborative prediction model converges to obtain a final collaborative prediction model; S7: During the model deployment phase, the target vehicle obtains its real-time motion state through on-board sensors to form a trajectory sequence. The trajectory encoding module of the collaborative prediction model is used to extract semantic feature vectors. Subsequently, the semantic feature vectors of the neighboring vehicle trajectories are obtained, and the vehicle representation is obtained through the graph interaction fusion module. Finally, the trajectory decoding module outputs the trajectory prediction sequence of the target vehicle and its neighboring vehicles.

[0027] In this embodiment, first, the historical trajectory data of the target vehicle is collected and stored, including position, speed, acceleration, heading angle and timestamp information. The Z-score normalization method is used to normalize each dimension of the historical trajectory data. The normalized historical trajectory data is divided into fixed time windows. The trajectory sequence of each target vehicle is locally maintained in a collaborative prediction model including a trajectory encoding module, a graph interaction fusion module and a trajectory decoding module. A contrastive learning task is designed, and the trajectory sequence is input into the trajectory encoding module to obtain the semantic feature vector of the trajectory. The trajectory encoding module is trained by minimizing the contrastive learning loss function. Subsequently, the target vehicle communicates with the neighboring vehicles to obtain the semantic feature vector of the neighboring vehicle trajectory, calculates the distance between each vehicle, and defines three graphs based on the similarity relationship between the distance and the semantic feature vector. Combined with the multi-relation graph attention mechanism in the graph interaction fusion module, message passing and feature aggregation are performed on the three graphs to generate a vehicle representation. Then, the vehicle representation is input into the trajectory decoding module to obtain the future distance between the target vehicle and the neighboring vehicles. The trajectory prediction sequence of one time step is combined with a prediction loss function and a contrastive learning loss function, and the cooperative prediction model parameters of the target vehicle are updated by a back propagation algorithm; then, the participating vehicles upload the locally trained cooperative prediction model parameters to the central server, the central server aggregates the global cooperative prediction model according to the contribution of each participating vehicle, and the global cooperative prediction model parameters are distributed to each participating vehicle to replace the local cooperative prediction model parameters; steps S1-S5 are repeatedly executed until the global cooperative prediction model converges, and the final cooperative prediction model is obtained; finally, in the model deployment stage, the target vehicle obtains its real-time motion state through the vehicle-mounted sensor to form a trajectory sequence, extracts the semantic feature vector by using the trajectory encoding module of the cooperative prediction model, then obtains the semantic feature vector of the neighbor vehicle trajectory, obtains the vehicle representation through the graph interaction fusion module, and finally outputs the trajectory prediction sequence of the target vehicle and the neighbor vehicle through the trajectory decoding module.

[0028] Further, the Z-score standardization method is used to normalize each dimension of the historical trajectory data, including: The position is divided into two dimensions of longitude and latitude, and the speed, acceleration and heading angle are one dimension each, and the five dimensions are processed and represented as The normalization process is: , wherein, is the standardized feature value of the historical trajectory data feature dimension , is the original feature value of the historical trajectory data feature dimension , is the mean value of the historical trajectory data feature dimension , is the standard deviation of the historical trajectory data feature dimension , and is the historical trajectory data feature dimension . .

[0029] Further, the contrastive learning task includes: A segment of the trajectory sequence of the target vehicle is taken as an anchor sample, a positive sample of the anchor sample is generated by time clipping, a plurality of target vehicle trajectory sequences are constructed into a training batch, and the trajectory sequences other than the anchor sample in the batch are taken as negative samples of the anchor sample. By minimizing the contrastive learning loss function, the trajectory encoding module learns to obtain a semantic feature vector with driving intention discriminability.

[0030] Further, the time clipping includes: The anchor sample trajectory sequence ​​In the middle, randomly select the start time step and the end time step , satisfy and , intercept the subsequence as the cropped trajectory segment, the length of the cropped trajectory segment; For each time step , , the length of the cropped trajectory segment is restored to a fixed time window by a padding mechanism: For , the original data is retained, For the missing frames in the front part of , a forward extension strategy is adopted, and the start frame is copied to fill in, For the missing frames in the rear part of , the end frame is copied to fill in.

[0031] Further, the contrastive learning loss function comprises: The cosine similarity between the anchor sample and its positive sample is taken as the positive term, and the cosine similarity between the anchor sample and multiple negative samples is taken as the negative term; The positive term and the negative term are respectively exponentially weighted with a natural constant as the base and a temperature parameter as the denominator; All weighted terms are normalized by a normalized exponential function to obtain the relative probability of the anchor sample and the positive sample; Minimizing the negative logarithm of the relative probability is taken as the contrastive learning loss function to train the trajectory encoding module; Wherein, the temperature parameter satisfies ; The specific contrastive learning loss function is calculated as: , Wherein, is the loss function of contrastive learning, is the semantic feature vector of the anchor sample, is the semantic feature vector of the positive sample, is the semantic feature vector of the negative sample, is the cosine similarity function, which is defined as , is the temperature parameter, .

[0032] Further, the trajectory encoding module comprises: The trajectory encoding module includes an encoder composed of two layers of stacked long short-term memory networks (LSTM), which respectively process anchor sample trajectory sequences and positive sample trajectory sequences . is a fixed time window, is the number of feature dimensions of the trajectory sequence, . For each input sequence, the hidden state of the long short-term memory network at each time step is obtained to form a sequence of hidden vectors with a length of a fixed time window . The hidden state sequence is flattened into a one-dimensional vector in the time dimension, and is mapped into a semantic feature vector with a fixed dimension by a fully connected layer, where is a preset semantic feature vector length.

[0033] Further, the target vehicle communicates with neighbor vehicles, including: The target vehicle takes the position at the end time of its anchor sample trajectory sequence as a reference point to calculate the Euclidean distance with surrounding vehicles at the same time, and based on the Euclidean distance, candidate neighboring vehicles with a distance less than are selected within the set maximum perception and communication range , and the K closest vehicles are selected as neighbor vehicles, and if the number of candidate neighboring vehicles is less than K, a zero vector with the same dimension as the semantic feature vector is used to replace the missing vehicle feature input.

[0034] Further, the three graphs are defined by distance and semantic feature vector similarity, including: The first graph is a connection graph based on spatial proximity : the reciprocal of the Euclidean distance between vehicles is calculated as the initial weight, and the initial weight of each vehicle is normalized, and if the normalized weight exceeds a preset threshold , a connection is established between the two vehicles, indicating a spatial connectivity relationship. The second graph is a connection graph based on semantic behavior similarity : the cosine similarity of the semantic feature vectors between vehicles is calculated as the initial weight, and the normalized weight is calculated, and if the normalized weight exceeds a preset threshold , a connection is established, indicating a semantic connectivity relationship. The third graph is a comprehensive connection graph based on spatial-semantic fusion : the corresponding normalized weights in the first two graphs are weighted and summed according to a preset proportion to obtain a fusion weight, and the fusion weight is normalized again; if the fusion weight exceeds a preset threshold , a connection is established, indicating a comprehensive interactive connectivity relationship.

[0035] Further, the multi-relation graph attention mechanism in the graph interaction fusion module comprises: a connection graph based on spatial proximity , a connection graph based on semantic behavior similarity , and a comprehensive connection graph based on spatial-semantic fusion respectively construct independent graph attention layers; In each graph attention layer, for any vehicle node in the graph , its semantic feature vector is subjected to a learnable linear transformation to serve as a query vector, and the semantic feature vectors of all neighbor nodes connected to it under the relation graph are respectively subjected to learnable linear transformations to serve as key vectors and value vectors, and the attention weights are calculated in the following manner: , wherein, denotes the relation graph type, is the attention weight of the vehicle node and its neighbor node in the relation graph, are the semantic feature vectors of the vehicle node and the neighbor node respectively, is the corresponding learnable projection matrix in the relation graph, is the vector dimension of , is normalized along the neighbor node dimension , is the set of neighbor nodes connected to the vehicle node in the relation graph; Based on the calculated attention weights, the semantic feature vectors of the neighbor nodes are weighted and aggregated to obtain the context representation of the vehicle node under the th relation: , wherein, is the context representation of the vehicle node in the relation graph, is the attention weight of the vehicle node and its neighbor node in the relation graph, is the set of neighbor nodes connected to the vehicle node a set of neighbor nodes connected with the vehicle node in the relation graph a set of neighbor nodes connected with the vehicle node in the relation graph a projection matrix for the value vector, a semantic feature vector of the neighbor node ; concatenate the context representations obtained from the three relation graphs , and generate the vehicle representation of the vehicle node through an output projection layer: , wherein, the vehicle representation of the vehicle node, denotes vector concatenation, is an output projection matrix, is the context representation obtained from the three relation graphs. Further, the trajectory decoding module comprises:

[0036] The trajectory decoding module comprises a feedforward neural network composed of multiple fully connected layers, taking the vehicle representation of each vehicle node output in step S3 as input, performing layer-by-layer processing through linear transformation and ReLU activation function, and finally outputting a prediction vector containing elements; In the prediction vector, every two consecutive elements respectively represent the predicted latitude and longitude coordinates of the future time step, arranged in time sequence to form a trajectory sequence of the target vehicle in the future time steps. Further, the combination of the prediction loss function and the contrastive learning loss function comprises:

[0037] The prediction loss function is defined to measure the difference between the predicted trajectory and the real future trajectory , and the average of the sum of squared Euclidean distances of each time step is used as the loss: , wherein, the prediction loss function, is the time step length of the predicted trajectory, is the position coordinate of the real trajectory at the time step, is the position coordinate of the predicted trajectory at the time step, is the squared Euclidean distance between the predicted point and the real point. ​​The prediction loss function and the contrastive learning loss function are weighted and summed to construct a comprehensive loss function: , wherein, is an adjustable hyperparameter for balancing trajectory prediction accuracy and feature representation quality, is the comprehensive loss function, is the prediction loss function, is the contrastive learning loss function; based on the comprehensive loss function , the gradient is calculated by the back propagation algorithm, and the Adam optimizer is used to jointly update all trainable parameters of the trajectory encoding module, the graph interaction fusion module and the trajectory decoding module.

[0038] Further, the weighted aggregation according to the contribution degree of each participating vehicle includes: The central server takes the number of local historical trajectory data used by each participating vehicle in the current round of training as the measure of its contribution degree to calculate the aggregation weight; For the vehicle set participating in the current round of training , the aggregation weight of the vehicle is determined by the ratio of the number of its local historical trajectory data to the sum of the total data of all participating vehicles: , wherein, represents the number of trajectory sequences of vehicle used for global collaborative prediction model update in the current round of training, represents the number of trajectory sequences of vehicle used for global collaborative prediction model update in the current round of training, is the aggregation weight of vehicle used for global collaborative prediction model update in the current round of training, is the vehicle set participating in the current round of training; The central server performs weighted averaging on the local collaborative prediction model parameters uploaded by all participating vehicles based on the calculated weight to generate new global collaborative prediction model parameters : , wherein, is the new global collaborative prediction model parameters, is the aggregation weight of vehicle used for global collaborative prediction model update in the current round of training, is the vehicle uploaded local collaborative prediction model parameters, for the vehicle set participating in this round of training.

[0039] Further, the model deployment stage includes the following steps: Each vehicle will continuously collect the motion data (including position, speed, acceleration, heading angle) of the vehicle sensor in real time The motion data of T time steps constitutes a short-time historical trajectory sequence, and the same Z-score normalization method as in step S1 is used for normalization processing; The normalized short-time historical trajectory sequence is input into the trajectory encoding module of the final collaborative prediction model deployed locally and trained by federated learning, to obtain the semantic feature vector of the vehicle ; The vehicle broadcasts its semantic feature vector through vehicle-to-vehicle (V2V) communication, and simultaneously receives the semantic feature vectors broadcast by surrounding vehicles ; Based on the semantic feature vectors of the vehicle and the neighbor vehicles, a multi-relation graph structure is constructed locally according to the method defined in S3, including constructing three relation graphs based on spatial distance, semantic similarity and fusion weight; The constructed multi-relation graph and the semantic feature vectors of all vehicles are input into the graph interaction fusion module of the local model, and message passing and feature aggregation are performed through the multi-relation graph attention mechanism to generate a vehicle representation containing surrounding traffic environment information; The vehicle representation is input into the trajectory decoding module of the local model, and the trajectory decoding module directly outputs the predicted trajectory sequence of the vehicle and the neighbor vehicles in the future T time steps.

[0040] Figure 2 is an architecture of a multi-vehicle collaborative trajectory prediction method based on federated learning provided by an embodiment of the present application, as shown in the figure, the system includes a plurality of participating vehicle clients and a central server, each participating vehicle locally deploys a collaborative prediction model, and the model is composed of three core modules: Trajectory encoding module: used for encoding the historical trajectory sequence of the vehicle itself into a semantic feature vector with driving intention discrimination; Graph interaction fusion module: obtain the semantic features of surrounding vehicles through vehicle-to-vehicle communication, construct a multi-relation graph based on spatial distance and semantic similarity relationship, and perform context information fusion using graph attention mechanism; Trajectory decoding module: according to the fused vehicle representation, decode to generate trajectory prediction results in the future T time steps.

[0041] In the federated learning training process, after each vehicle completes local model training, the complete parameters of the local model are uploaded to the central server; the server aggregates all uploaded model parameters by weighting, generates a global model, and distributes the updated global model to each participating vehicle for the next round of training; through multiple iterations, a converged collaborative prediction model is finally obtained, realizing privacy protection and knowledge sharing among multiple vehicles.

[0042] Figure 3 is a module diagram of a multi-vehicle collaborative trajectory prediction system based on federated learning provided by an embodiment of the application. The multi-vehicle collaborative trajectory prediction system described below can be mutually corresponding to the multi-vehicle collaborative trajectory prediction method described above.

[0043] Data acquisition and preprocessing module: used for collecting the historical trajectory data of the vehicle itself, and normalizing by using the Z-score standardization method, and dividing into trajectory sequences of fixed time windows ; Trajectory encoding calculation module: used for locally performing a contrast learning task to encode the input trajectory sequence into a semantic feature vector with driving intention discriminability; Graph interaction fusion calculation module: used for obtaining semantic feature vectors of surrounding vehicles through inter-vehicle communication, constructing a multi-relation graph based on spatial distance and semantic similarity, and performing message passing and feature aggregation using a multi-relation graph attention mechanism to generate vehicle representations; Trajectory decoding calculation module: used for decoding the fused vehicle representations into position sequences of the target vehicle and neighbor vehicles in the future time steps; Model aggregation and update module: used for uploading the collaborative prediction model parameters after local training to the central server, and receiving the global collaborative prediction model parameters distributed by the central server to replace the local model for the next round of training. The central server is independently configured to receive collaborative prediction model parameters uploaded by each participating vehicle, calculate aggregation weights according to the local data volume of each vehicle, and perform weighted averaging on the collaborative prediction model parameters to generate updated global collaborative prediction model parameters and distribute them to each participating vehicle.

[0044] Each embodiment in the specification is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts of each embodiment can be mutually referred to.

[0045] The foregoing description of the disclosed embodiments enables a person skilled in the art to make or use the application. Modifications of these embodiments will occur to persons of skill in the art, and that the appended claims are intended to cover all such modifications that do not depart from the true spirit and scope of the application. Therefore, the application is not limited to the embodiments shown but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A multi-vehicle collaborative trajectory prediction method based on federated learning, characterized in that: The steps are: S1: Collect and store the historical trajectory data of the target vehicle, including position, speed, acceleration, heading angle and timestamp information, and use the Z-score normalization method to normalize each dimension of the historical trajectory data. The normalized historical trajectory data is divided into fixed time windows. Each target vehicle maintains a collaborative prediction model locally, which includes a trajectory encoding module, a graph interaction fusion module, and a trajectory decoding module. S2: Design a contrastive learning task, input the trajectory sequence into a trajectory encoding module to obtain the semantic feature vector of the trajectory, and train the trajectory encoding module by minimizing the contrastive learning loss function; S3: The target vehicle communicates with neighboring vehicles to obtain the semantic feature vectors of their trajectories, calculate the distances between each vehicle, and define three graphs based on the similarity between distances and semantic feature vectors. Combined with the multi-relational graph attention mechanism in the graph interaction fusion module, message passing and feature aggregation are performed on the three graphs to generate a vehicle representation. S4: Input the vehicle representation into the trajectory decoding module to obtain the future trajectory of the target vehicle and neighboring vehicles. The trajectory prediction sequence of time steps is combined with the prediction loss function and the contrastive learning loss function to jointly update the target vehicle's collaborative prediction model parameters through the back propagation algorithm; S5: The participating vehicles upload the locally trained collaborative prediction model parameters to the central server. The central server performs weighted aggregation based on the contribution of each participating vehicle to generate a global collaborative prediction model. The global collaborative prediction model parameters are distributed to each participating vehicle, replacing their local collaborative prediction model parameters. S6: Repeat S1-S5 until the global collaborative prediction model converges to obtain a final collaborative prediction model; S7: During the model deployment phase, the target vehicle obtains its real-time motion state through on-board sensors to form a trajectory sequence. The trajectory encoding module of the collaborative prediction model is used to extract semantic feature vectors. Subsequently, the semantic feature vectors of the neighboring vehicle trajectories are obtained, and the vehicle representation is obtained through the graph interaction fusion module. Finally, the trajectory decoding module outputs the trajectory prediction sequence of the target vehicle and its neighboring vehicles.

2. The method for multi-vehicle collaborative trajectory prediction based on federated learning according to claim 1, characterized in that: The Z-score normalization method is used to normalize each dimension of the historical trajectory data, including: The position is divided into two dimensions: longitude and latitude, and speed, acceleration, and heading angle are each one dimension. The five dimensions are processed and expressed as , and its normalization process is: ， in, is the feature dimension of historical trajectory data The standardized eigenvalues ​​of is the feature dimension of historical trajectory data The original eigenvalues ​​of is the feature dimension of historical trajectory data The mean of is the feature dimension of historical trajectory data The standard deviation of Represents the characteristic dimension of historical trajectory data, .

3. The method for multi-vehicle collaborative trajectory prediction based on federated learning according to claim 1, characterized in that: The contrastive learning task includes: A trajectory sequence of the target vehicle is used as an anchor sample. A positive sample of the anchor sample is generated by time clipping. Multiple target vehicle trajectory sequences are combined into a training batch, and the other trajectory sequences in the batch except the anchor sample are used as negative samples of the anchor sample. By minimizing the contrastive learning loss function, the trajectory encoding module is trained to learn a semantic feature vector that is discriminative of driving intention.

4. The method for multi-vehicle collaborative trajectory prediction based on federated learning according to claim 3, characterized in that: The time clipping includes: From the anchor sample trajectory sequence In the example, randomly select the starting time step and end time step ,satisfy and , extract subsequence As the clipped trajectory segment, is the length of the cropped trajectory segment; For each time step , , the length of the cropped trajectory segment is restored to the fixed time window through the filling mechanism : for , keep the original data, for The front frame is missing, so the forward extension strategy is adopted to copy the starting frame Fill it up. for The last frame is missing, copy the termination frame Fill it.

5. The method for multi-vehicle collaborative trajectory prediction based on federated learning according to claim 1, characterized in that: The contrastive learning loss function includes: The cosine similarity between the anchor sample and its positive sample is taken as the positive term, and the cosine similarity between the anchor sample and multiple negative samples is taken as the negative term; The positive and negative terms are respectively calculated with the natural constant as the base and the temperature parameter as the The exponential weighting of the denominator; Normalize all weighted items through the normalized exponential function to obtain the relative probability of anchor samples and positive samples; The trajectory encoding module is trained by minimizing the negative logarithm of the relative probability as a contrastive learning loss function; Among them, the temperature parameter satisfy .

6. The method for multi-vehicle collaborative trajectory prediction based on federated learning according to claim 1, characterized in that: The trajectory encoding module includes: The trajectory encoding module includes an encoder consisting of two stacked long short-term memory networks, which process the anchor sample trajectory sequence and the positive sample trajectory sequence , is a fixed time window, is the number of feature dimensions of the trajectory sequence, ; For each input sequence, obtain the hidden state of the long short-term memory network at each time step to form a fixed time window of length The sequence of hidden vectors of The hidden state sequence is flattened into a one-dimensional vector along the time dimension and mapped to a fixed dimension through a fully connected layer The semantic feature vector of is the preset semantic feature vector length.

7. The method for multi-vehicle collaborative trajectory prediction based on federated learning according to claim 1, characterized in that: The target vehicle communicates with neighboring vehicles, including: The target vehicle takes its position at the end of the anchor sample trajectory sequence as a reference point, calculates the Euclidean distance with the surrounding vehicles at the same time, and calculates the maximum perception and communication range within the set range based on the Euclidean distance. Filter out the distance less than The candidate neighboring vehicles are selected from the K vehicles closest to them as neighboring vehicles. If the number of candidate neighboring vehicles is less than K, a zero vector with the same dimension as the semantic feature vector is used to replace the feature input of the missing vehicle.

8. The method for multi-vehicle collaborative trajectory prediction based on federated learning according to claim 1, characterized in that: The three graphs are defined by the distance and semantic feature vector similarity relationship, including: The first figure is a connection graph based on spatial proximity : Calculate the inverse of the Euclidean distance between vehicles as the initial weight, and normalize the initial weight of each vehicle. If the normalized weight exceeds the preset threshold , then a connection is established between the two vehicles, indicating a spatial connectivity relationship; The second figure is a connection diagram based on semantic behavior similarity :Calculate the cosine similarity of the semantic feature vectors between vehicles as the initial weight and perform normalization. If the normalized weight exceeds the preset threshold , then a connection is established, indicating a semantic connectivity relationship; The third figure is a comprehensive connection diagram based on spatial-semantic fusion :Sum the corresponding normalized weights in the first two figures according to the preset ratio to obtain the fusion weight, and normalize the fusion weight again; if the fusion weight exceeds the preset threshold , then a connection is established, indicating a comprehensive interactive connectivity relationship.

9. The method for multi-vehicle collaborative trajectory prediction based on federated learning according to claim 1, characterized in that: The multi-relational graph attention mechanism in the graph interaction fusion module includes: The spatial proximity-based connection graph , connection graph based on semantic behavior similarity and a comprehensive connectivity graph based on spatial-semantic fusion , construct independent graph attention layers respectively; In each graph attention layer, for any vehicle node in the graph , its semantic feature vector After the learnable linear transformation, it is used as the query vector, and all the neighbor nodes connected to it in the relationship graph are Semantic feature vector of After being transformed into key vectors and value vectors through learnable linear transformation, the attention weights are calculated as follows: ， in, Indicates the relationship diagram type, For Vehicle nodes in the relationship graph Its neighboring nodes The attention weight, Vehicle nodes and neighbor nodes The semantic feature vector of for The corresponding learnable projection matrix in the relationship diagram, for The vector dimension of Along the neighbor node dimension Normalize, For Relationship diagram with vehicle nodes The set of connected neighbor nodes; Based on the calculated attention weights, the semantic feature vectors of neighbor nodes are weighted and aggregated to obtain the first Vehicle nodes under the relationship Contextual representation of: ， in, For Vehicle nodes in the relationship graph Context representation, For the relationship Vehicle nodes in the figure Its neighboring nodes The attention weight, For Relationship diagram with vehicle nodes The set of connected neighbor nodes, is the projection matrix of the value vector, Neighbor nodes Semantic feature vector of The context representation obtained under the three relationship diagrams Splice and generate vehicle nodes through an output projection layer Vehicle characterization : ， in, Vehicle node Vehicle characterization, represents vector concatenation, is the output projection matrix, It is the context representation obtained under the three relationship diagrams.

10. The method for multi-vehicle collaborative trajectory prediction based on federated learning according to claim 1, characterized in that: The trajectory decoding module includes: The trajectory decoding module includes a feedforward neural network composed of multiple fully connected layers, which takes the vehicle representation of each vehicle node output in step S3 as input, processes it layer by layer through linear transformation and ReLU activation function, and finally outputs a elements of the prediction vector; In the prediction vector, each two consecutive elements represent the future The predicted longitude and latitude coordinates of the time steps are arranged in chronological order to form the future coordinates of the target vehicle. A trajectory sequence of time steps.

11. The method for multi-vehicle collaborative trajectory prediction based on federated learning according to claim 1, characterized in that: The combined prediction loss function and contrastive learning loss function include: Define the prediction loss function Used to measure the model prediction trajectory and the real future trajectory The difference between them is calculated by taking the average of the sum of squared Euclidean distances at each time step as the loss: ， in, is the prediction loss function, is the time step length of the predicted trajectory, The true trajectory is The position coordinates of the time step, To predict the trajectory The position coordinates of the time step, is the square of the Euclidean distance between the predicted point and the true point; The prediction loss function and the contrastive learning loss function are weighted summed to construct a comprehensive loss function: ， in, is an adjustable hyperparameter used to balance trajectory prediction accuracy and feature representation quality. is the comprehensive loss function, is the prediction loss function, is the contrastive learning loss function; Based on the comprehensive loss function , the gradient is calculated by the back-propagation algorithm, and the Adam optimizer is used to jointly update all the trainable parameters of the trajectory encoding module, graph interaction fusion module and trajectory decoding module.

12. The method for multi-vehicle collaborative trajectory prediction based on federated learning according to claim 1, characterized in that: The weighted aggregation according to the contribution of each participating vehicle includes: The central server uses the amount of local historical trajectory data used by each participating vehicle in this round of training as a measure of its contribution and calculates the aggregation weight; For the vehicle set participating in this round of training ,vehicle Aggregation weight The number of local historical trajectory data The ratio to the total data volume of all participating vehicles is determined as follows: ， in, Indicates vehicle The number of trajectory sequences used to update the global collaborative prediction model in this round of training, Indicates vehicle The number of trajectory sequences used to update the global collaborative prediction model in this round of training, For vehicles The aggregation weight used to update the global collaborative prediction model in this round of training, Gather the vehicles participating in this round of training; The central server calculates the weight based on , for all participating vehicles to upload the local collaborative prediction model parameters Perform weighted averaging to generate new global collaborative prediction model parameters : ， in, To generate new global collaborative prediction model parameters, For vehicles The aggregation weight used to update the global collaborative prediction model in this round of training, For vehicles Uploaded local collaborative prediction model parameters, Gather the vehicles participating in this round of training.

13. A multi-vehicle collaborative trajectory prediction system based on federated learning, characterized in that: Implementing a multi-vehicle collaborative trajectory prediction method based on federated learning as described in any one of claims 1 to 12, comprising: Data collection and preprocessing module: used to collect the vehicle's own historical trajectory data, and normalize it using the Z-score standardization method, dividing it into fixed time windows The trajectory sequence of Trajectory encoding calculation module: used to perform contrastive learning tasks locally and encode the input trajectory sequence into a semantic feature vector with driving intention discrimination; Graph interaction fusion computing module: This module is used to obtain semantic feature vectors of surrounding vehicles through inter-vehicle communication, construct a multi-relationship graph based on spatial distance and semantic similarity, and use the multi-relationship graph attention mechanism to perform message transmission and feature aggregation to generate vehicle representations. Trajectory decoding calculation module: used to decode the fused vehicle representation into the future trajectory of the target vehicle and neighboring vehicles. Position sequence of time steps; Model aggregation and update module: used to upload the collaborative prediction model parameters after local training to the central server, and receive the global collaborative prediction model parameters issued by the central server to replace the local model for the next round of training; Among them, the central server is independently configured to receive the collaborative prediction model parameters uploaded by each participating vehicle, calculate the aggregation weight according to the local data volume of each vehicle, and perform weighted averaging on the collaborative prediction model parameters to generate updated global collaborative prediction model parameters and send them to each participating vehicle.

Images