A personalized semi-supervised traffic pattern recognition method based on federated learning

Abstract

The present invention discloses a personalized semi-supervised traffic mode recognition method based on federated learning. The method firstly generates an initialized basic model by a central server, and selects participating users to send an autoencoder generated by a hypernetwork. Secondly, the participating users use local GPS trajectory data to train the autoencoder, return it to the central server, update the user embedding and hypernetwork parameters, and obtain potential features. Then the central server uses the potential features to train the basic model, and the participating users use the labeled GPS trajectory data set and the GPS trajectory label-free data set to train the supervised MLP model and the semi-supervised convolutional autoencoder model locally. Finally, the basic model, the semi-supervised convolutional autoencoder model and the mixed output of the two are sent to a soft voter to obtain the traffic mode. The present invention solves the problem of data leakage that may occur when uploading data and the problem of the central server obtaining user privacy.

Description

Technical Field

[0001] The present invention belongs to the field of pattern recognition, and in particular relates to a personalized semi-supervised traffic pattern recognition method based on federated learning. Background Art

[0002] Traffic pattern identification (TMI) aims to infer traffic patterns from users' mobility data. As a core application of intelligent transportation systems (ITS), accurate TMI can help solve various traffic-related problems, such as public transportation planning, route recommendation, traffic signal optimization, etc. With the recent development of data-driven analysis (especially machine and deep learning algorithms), various IoT devices have attracted extensive research due to their ability to capture large amounts of data. Currently, sensors such as smartphones and car-mounted GPS locators have simplified the acquisition of different user trajectories, paving the way for IoT-enabled TMI.

[0003] The current technical solutions for traffic pattern recognition have problems such as privacy leakage and insufficient labeled data. Although traditional federated learning can solve these problems, it still cannot solve the heterogeneity of participating client data and cannot generate personalized models. Therefore, a feasible technical solution is urgently needed for TMI tasks and the above problems. Summary of the invention

[0004] In view of the shortcomings of the prior art, the present invention proposes a personalized semi-supervised traffic pattern recognition method based on federated learning to solve the problem of insufficient data in the process of traffic pattern recognition. Under the premise of fully protecting user privacy, the task of traffic pattern classification is excellently completed, and a trade-off between model performance and privacy protection is achieved. A personalized local model is generated for each participating user, which solves the problem that the data of each participant may have different distributions, characteristics and quantities, resulting in a decrease in model performance in certain areas.

[0005] The technical solution of the present invention supports clients with no labels or limited labeled data sets and a large amount of unlabeled data that is not enough to train a model with good performance. The solution combines a personalized federated learning framework with a local semi-supervised deep learning model. Among them, the personalized federated learning framework uses a hypernetwork to embed the user as input and generate a personalized autoencoder for the user. At the same time, the server generates a basic model for the user to perform traffic mode recognition and classification. The goal of the personalized federated learning framework is to generate a set of personalized models for each user to solve the performance problems of traditional federated learning methods in terms of user domain differences. The local semi-supervised deep learning model includes Conv-AE and CNN classifiers for unsupervised learning and supervised learning, respectively, using unlabeled and labeled GPS trajectory data as input, and deriving the final traffic mode category through the model. The technical solution of the present invention performs well for users without labels or with limited labeled data sets. The purpose of traffic mode recognition in the embodiment of the present invention is to identify specific traffic modes, where the traffic mode categories are walking, car, bicycle, train, and bus.

[0006] The present invention adopts the following technical solution: a personalized semi-supervised traffic mode recognition method based on federated learning, comprising the following steps:

[0007] (1) Using a central server capable of performing federated learning training, the federated learning training environment is initialized to generate an initialized basic model for participating users; in order to improve the classification accuracy of traffic mode recognition, the participating users also include a semi-supervised convolutional autoencoder model locally. Multiple participating users with GPS trajectory data are connected through the central server. The central server and participating users pre-process the local GPS data.

[0008] (2) The central server randomly selects a participating user, uses the hypernetwork to generate an autoencoder based on the user embedding, and sends it to the corresponding participating user.

[0009] (3) The participating users use their local GPS trajectory data to train the received autoencoder and return it to the central server after training. The central server updates the user embedding and hypernetwork parameters based on the encoder and selects data with a distribution similar to that of the participating users’ local GPS trajectory data on the central server side. The data is encoded using the autoencoder to obtain the encoded latent features.

[0010] (4) The server uses the latent features generated in step (3) to train the basic model generated by the central server in step (1) for the participating user to obtain a supervised MLP model, and sends the model to the participating user as one of the two models used by the participating user locally for traffic mode recognition and classification.

[0011] (5) Participating users locally use the labeled GPS trajectory dataset and the unlabeled GPS trajectory dataset to train the supervised MLP model and the semi-supervised convolutional autoencoder model.

[0012] (6) The outputs of the supervised MLP model and the semi-supervised convolutional autoencoder model are concatenated and passed through a fully connected network to generate a mixed output of the two models. Finally, the basic model output, the semi-supervised convolutional autoencoder model output, and the mixed output are sent to the soft voter to obtain the final transportation mode.

[0013] (7) The participating users use local data for training, update the model, share the model parameters with the central server, and then return to step (2) until the participating user model converges or reaches the specified number of training times.

[0014] Furthermore, in step (1), the participating users pre-process the local GPS data by segmenting the GPS trajectory data and extracting features, and using the data including four motion features of relative distance, speed, acceleration and jerk as the input data of the model.

[0015] Furthermore, in step (1), the central server and the participating users divide the GPS data into training data sets and test data sets in a ratio of 5:1. Among the participating user training data, the labeled data accounts for 45% of the total data; the central server training data only contains the labeled data.

[0016] Furthermore, in step (2), the central server generates a personalized autoencoder from the user embedding list of each participating user, the hypernetwork parameter θ, and the hypernetwork model, and sends it to the selected participating user.

[0017] Furthermore, in step (3), the central server determines whether its data is similar to the local data distribution of the participating user by calculating the autoencoder loss function, using the autoencoder returned to the central server after the participating user's training to check whether the distance between its data value and the reconstructed value is less than a specified threshold, and selects the data less than the threshold to train the basic model generated by the central server for the participating user in step (1).

[0018] Furthermore, the basic model generated by the central server is an MLP model including two fully connected layers, and a ReLu layer is added as an activation function after each linear layer.

[0019] Furthermore, the semi-supervised convolutional autoencoder model (SECA) combines two main components: (1) a CNN classifier that only receives labeled trajectories; (2) a convolutional-deconvolutional autoencoder (Conv-AE), which receives labeled and unlabeled trajectories; for the CNN classifier, the cross entropy is used as the loss function. For the convolutional-deconvolutional autoencoder, the mean squared error (MSE) is used as the loss function. Finally, the total loss function is calculated by the cross entropy loss function and the mean squared error loss function.

[0020] Furthermore, in step (6), the output of the local basic model of the participating user and the output of the semi-supervised convolutional autoencoder model (SECA) are fed into a meta-model comprising three fully connected layers, and a new output is generated through the meta-model.

[0021] Furthermore, in step (6), the basic model output, the semi-supervised convolutional autoencoder model output and the mixed output are converted into probabilities for each category using a softmax function, which are then sent to a soft voter to add up the probabilities for different categories, and the category with the highest probability is the voting result.

[0022] Compared with the prior art, the beneficial technical effects of the present invention are:

[0023] 1. The present invention uses a personalized federated learning method to perform traffic pattern recognition. Compared with the traditional traffic pattern recognition method, this method does not require the client to upload local GPS data to the central server, which solves the problem of data leakage that may occur during data upload and the problem of the central server obtaining user privacy through GPS data.

[0024] 2. Compared with the traditional federated learning method that aggregates models on the central server side, the present invention aims to generate a personalized local model for each participating user. The present invention uses a hypernetwork on the central server to generate a personalized autoencoder for the user. After the central server receives the user's update, the central server generates a basic model for each user, and the user trains them locally using his own labeled dataset.

[0025] 3. In order to improve the classification accuracy of local traffic pattern recognition, ensemble learning is introduced. In addition to the basic model generated by the central server, the user also sets up a convolution-deconvolution autoencoder model. Finally, the outputs of multiple models are integrated through a soft voter to generate the final classification.

[0026] 4. In view of the lack of labeled data in real life, the present invention adopts a semi-supervised model to support users without labels or limited labeled data sets and a large amount of unlabeled data that is insufficient to train a model with good performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art are briefly introduced below.

[0028] Figure 1 A schematic diagram of a personalized semi-supervised traffic pattern recognition technology framework based on federated learning in the present invention;

[0029] Figure 2 A schematic diagram of a personalized semi-supervised traffic pattern recognition technology process based on federated learning in the present invention;

[0030] Figure 3 The user-personalized automatic encoder and basic model structure in the present invention;

[0031] Figure 4 This is the user product-deconvolution autoencoder model structure in the present invention. DETAILED DESCRIPTION

[0032] The specific implementation modes of the present invention are described below in conjunction with the accompanying drawings and examples. However, the following examples are only used to illustrate the present invention in detail and do not limit the scope of the present invention in any way.

[0033] The present invention discloses a personalized semi-supervised traffic pattern recognition method based on federated learning, such as Figure 1 , Figure 2 The system architecture and method flow of the present invention are respectively given. The present invention is described in detail below in conjunction with Example 1, which specifically includes the following steps:

[0034] Step 1: Use a central server for federated learning training to initialize the federated learning training environment and generate an initialized basic model for participating users; the user's local also includes a semi-supervised convolutional autoencoder model; connect multiple participating users with GPS trajectory data through the central server.

[0035] Step 11: The central server stores a list of users connected to the central server. The central server needs to initialize the federated learning training environment and generate an initial basic model for each participating user. The specific calculation formula is as follows:

[0036]

[0037] in represents the autoencoder generated by the central server for the selected user j, h represents the hypernetwork, and its input parameter is the user embedding α j and the hypernetwork parameters θ.

[0038] Step 12, the central server and the participating users perform data preprocessing on the GPS trajectory data. In Example 1, the GPS data is preprocessed by segmenting the GPS trajectory data and extracting features, and finally selecting the data of four motion features of relative distance, speed, acceleration, and jerk from the seven motion features of relative distance, interval time, relative speed, speed, acceleration, jerk, and azimuth as the input features of the model.

[0039] The central server and the user divide the GPS data into training data sets and test data sets, with a ratio of 5:1. Among them, the labeled data accounts for 45% of the total data in the user training data, and the central server training data only contains labeled data.

[0040] Step 2: The central server randomly selects a participating user, uses the hypernetwork to generate an autoencoder based on the user embedding, and sends it to the corresponding participating user. The hypernetwork will pass the user's user embedding vector to the multi-layer perceptron part of the hypernetwork to obtain the feature vector features, and calculate the weights and biases of the encoder and decoder by using the feature vector features, and store them in an ordered dictionary weights. The weights and biases of each layer of the user's personalized autoencoder can be obtained by accessing the ordered dictionary weights.

[0041] The hypernetwork is located on the server side and acts on the client descriptor α j The client descriptor can be a trainable embedding vector or a fixed parameter, but it is necessary to ensure that the descriptor is a good client representation that has been trained. j Embedding vector for the user, which will be trained and updated in subsequent steps.

[0042] In the present invention, the hypernetwork is not used to learn the entire network, but only the feature extraction part of the target network is generated using the hypernetwork, which contains most of the trainable parameters, and the local output layer is learned for each client. In the present invention, the hypernetwork is used to generate an autoencoder as the feature extraction part of the traffic pattern recognition network.

[0043] The personalized automatic encoder generated for the user by the central server used in this embodiment includes three fully connected modules, the first two fully connected modules include a fully connected linear layer and a ReLu activation function layer, and the last fully connected module includes a fully connected linear layer and a Tanh activation function layer.

[0044] Step 3: The user trains the autoencoder using local data, such as Figure 3 As shown, after fine-tuning, the autoencoder is returned to the central server:

[0045] The user updates the received autoencoder model using the entire unlabeled dataset and labeled dataset in round T, using mean square error MSE as the loss function during the update process, and sends the updated autoencoder back to the central server. The update equation is:

[0046]

[0047] in, represents the autoencoder model of user j in round r, η represents the learning rate, and represents the gradient of the autoencoder, It represents the loss function of the automatic encoder. In the present invention, mean square error MSE is used as the loss function.

[0048] Step 4: The central server updates the corresponding user embedding variables and hypernetwork parameters based on the fine-tuned encoder, selects data with a distribution close to the user's local data, and uses an autoencoder to encode the data with a similar distribution to obtain the encoded latent features.

[0049] Before the central server uses the autoencoder to encode the central server-side data, it needs to filter the data first to filter out the data that is close to the user's local data distribution. The screening method is to calculate the autoencoder loss function and use the autoencoder model fine-tuned by user j to check whether the distance between its data value and the reconstructed value is less than the specified threshold. The data less than the threshold means that the distribution is close to the user's local data. The autoencoder loss function is calculated as the mean square error MSE. The specific calculation formula is as follows:

[0050]

[0051] Where l refers to the loss function of the autoencoder, X is the data value, represents the reconstructed value generated after the autoencoder, τ refers to the specified threshold, and D S refers to the central server dataset.

[0052] Step 5: The central server uses the latent features generated in step 4 to train the basic model generated by the central server for the user, and sends the trained basic model to the user as one of the two models used by the user locally for traffic pattern recognition and classification.

[0053] The basic model generated by the central server for users includes two linear fully connected modules. The first fully connected module consists of a fully connected layer and a ReLu activation function, and the second fully connected layer consists of a fully connected layer and a log_softmax function. A normalization operation is performed before each fully connected module.

[0054] Step 6: In each round, the user locally trains the base model and the semi-supervised convolutional autoencoder model (SECA) using the labeled dataset and the unlabeled dataset.

[0055] The semi-supervised convolutional autoencoder model used in this framework receives labeled and unlabeled trajectory data represented as:

[0056] X comb =(X l +X u )∈R (1*M*4)

[0057] Among them, X l and X u 4-channel tensors corresponding to marked and unmarked GPS segments respectively. The GPS segment data is generated by the data preprocessing in step 1. In the present invention, the value of M is set to 248.

[0058] like Figure 4 As shown in Figure 2, the semi-supervised convolutional autoencoder model consists of two main components: (1) a CNN-based classifier that only receives labeled trajectories; (2) a convolutional-deconvolutional autoencoder (Conv-AE), which receives both labeled and unlabeled trajectories.

[0059] The encoder in the convolution-deconvolution autoencoder consists of three groups of layers, each with two convolutional layers followed by a max pooling layer. The stride of the max pooling layer is 2. According to the above settings, the spatial size is reduced to the general size of the previous layer only after the max pooling layer, and remains unchanged after the convolutional layer.

[0060] The rectified linear unit (ReLu) function is used as the activation function in the encoder part of the convolution-deconvolution autoencoder.

[0061] The decoder in a convolution-deconvolution autoencoder has the same number of layers as the decoder, and the decoder performs various inverse operations including unpooling and deconvolution to generate an output of the same size as the input in the corresponding layer of the encoder part.

[0062] In the decoder, except for the last layer, the ReLu function is used as the activation function, and the sigmoid function is used as the activation function in the last layer.

[0063] The CNN-based classifier consists of a convolutional part and a fully connected part. The convolutional part is the same as the encoder part of Conv-AE, but accepts labeled data X l As input data, after the convolution part, the generated latent features will be flattened and sent to the fully connected layer, and the softmax function will be called to generate the GPS segment X lThe probability distribution of traffic labels. The classifier uses classification cross entropy as the loss function of the CNN classifier.

[0064] The training strategy of the semi-supervised convolutional autoencoder model is to train Conv-AE and CNN classifiers simultaneously. The basic principle is to extract useful information from the underlying distribution of the input data through ConvAE, while using the classifier to enhance the discriminative ability of the architecture.

[0065] The semi-supervised convolutional autoencoder model is jointly learned by minimizing the following total loss function, which is calculated as follows:

[0066] l semi-ae+cls =αl Conv-AE +βl labeled-classifier

[0067] Where α and β are model hyperparameters whose sizes are defined by the user, who balances the relative importance of the losses in the two parts of the model. Conv-AE and labeled-classifier are the loss functions of Conv-AE and CNN classifiers, respectively. semi-ae+cls It is the loss function of the whole model.

[0068] Step 7: Concatenate the outputs of the basic model and the semi-supervised convolutional autoencoder model, and send them into the meta-model to generate new outputs. Send the outputs of the basic model, the semi-supervised convolutional autoencoder model, and the meta-model into the soft voter, add up the probabilities of different categories, and the category with the highest probability is the voting result.

[0069] As a 5-class model, the final output of the basic model and the semi-supervised convolutional autoencoder model is a 1*5 vector, which represents the probability of each category. Use torch.cat to concatenate the output vectors in the first dimension to obtain a 1*10 vector result.

[0070] The meta-model consists of three fully connected modules. The first two fully connected modules include a fully connected layer and a ReLu activation function layer. The last fully connected module executes the softmax function after the fully connected layer to generate the probability distribution of traffic labels.

[0071] Step 8: The user uses local data for training, updates the model, shares the model parameters with the central server, and then returns to step 2 until the user model converges or reaches the specified number of training times.

[0072] In some embodiments, in order to better verify and understand the effects of the embodiments of the present invention, verification is performed through experiments in this embodiment.

[0073] Specifically, in this embodiment, the personalized federated learning technology based on federated learning is evaluated on the Geolife dataset of Microsoft, which contains 17,621 GPS trajectory data, of which only a small part has traffic mode labels. The traffic modes include "walking", "bicycle", "bus", "driving" and "train".

[0074] Specifically, after preprocessing the above 17,621 GPS trajectory data, 24,156 labeled trajectory data feature segments and 72,287 unlabeled trajectory data feature segments were obtained.

[0075] In the overall experiment, the model divides the overall labeled data into 5 folds for training and testing, 2 folds of the labeled data are used as central server-side data, and the remaining 3 folds of labeled data are equally divided among participating users. The number of participating users in this experiment is set to 5. Using stratified sampling, 10% of the labeled training data in each fold is selected as the validation set used in the deep learning-based model. All unlabeled data during training will be used for training the semi-supervised model.

[0076] During training, the hyperparameters α and β of the semi-supervised convolutional autoencoder model were set to 1.0 and 1.0 respectively. In the context of machine learning, hyperparameters are parameters whose values ​​are set before starting the learning process, rather than parameter data obtained through training. These parameters appear in the best form on a separate validation set.

[0077] Specifically, in this embodiment, the Adam optimizer is also used to minimize the total loss of the traffic mode recognition model. The total number of training rounds is set to 50 epochs, and the number of rounds for each user to fine-tune the autoencoder and the local model is set to 5; epoch represents one round of training, and the model is fully trained once using all the data in the training set.

[0078] Specifically, the hardware configuration of the experiment in this embodiment is as shown in Table 1 below:

[0079] Table 1 Hardware Configuration Table

[0080] CPU Intel(R)Xeon(R)Gold 5120 CPU GPU NVIDIA RTX A6000 Simulation Tools Python Modeling Platform Pytorch Platform

[0081] In Table 1, the CPU, central server GPU, simulation tool and modeling platform used in this embodiment are defined, which provides a hardware foundation for experimentally verifying the accuracy of the traffic pattern recognition model.

[0082] In this embodiment, eight methods are introduced to verify the reliability of traffic mode recognition in the embodiment of the present invention. The eight methods are:

[0083] Multilayer Perceptron (MLP), K Nearest Neighbor (KNN), Support Vector Machine (SVM), CNN Classifier (CNN) with the same settings as the proposed model, Recurrent Neural Network (RNN) with Long Short-Term Memory (LSTM) module, Decision Tree (DT), Semi-Two-Steps, Semi-Supervised Convolutional Autoencoder (SECA).

[0084] Table 2 Comparison of recognition accuracy of selected algorithms

[0085]

[0086] Specifically, the recognition accuracy comparison between the present invention and other algorithms is shown in Table 2. As shown in Table 2, when using any percentage of labeled data, the accuracy of the present invention is higher than that of the above 8 methods.

[0087] The embodiments of the present invention solve the problem of classifying user travel modes with less labeled data. At the same time, in the classification process, the privacy of users is fully protected through federated learning, a trade-off between model performance and privacy protection is achieved, and a personalized local model is generated for each participating user, solving the problem that the data of each participant may have different distributions, characteristics and quantities, resulting in reduced performance of the model in certain areas.

[0088] The embodiments of the present invention are described in detail above with reference to the accompanying drawings, but the present invention is not limited to the described embodiments. For those skilled in the art, various changes, modifications, substitutions and variations of these embodiments including components are made without departing from the principles and spirit of the present invention, and still fall within the scope of protection of the present invention.

Claims

1. A personalized semi-supervised traffic pattern recognition method based on federated learning, characterized in that: The steps include: S1. Using the central server for federated learning training, initialize the federated learning training environment and generate an initialized basic model for the participating users; the participating users also have a semi-supervised convolutional autoencoder model locally; the central server connects multiple participating users with GPS trajectory data; the central server and participating users pre-process the local GPS data; The preprocessing method is: segmenting the GPS trajectory data and extracting features, using data of four motion features, namely, relative distance, speed, acceleration, and jerk, as input data; S2, the central server randomly selects a participating user, uses the hypernetwork to generate an autoencoder based on the user embedding, and sends it to the corresponding participating user; S3, the participating user uses the local GPS trajectory data to train the received autoencoder, and returns it to the central server after training. The central server updates the user embedding and hypernetwork parameters according to the encoder, and selects data with a distribution similar to the local GPS trajectory data of the participating user on the central server side, and encodes it using the autoencoder to obtain the encoded latent features; S4, the central server uses the latent features to train the basic model generated by the central server in S1 for the participating users to obtain a supervised MLP model, and sends the model to the participating users; S5, the user locally uses the labeled GPS trajectory dataset to train the supervised MLP model, and the user locally uses the labeled GPS trajectory dataset and the unlabeled GPS trajectory dataset to train the semi-supervised convolutional autoencoder model; S6, the output of the spliced ​​supervised MLP model and the semi-supervised convolutional autoencoder model, which is passed through a fully connected network to generate a mixed output of the two models; The base model output, the semi-supervised convolutional autoencoder model output, and the mixed output are fed into a soft voter to derive the final transportation mode. S7: The participating user uses local data for training, updates the model, shares the model parameters with the central server, and then returns to S2 until the participating user model converges or reaches the specified number of training times.

2. The personalized semi-supervised traffic pattern recognition method based on federated learning according to claim 1 is characterized in that: The step S1 also includes that the central server and the participating users divide the GPS data into a training data set and a test data set, wherein in the participating user training data, the marked data accounts for 45% of the total data; and the server training data only includes the marked data.

3. The personalized semi-supervised traffic pattern recognition method based on federated learning according to claim 2 is characterized in that: In step S2, using the hypernetwork to generate an autoencoder according to user embeddings is specifically as follows: the server generates a personalized autoencoder based on the user embedding list of each participating user, the hypernetwork parameter θ, and the hypernetwork model.

4. The personalized semi-supervised traffic pattern recognition method based on federated learning according to claim 3 is characterized in that: In step S3, the server determines whether its data is similar to the user's local data distribution by calculating the autoencoder loss function, using the autoencoder returned to the central server after the participating user's training to check whether the distance between its data value and the reconstructed value is less than a specified threshold, and selects data with a value less than the threshold to train the basic model generated by the central server for the participating user in step S1.

5. The personalized semi-supervised traffic pattern recognition method based on federated learning according to claim 4 is characterized in that: The semi-supervised convolutional autoencoder model combines two components: a CNN classifier that only receives labeled trajectories and a convolutional-deconvolutional autoencoder that accepts both labeled and unlabeled trajectories. For the CNN classifier, cross entropy is used as the loss function; for the convolution-deconvolution autoencoder, mean square error MSE is used as the loss function; finally, the total loss function is calculated by the cross entropy loss function and the mean square error loss function.

Images