A high-quality trajectory depth generation method satisfying differential privacy

Abstract

This invention discloses a high-quality trajectory depth generation method that satisfies differential privacy, comprising: constructing a trajectory depth generation model and obtaining a public dataset and a real dataset; training the trajectory depth generation model based on the public dataset, the real dataset, and a stochastic gradient descent algorithm based on differential privacy to obtain a trained trajectory depth generation model; and obtaining a trajectory dataset based on the trained trajectory depth generation model. This invention first uses a public dataset containing protected information about the city where the dataset is located to pre-train the generator and discriminator, then formally trains the generator and discriminator adversarially, and adds noise to the gradient of the discriminator based on DP-SGD during the discriminator training process. Compared with traditional trajectory generation methods that satisfy differential privacy, this method captures more features of the original trajectory under the same privacy budget, resulting in a more useful trajectory dataset. It overcomes the lack of privacy guarantees in existing trajectory depth generation models.

Description

Technical Field

[0001] This invention belongs to the field of trajectory data generation technology, and in particular relates to a high-quality trajectory depth generation method that satisfies differential privacy. Background Art

[0002] With the widespread adoption of smart devices and location-based services, location service providers and mobile operators have generated and collected vast amounts of user trajectory data. These trajectories hold significant value for many fields, including intelligent transportation and pandemic transmission simulation. However, because trajectory data involves personal privacy, directly publishing it poses a threat to user privacy. Therefore, ensuring user privacy while sharing trajectory data is a crucial issue of concern for both academia and industry, and trajectory generation is an important method for achieving these two goals.

[0003] In recent years, with the development of deep generative models, especially Generative Adversarial Networks (GANs), the technology for generating high-quality synthetic data such as images and videos has become quite mature. Generating high-quality synthetic trajectories using deep generative models has also become a research hotspot. Compared to traditional trajectory generation models, these deep generative models can capture richer spatiotemporal information and generate higher-quality trajectory data. However, deep models pose a risk of leaking the privacy of training data. Therefore, incorporating differential privacy into the design of deep generative models that satisfy differential privacy has recently attracted widespread attention from researchers. Summary of the Invention

[0004] The purpose of this invention is to provide a high-quality trajectory depth generation method that satisfies differential privacy, so as to solve the problems existing in the prior art.

[0005] To achieve the above objectives, this invention provides a high-quality trajectory depth generation method that satisfies differential privacy, comprising:

[0006] Construct a trajectory depth generation model and obtain a public dataset and a real dataset; train the trajectory depth generation model based on the public dataset, the real dataset, and a stochastic gradient descent algorithm based on differential privacy to obtain a trained trajectory depth generation model; obtain a trajectory dataset based on the trained trajectory depth generation model.

[0007] Optionally, the trajectory depth generation model includes a generator and a discriminator.

[0008] Optionally, the training includes: pre-training and formal training, wherein pre-training is performed based on the public dataset, and the model after pre-training predicts some trajectories; formal training is performed based on the real dataset and the partial trajectories, and the pre-training includes trajectory prediction training and trajectory pattern recognition training.

[0009] Optionally, the trajectory prediction training process includes: dividing the public dataset into a training dataset and a test dataset, and using two adjacent points in each trajectory consisting of n points, with the former point as input and the latter point as target, to pre-train the generator.

[0010] Optionally, the trajectory pattern recognition training process includes: processing the public dataset to obtain the processed dataset, and training the discriminator based on the processed dataset and the public dataset. The processing of the public dataset includes two methods: the first trajectory alteration method randomly selects a position point in the trajectory and replaces it with a point that is physically far away; the second trajectory alteration method randomly shuffles the order of positions in the trajectory within a time period; when processing each trajectory in the public dataset, either the first trajectory alteration method or the second trajectory alteration method is selected.

[0011] Optionally, the formal training process includes: discriminator training and generator training.

[0012] Optionally, the discriminator training process includes: completing the partial trajectory based on the pre-trained generator to obtain the completed trajectory; calculating the gradient based on the completed trajectory and the real trajectory in the real dataset; pruning the gradient; summing the processed gradient and adding Gaussian noise that satisfies differential privacy; updating the pre-trained discriminator based on the gradient descent method; and returning the loss of the discriminator to the pre-trained generator.

[0013] Optionally, the generator is trained using a reinforcement learning method, where the generator acts as an agent, a partial trajectory acts as a state, the next generated position acts as an action, the loss acts as a reward, and the generator is updated based on the gradient and gradient ascent method.

[0014] Optionally, the trainer and discriminator are continuously challenged until the trajectory depth generation model converges, thus obtaining the trained trajectory depth generation model.

[0015] The technical effects of this invention are as follows:

[0016] This invention is a trajectory data depth generation method based on MoveSim and DP-SGD, DP-MoveSim, which overcomes the lack of privacy protection in existing trajectory depth generation models. This invention first uses a public dataset of the city where the dataset is located to pre-train the generator and discriminator in the trajectory depth generation model DP-MoveSim. Then, it formally trains the generator and discriminator adversarially, and adds noise to the gradient of the discriminator based on DP-SGD during the training process. Compared with traditional trajectory generation methods that satisfy differential privacy, it captures more features of the original trajectory under the same privacy budget, resulting in a more useful trajectory dataset. Attached Figure Description

[0017] The accompanying drawings, which form part of this application, are used to provide a further understanding of this application. The illustrative embodiments and descriptions of this application are used to explain this application and do not constitute an undue limitation of this application. In the drawings:

[0018] Figure 1 This is a flowchart of the method in an embodiment of the present invention;

[0019] Figure 2 This is a schematic diagram of the system model in an embodiment of the present invention;

[0020] Figure 3 This is a schematic diagram of pre-training in an embodiment of the present invention;

[0021] Figure 4 This is a flowchart of the discriminator training process in an embodiment of the present invention. DETAILED DESCRIPTION

[0022] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.

[0023] It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases the steps shown or described may be executed in a different order than that shown here.

[0024] Example 1

[0025] like Figure 1-4 As shown, this embodiment provides a high-quality trajectory depth generation method that satisfies differential privacy, including:

[0026] DP-SGD specifically refers to a stochastic gradient descent algorithm based on differential privacy, proposed by Abadi et al. in their 2016 paper "Deep learning with Differential Privacy".

[0027] This embodiment uses a public dataset to pre-train the model and adds noise to the gradient of the discriminator based on DP-SGD.

[0028] The following steps are involved:

[0029] Step 1: Construct fake trajectories, and pre-train generator G and discriminator D based on fake trajectories and real trajectories;

[0030] Step 2: Begin adversarial training, generating the next location point based on generator G and partial trajectory prediction;

[0031] Step 3: The point generated in Step 2, together with the portion of the trajectory preceding that point, forms a new partial trajectory, which is then input into the discriminator D along with the actual trajectory.

[0032] Step 4: Based on the new generator G obtained from training, complete part of the trajectory, and train and update the discriminator D based on the completed trajectory and the real trajectory;

[0033] Step 5: Discriminator D returns the reward to the generator, and the generator G is updated;

[0034] Step 6: Repeat steps 2 through 5 until the model converges.

[0035] Specifically, the steps include the following:

[0036] (1) Using a public dataset to pre-train the model: Since both the discriminator and generator in MoveSim need to be pre-trained, unlike adversarial training, the generator and discriminator need to be exposed to real trajectory data during the pre-training stage. However, due to the excessive number of iterations during the pre-training stage, adding noise that satisfies differential privacy would significantly reduce the usefulness of the generated trajectories. Therefore, this invention uses a public dataset from the same city as the original dataset to pre-train the generator and discriminator.

[0037] (2) Noise Addition to the Discriminator Gradient Based on DP-SGD: Due to the high complexity of GAN network models, the GAN network may memorize some information from the training samples. Therefore, this invention uses the idea of ​​DP-SGD to perturb the gradient of the discriminator (by pruning, aggregating, and adding noise to the gradient of the discriminator) to make the discriminator satisfy differential privacy. Then, based on the post-processing theorem in differential privacy, it is proved that the generator and the synthetic trajectory data it generates satisfy differential privacy.

[0038] The model training in this method includes two processes: pre-training and formal training.

[0039] (1) Pre-training: such as Figure 3As shown, directly training a model using a dataset containing privacy information may lead to privacy leaks. Therefore, this invention uses a public dataset from the same city for pre-training to avoid privacy leaks. Pre-training mainly includes two tasks: trajectory prediction and trajectory pattern recognition. 1) Trajectory prediction: During the pre-training phase, the generator needs to predict the next position point based on the portion of the trajectory before the current position point. This invention pre-trains the generator G using trajectory prediction. First, the entire public trajectory dataset is divided into a training dataset and a test dataset. For each trajectory consisting of n points, between two adjacent positions, the previous point is used as the input, and the next point as the target, resulting in a total of n-1 sets of input and output. 2) Trajectory pattern recognition: The discriminator D is pre-trained using the public trajectory pattern recognition task. The trajectory pattern recognition task is essentially a binary classification task, used to distinguish whether the input trajectory satisfies important trajectory patterns, namely temporal periodicity and spatial continuity. First, fake trajectories are constructed by perturbing the original trajectory patterns. This invention disrupts the original trajectory pattern in two ways: a) randomly selecting a location point from the real trajectory and replacing it with a point physically far away; b) randomly shuffling the order of positions within a time period of the trajectory. Therefore, this invention pre-trains a discriminator based on fake trajectories constructed from public datasets and real trajectories.

[0040] (2) Formal Training: After pre-training the generator and discriminator, this invention trains the generator and discriminator using reinforcement learning methods. For example... Figure 4 As shown, the next location point is first predicted and generated based on the generator and partial trajectory. This generated location point is then combined with the previous partial trajectory to form a new partial trajectory, which, along with the real trajectory containing privacy information, is input into the discriminator. The discriminator update and the implementation of differential privacy (DP-SGD) mainly include the following steps:

[0041] Step 1: Complete the partial trajectory based on the latest generator G, and calculate the gradient based on the completed trajectory and the real trajectory containing privacy information.

[0042] Step 2: Given a threshold C, clip the gradient so that its magnitude does not exceed C.

[0043] Step 3: Sum the gradients calculated for different trajectories and add Gaussian noise that satisfies differential privacy.

[0044] Step 4: Update the discriminator parameters based on gradient descent.

[0045] After updating the discriminator, a generator is trained using reinforcement learning. The generator acts as the agent, with partially generated trajectories as its state, the next generated position as its action, and the loss from the discriminator as its reward. Based on the policy gradient algorithm, this invention generates the policy gradient by receiving the reward R(x) from the discriminator using a reinforcement learning algorithm.

[0046]

[0047] Where θ is the parameter of the generator G, x is the state, and the reward R(x) is the loss from the discriminator. Based on the gradient above... The parameter θ is updated using the gradient ascent method.

[0048] Finally, the generator and discriminator are trained repeatedly until the model converges.

[0049] Example 2

[0050] This embodiment provides a high-quality trajectory depth generation method that satisfies differential privacy, including:

[0051] Construct a trajectory depth generation model and obtain public and real datasets; train the trajectory depth generation model based on the public and real datasets and a stochastic gradient descent algorithm based on differential privacy to obtain the trained trajectory depth generation model; obtain the trajectory dataset based on the trained trajectory depth generation model.

[0052] In some embodiments, the trajectory depth generation model includes a generator and a discriminator.

[0053] In some embodiments, training includes pre-training and formal training, wherein pre-training is performed based on a public dataset, and the model predicted part of the trajectory after pre-training is used to predict the trajectory; formal training is performed based on a real dataset and part of the trajectory, and pre-training includes trajectory prediction training and trajectory pattern recognition training.

[0054] In some embodiments, the trajectory prediction training process includes: dividing the public dataset into a training dataset and a test dataset, and using two adjacent points in each trajectory consisting of n points, with the former point as the input and the latter point as the target, to pre-train the generator.

[0055] In some embodiments, the trajectory pattern recognition training process includes: processing a public dataset to obtain a processed dataset, and training a discriminator based on the processed dataset and the public dataset. The processing of the public dataset includes two methods: the first trajectory alteration method randomly selects a position point in the trajectory and replaces it with a point that is physically far away; the second trajectory alteration method randomly shuffles the order of positions in the trajectory within a time period; when processing each trajectory in the public dataset, either the first trajectory alteration method or the second trajectory alteration method is selected.

[0056] In some embodiments, the formal training process includes: discriminator training and generator training.

[0057] In some embodiments, the discriminator training process includes: completing a partial trajectory based on the pre-trained generator to obtain the completed trajectory; calculating the gradient based on the completed trajectory and the real trajectory in the real dataset; pruning the gradient; summing the processed gradient and adding Gaussian noise that satisfies differential privacy; updating the pre-trained discriminator based on the gradient descent method; and returning the loss of the discriminator to the pre-trained generator.

[0058] In some embodiments, a generator is trained using a reinforcement learning method, where the generator acts as an agent, a partial trajectory acts as a state, the next generated position acts as an action, the loss acts as a reward, and the generator is updated based on gradients and gradient ascent.

[0059] In some embodiments, the training generator and discriminator are continuously challenged until the trajectory depth generation model converges, thus obtaining the trained trajectory depth generation model.

[0060] The above description is merely a preferred embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A high-quality trajectory depth generation method that satisfies differential privacy, characterized in that, The following steps are involved: Construct a trajectory depth generation model and obtain public and real datasets; The trajectory depth generation model is trained based on the public dataset, the real dataset, and the stochastic gradient descent algorithm based on differential privacy to obtain the trained trajectory depth generation model. A trajectory dataset is obtained based on a trained trajectory depth generation model; The trajectory depth generation model includes a generator and a discriminator; The training includes pre-training and formal training. Pre-training is performed based on the public dataset, and the pre-trained model is used to predict some trajectories. Formal training is performed based on the real dataset and the partial trajectories. The pre-training includes trajectory prediction training and trajectory pattern recognition training. The trajectory prediction training process includes: dividing the public dataset into a training dataset and a test dataset, and using two adjacent points in each trajectory consisting of n points, with the former point as the input and the latter point as the target, to pre-train the generator; The trajectory pattern recognition training process includes: processing the public dataset to obtain a processed dataset; training the discriminator based on the processed dataset and the public dataset; wherein, there are two ways to process the public dataset: the first trajectory alteration method randomly selects a position point in the trajectory and replaces it with a point that is physically far away; the second trajectory alteration method randomly shuffles the order of positions in the trajectory within a time period; when processing each trajectory in the public dataset, either the first trajectory alteration method or the second trajectory alteration method is selected. The formal training process includes: discriminator training and generator training.

2. The high-quality trajectory depth generation method satisfying differential privacy according to claim 1, characterized in that, The discriminator training process includes: completing the partial trajectory based on the pre-trained generator to obtain the completed trajectory; calculating the gradient based on the completed trajectory and the real trajectory in the real dataset; pruning the gradient; summing the processed gradient and adding Gaussian noise that satisfies differential privacy; updating the pre-trained discriminator based on the gradient descent method; and returning the loss of the discriminator to the pre-trained generator.

3. The high-quality trajectory depth generation method satisfying differential privacy according to claim 2, characterized in that, The generator is trained using a reinforcement learning method, where the generator acts as an agent, a partial trajectory is the state, the next generated position is the action, the loss is the reward, and the generator is updated based on the gradient and gradient ascent method.

4. The high-quality trajectory depth generation method satisfying differential privacy according to claim 3, characterized in that, The training generator and discriminator are continuously challenged until the trajectory depth generation model converges, thus obtaining the trained trajectory depth generation model.

Images