System and method for generating synthetic data set based on road vehicle positioning data

Abstract

The invention relates to a system and method for generating a synthetic data set based on road vehicle positioning data. The method includes iteratively selecting a seed record and determining a set of candidate partner records based on a matching two-tuple between the selected seed record and a potential partner record. The method further includes determining a sub-sequence of candidate partner records and selected seed records based on locations of the matching two-tuples; selecting a candidate partner record based on whether a length difference between the subsequences in terms of time, distance, or number of road events is less than a defined threshold; replacing the sub-sequence of the selected seed record with the sub-sequence of the selected candidate partner record to generate a mutated seed record; the mutated seed records are combined to generate a synthetic data set.

Description

Technical Field

[0001] This disclosure generally relates to generating synthetic mobility datasets based on raw datasets, and more specifically, to generating synthetic datasets based on road vehicle location data. Background Technology

[0002] With the development of location-based services and systems (such as location-based sensors and the Global Positioning System), mobility and trajectory data have been widely adopted. Acquiring mobility data is key to unlocking numerous applications, including urban planning, traffic monitoring, optimizing parking and charging infrastructure (known as "curb management"), altering travel behavior, monitoring vehicle health for software upgrades, and identifying dangerous driving behaviors to proactively mitigate risks. The impact of this acquisition is far-reaching, including reducing traffic congestion and pollution caused by searching for parking and charging stations, freeing up urban space for alternative uses such as bike lanes, delivery services, and green spaces, and improving public health, vehicle reliability, efficiency, and road safety.

[0003] The public availability of this type of data increases the risk of re-identification based on trajectory patterns that may highlight points of interest (POIs), whether private POIs such as home locations or public POIs such as office buildings, religious buildings, train and bus stations, and schools. Given the prevalence of personal information on public forums and social networks, this provides quasi-identifiers and could introduce another layer of risk. If an attacker possesses prior knowledge of an individual within the dataset, they could potentially launch inference attacks when that information is correlated with the trajectory data.

[0004] Traditional anonymization techniques can be applied to this type of data to help prevent identity re-identification, and these techniques typically involve data aggregation and obfuscation. However, this technique can significantly impact the usability of such data. Coarse data may lack the intended utility; for example, spatial density and location, frequent travel patterns, and spatiotemporal data points. Aggregation methods tend to remove granular information that is more important in the above situations, thus these methods tend to have insufficient fidelity. Summary of the Invention

[0005] Given the shortcomings of the aforementioned traditional methods, there is an urgent need to develop methods that can maintain the fidelity and usability of data, thereby enabling the application of aggregated insights and user-specific patterns to various application scenarios.

[0006] The disclosed embodiments provide a method for generating high-fidelity trajectory data, which is represented, for example, as a sequence of timestamped latitude and longitude points. The disclosed technique blurs the data in a specific manner, preserving the statistical properties of the data while making the precise location of individuals unidentifiable, thus achieving a balance between data fidelity and data privacy.

[0007] The disclosed embodiments synthesize data by reverse geocoding trajectories (e.g., GPS trajectories) into sequences of street names, and then exchanging similar street name patterns between records. Furthermore, dates of statistically similar trajectories are exchanged, and frequently acquired points of interest (POIs) are exchanged with similar POIs. The synthesized data is statistically similar to the source data and captures potential correlations and movement patterns in the source data. The disclosed embodiments do not require retaining records of the original data and avoid exposing specific POIs of the original data, thus enabling the disclosed method to have practical application value while preventing identity re-identification.

[0008] The disclosed embodiments provide an adjustable algorithm that can be configured to adjust fidelity and privacy. If the focus is on user-level behavioral analysis, the adjustable algorithm can be configured to adapt to the specific situation, and / or it can be configured for analysis at daily / weekly / monthly / grade-level or other timescales of interest. In embodiments, noise can be added to location system coordinates (e.g., GPS coordinates) and timestamps for further anonymization.

[0009] The disclosed technology maintains data fidelity while obfuscating the data in a specific way. This method preserves the statistical characteristics of the data while ensuring that the data cannot be identified by unauthorized users. This allows researchers and developers to access and analyze this data more broadly, thereby accelerating the development and improvement of new applications.

[0010] The disclosed embodiments include generating a synthetic longitudinal vehicle localization dataset, which includes converting a sequence of vehicle localization record data into a road sequence and identifying subsequence patterns in the road sequence. Feature vectors are determined based on these subsequence patterns, each feature vector representing a corresponding record. These feature vectors are embedded in a low-dimensional space. Seed records are iteratively selected from the records, and in each iteration: subsequence patterns are identified in a subset of records. Instances of subsequence patterns in the seed record are replaced with instances of similar subsequence patterns identified in the subset of records to form modified seed records. The iterations are repeated until all records have been selected as seed records. For each modified seed record, a timestamp is generated by preserving the lag between consecutive roads and exchanging start timestamps with similar records. The modified seed records are combined to form a synthetic road sequence dataset. The road sequences are then converted into vehicle localization records and compiled into a synthetic vehicle localization dataset.

[0011] In one aspect, the disclosed embodiments provide a method, system, and computer-readable medium for generating synthetic datasets based on road vehicle location data. The method includes: determining a sequence of road events for each of the N road vehicles based at least in part on trajectory records of N road vehicles, each road event corresponding to the travel of a corresponding vehicle on a specific road, thereby generating N records, where N is an integer. The method further includes: in each of the N records, identifying the occurrence of tuples in a unique set of tuples generated from the N records, each tuple corresponding to a transition between road events.

[0012] The method further includes: iteratively selecting a seed record from N records, and performing the following operations on each of the selected seed records. For each selected seed record, a set of candidate partner records is determined from the N records based at least in part on the number of matching tuples between the selected seed record and potential partner records from the N records. For each candidate partner record, a subsequence of each candidate partner record and a corresponding subsequence of the selected seed record are determined based at least in part on the corresponding positions of the matching tuples. A candidate partner record is selected from the set of candidate partner records based at least in part on the length difference in time, distance, or number of road events between the subsequence of the selected seed record and the corresponding subsequence of each candidate partner record being less than a defined threshold. The subsequence of the selected seed record is replaced with the subsequence of the selected candidate partner record to generate a mutated seed record.

[0013] After iteratively selecting all seed records from N records, the mutated seed records and any seed records without replacement subsequences are combined to generate a synthetic dataset containing N records.

[0014] The embodiments may individually or in combination include one or more of the following features.

[0015] The method may further include: receiving a raw dataset comprising trajectory records of N road vehicles, the trajectory records comprising variable-length sequences, each variable-length sequence comprising ordered or timestamped geographic coordinates; assigning each geographic coordinate to a geographic hexbin; performing reverse geocoding on the trajectory records to obtain the road name for each geographic coordinate; and combining the trajectory records to form a composite dataset, each record in the composite dataset comprising a vehicle identifier, latitude coordinates, longitude coordinates, date and timestamp, road name, and hexbin assignment.

[0016] This method may further include: associating N records ( r The method generates a dictionary of tuples from the pairs in the trajectory record to create a unique set of tuples and determines the frequency of each occurrence in each unique tuple. The method may further include swapping points of interest (POIs) within the trajectory record with POIs within a defined distance.

[0017] For each candidate partner record, the subsequence of the selected seed record and the corresponding subsequence of each candidate partner record can be defined as a set of tuples located between the position of the first selected matching tuple and the position of the second selected matching tuple in at least two matching tuples.

[0018] Determining the candidate partner record set may include: selecting q partner record sets from N records that are most similar to the selected seed record, and the similarity is based at least in part on the number of matching pairs between the selected seed record and potential partner records among the N records, where q is an integer. When selecting the q partner record sets, the similarity may be based at least in part on the number of matching pairs between the selected seed record and potential partner records among the N records divided by the total number of unique pairs within the selected seed record.

[0019] The method may further include: iteratively selecting seed records from N records in the synthetic dataset, and performing the next steps for each of the selected seed records.

[0020] For a selected seed record, a set of candidate partner records can be determined from the N records, at least in part, based on the number of matching pairs between the selected seed record and potential partner records among the N records. For each candidate partner record, a subsequence of each candidate partner record and a corresponding subsequence of the selected seed record can be determined, at least in part, based on the corresponding positions of the matching pairs. For each candidate partner record, the subsequence of the selected seed record and the corresponding subsequence of each candidate partner record can be defined as: (i) a set of pairs located between the start position of the seed record or each candidate partner record and the position of the matching pairs of one or more matching pairs, or (ii) a set of pairs located between the position of the matching pairs of one or more matching pairs and the end position of the seed record or each candidate partner record. Candidate partner records can be selected from the set of candidate partner records, at least in part, based on the determination that the length difference between the subsequence of the selected seed record and the corresponding subsequence of each candidate partner record in terms of time, distance, or number of road events is less than a defined threshold. The subsequence of the selected seed record can be replaced with the subsequence of the selected candidate partner record to generate a mutated seed record.

[0021] After iteratively selecting all seed records from N records, the mutated seed records and any seed records without replacement subsequences can be combined to generate a mutated synthetic dataset containing N records.

[0022] Determining the candidate partner record set may include: selecting a set of partner records with the same starting hexbin from N records, and for each candidate partner record, the subsequence of the selected seed record and the corresponding subsequence of each candidate partner record may be defined as a set of tuples located between the starting position of the seed record or the starting position of each candidate partner record and the position of the selected matching tuple of one or more matching tuples.

[0023] Determining the candidate partner record set may include: selecting a set of partner records with the same ending hexbin from N records, and for each candidate partner record, the subsequence of the selected seed record and the corresponding subsequence of each candidate partner record may be defined as a set of tuples located at the position of the selected matching tuple of one or more matching tuples and between the end position of the seed record or the end position of each candidate partner record.

[0024] When selecting one of the candidate partner records from the candidate partner record set, if the length difference of more than one candidate partner record is less than a defined threshold, one of the candidate partner records can be randomly selected. Attached Figure Description

[0025] Figure 1This is a block diagram of a system for generating and distributing synthetic longitudinal datasets based on road vehicle positioning data according to the disclosed embodiments.

[0026] Figure 2 This is a diagram of a synthetic data processing system according to a disclosed embodiment.

[0027] Figure 3 This describes an example of a double crossover exchange instance where a subsequence in the seed path is swapped with a subsequence from the contributing path.

[0028] Figure 4 This describes an example of a single-cross initiation swap instance where a subsequence in the seed path is exchanged with a subsequence from the contributing path.

[0029] Figure 5 This describes an example of a single-cross termination exchange instance where a subsequence in the seed path is exchanged with a subsequence from the contributing path.

[0030] Figure 6 It is a density map of GPS records / points contained in individual paths in the real and synthetic datasets of the T-drive dataset.

[0031] Figure 7 It is a density map of driving distances calculated for individual paths in the real and synthetic datasets of the T-drive dataset.

[0032] Figure 8 It is a heatmap of GPS record / point density on all paths in the real and synthetic datasets of the T-drive dataset.

[0033] Figure 9 This is a comparison chart showing the density of GPS records / points within geospatial grid cells (i.e., hexbins) in the synthetic dataset and the real dataset of the T-drive dataset.

[0034] Figure 10 It is a polar coordinate plot representing the density of GPS records / points on all paths in the real and synthetic datasets of the T-drive dataset as a function of time.

[0035] Figure 11 It is a density map of GPS records / points on all paths in the real and synthetic datasets of the T-drive dataset.

[0036] Figure 12 This is a schematic diagram illustrating how POI anonymization is provided by identifying the original Point of Interest (POI) and replacing it with a new POI that meets the prescribed criteria.

[0037] Figure 13The diagram shows the time-synthesized data, where (A) represents similar dates that have been swapped in the synthesized data, and (B) represents similar individuals that have swapped their start dates and times.

[0038] Figure 14 A density error metric is defined, which represents the Jensen-Shannon Divergence (JSD) between the density distributions across hexbins in the real and synthetic datasets.

[0039] Figure 15 An example of a summary metric table for cross-validating real and synthetic datasets from the Microsoft T-Drive dataset is shown.

[0040] Figure 16 This is a flowchart of a method for generating a synthetic dataset based on road vehicle location data according to a disclosed embodiment.

[0041] Where deemed appropriate, reference numerals may be repeated in the accompanying drawings to indicate corresponding or similar elements. Furthermore, some blocks depicted in the drawings may be combined into a single functional block. Detailed Implementation

[0042] In the following detailed description, numerous specific details are set forth in order to provide a thorough understanding of embodiments of the invention. However, it will be understood by those skilled in the art that embodiments of the invention can be practiced without these specific details. In other instances, well-known methods, processes, components, and circuits have not been described in detail so as not to obscure the invention.

[0043] The disclosed embodiments provide a system and method for addressing the aforementioned anonymization and privacy challenges by generating synthetic longitudinal geospatial datasets that still maintain high fidelity. The method converts vehicle location data from street addresses to road names. For each driver, the method identifies other drivers with similar sequences and then exchanges similar pattern instances (i.e., subsequences). This method generates high-fidelity synthetic longitudinal data while performing sufficient transformations on the original sequences to maintain anonymity.

[0044] Maintaining anonymity for mobility data (e.g., GPS data) and preventing identity re-identification is a technical challenge, as it requires the development, implementation, and maintenance of sophisticated methods and tools to ensure that sensitive personal geolocation information is protected from disclosure during data analysis and sharing. While the underlying motivations for ensuring data privacy may stem from ethical, legal, and regulatory considerations, the actual process of achieving and maintaining anonymity in geospatial data involves several technical aspects, including:

[0045] De-identification and anonymization: These processes involve removing personally identifiable information (PII), such as the location of points of interest and the exact spatiotemporal coordinates of the trajectory. This process aims to ensure that any remaining information cannot be traced back to a specific individual, thus preventing identity re-identification.

[0046] Data encryption: Securely storing and transmitting trajectory data requires the use of reliable encryption algorithms to protect information from unauthorized access or interception.

[0047] Creating synthetic datasets for trajectory data (e.g., GPS data) is a technical solution to the aforementioned problems because it addresses several challenges inherent in processing real-world trajectory data. Synthetic data generation techniques can create datasets that mimic the characteristics of the original data without revealing sensitive information, allowing researchers to maintain anonymity and data privacy while using the data. In some cases, access to and use of trajectory data for research can be limited due to various factors, including data protection regulations, ownership, and individual privacy issues. Synthetic datasets can help overcome these barriers by generating additional data with similar properties to the original data, enabling researchers to develop and validate models without directly accessing and using real data.

[0048] Furthermore, this technical solution involves algorithms and / or rules with specific characteristics that generate highly obfuscated synthetic data while preserving the statistical properties of the original dataset. As described below, the core of this technical solution lies in making specific improvements to the generation of synthetic datasets through the use of specific algorithms and / or rules. It is precisely the implementation of specific algorithms and / or rules, rather than simply using computers, that enhances anonymity and data privacy by providing improved data obfuscation, thereby improving existing technical processes. Therefore, compared to abstract results or effects that merely invoke general processes and mechanisms, this technical solution provides a concrete method for improving related technologies.

[0049] In summary, creating synthetic datasets for trajectory data is a technical solution to a specific problem because it addresses the particular challenges of achieving and maintaining anonymity when using real-world trajectory data (e.g., GPS data). This solution uses specific algorithms and / or rules to transform information into a new dataset with a set of specific characteristics (i.e., statistical features similar to the original dataset), which is then used and applied to enable researchers to conduct clinical trial data analysis. The algorithms and / or rules described in this paper do more than simply organize existing information into a new form—they generate entirely new datasets. Using synthetic data generated in this way can help overcome these challenges and more effectively develop, test, and deploy data-driven solutions in mobility research.

[0050] The following terms are used in this document. The term “road” or “road event” refers to an event in which a vehicle or individual is on a specific street, road, highway, alley, etc., in a travel trajectory. “Sequence” refers to an ordered set of road events or groups of road events. “Subsequence” or “partial sequence” refers to a set of road events or groups of road events, representing a portion of a sequence. In a specific context, “record” refers to a series of events in a dataset associated with a specific vehicle or individual (or, in some cases, “record” may have a more general meaning). “Individual,” “object,” or “driver” refers to a collection of people or vehicles associated with mobility trajectories, vehicle location data, or other types of mobility data—these terms are used interchangeably with the term “vehicle” in this document. The terms “exchange,” “perform an exchange,” “switch,” or “perform a swap” can refer to a unidirectional action, such as replacing a subsequence of a seed record with a corresponding subsequence of another record, or a bidirectional action, such as replacing a subsequence in a seed record with a subsequence of another record, and vice versa. The term "bigram" refers to two subsequences of events that are directly adjacent to each other, such as two road events, which can actually define a specific road intersection.

[0051] Figure 1 This is a block diagram of a system 100 for generating and distributing synthetic longitudinal datasets. System 100 receives mobility data, such as Global Positioning System (GPS) data, collected via system 110 from pedestrian traffic or moving vehicles. The data from system 100 may be in the form of a dataset, referred to as “raw dataset” 120, denoted as X herein. As discussed in further detail below, raw dataset 120 is processed by synthetic data processing system 150 to generate synthetic dataset 160, denoted as X’ herein. Synthetic dataset 160 can be distributed to researchers for analysis, for example, via a secure data sharing platform referred to as synthetic data distribution system 170, which makes the data available via network 180.

[0052] The original dataset 120 may contain multiple variable-length sequences, each consisting of ordered or timestamped coordinates including latitude and longitude values. The original dataset 120 may include multiple tables (T1, T2, ..., Tn), each containing a separate sequence. For example, GPS mobility data may be stored as multiple separate tables (e.g., in CSV format, typically used in spreadsheets), where each table provides data on the movement of a specific individual.

[0053] In this embodiment, the tables and records of the raw dataset 120 may be preprocessed by the data preprocessing system 130 into street addresses in a process known as reverse geocoding, with the aim of obtaining the street name corresponding to each vehicle positioning system (e.g., GPS) coordinate. The system 130 may also preprocess the raw dataset 120 by assigning each coordinate to a hexagonal geographic cell, in a grid of such cells, which is referred to herein as a "hexbin".

[0054] System 130 can preprocess the original dataset 120 by exchanging points of interest (POIs) within the trajectory with similar and nearby POIs, such as... Figure 12 As shown. If the driver spends more than T minutes at the location, the original POI can be identified in each individual record in dataset X. New POIs can be identified as POIs similar to and / or nearby (e.g., within a distance D of the original POI) based on attributes (e.g., replacing popular, positively rated restaurants with other popular, positively rated restaurants). New POIs are inserted into the original trajectory by replacing old POIs with new POIs, where the time spent at the original POI can now be spent at the corresponding new POI. Routes between the remaining trajectory and the new POIs can be generated by synthesizing routes using the route generation API. Any time difference due to the additional time spent commuting from the new POI to the remaining trajectory can be adjusted by modifying the time spent at the new POI. If no POIs similar to and / or near the original POI are identified, new POIs can be identified by relaxing the constraints on parameter D and defining the way similar POIs are defined, resulting in a larger pool of POIs for selecting new POIs. If a POI cannot be identified or the algorithm does not want to insert a new POI, the original POI can be removed. The time spent on the original POI can also be removed or evenly distributed across k GPS records (for example, the time spent can be allocated as follows: if k = 1, it is allocated to the GPS record after the POI; if k = -1, it is allocated to the GPS record before the POI; if k = 2, it is allocated to the two GPS records after the POI).

[0055] The tables and records from the original dataset 120 can be combined to form a composite dataset 140, which is designed as Y in this paper. The composite dataset 140 stores data in a variable data structure such as rows and columns. The composite dataset Y can contain records of N individuals, where each row has the individual's object ID (or vehicle ID), latitude coordinates, longitude coordinates, date and timestamp, road name, and hexbin assignment.

[0056] In the disclosed embodiments, data received from data collection system 110 generates a synthetic longitudinal dataset 160. The synthetic longitudinal dataset 160 is designed to simulate the statistical properties of the original dataset 120. This preserves the statistical properties of the original dataset 120, allowing researchers to draw accurate conclusions, while de-identifying and obfuscating the data to reduce the risk of individuals being re-identified. The statistical properties of the synthetic dataset 160 can be validated against the original dataset 120 to ensure that the synthetic dataset 160 closely approximates the original dataset. This validation can be accomplished using various statistical tests, such as regression analysis.

[0057] Synthetic datasets can be distributed to researchers for analysis, for example, via a secure data-sharing platform such as a synthetic data distribution system 170, which may involve establishing private or public application programming interfaces (APIs), creating data repositories, data portals, and / or using cloud-based data storage services with appropriate access control, authentication, and encryption mechanisms. Researchers can utilize synthetic datasets 160 for longitudinal analysis, where the risk of identifying individual participants is minimal.

[0058] Figure 2 This is a schematic diagram of the synthetic data processing system 150. As described above, a raw longitudinal dataset (X) can be received, which includes mobility trajectory records of N individuals (where N is an integer). The tables and records of the raw longitudinal dataset (X) can be preprocessed by the data preprocessing system 130 and then combined to form a composite dataset (Y). Each record may include the individual's object ID (or vehicle ID), latitude coordinates, longitude coordinates, date and timestamp, road name, and hexbin assignment. Based on the information in Y, the road sequencer 210 generates N records defining the road sequence of the N individuals for each sequence of road events determined by the N individuals. r The set is a sequence of road events ordered by the start date and time of an individual's journey on a road. The length of the road sequence varies from individual to individual because the number of roads traversed by each individual or vehicle throughout the entire journey is different.

[0059] Refer again Figure 2 Modify the engine 220 receive record ( r ), and use records ( r The modified record set is generated using a binary counter 230, which is then fed into the modification engine 260 for further modification. Within the modification engine 220, the binary counter 230 first uses the record set (…) rA dictionary of tuples is created using a tuple. A tuple represents a subsequence of two roads—for example, if an individual or vehicle travels first on road A and then on road B, or first on road B and then on road A, this represents the occurrence of the tuple (road A, road B). For each of the N individuals, a tuple counter 230 can count the occurrences of each unique tuple. For example, for a given individual among the N individuals, there may be m unique tuples, where m is an integer. For each of the m tuples among the N individuals, there exists a unique integer count p for the occurrences of the corresponding tuple in the corresponding individual.

[0060] Next, modify the partner finder 240 within engine 220 to determine a set of q partner individuals for each record in record r. This involves retrieving data from record (r). r Iteratively select seed records in ) rs ), and then record ( r The determination of the seed record () rs The most similar q partner records. The similarity between records is determined by the seed record ( rs The degree of overlap between the seed record () and the unique pair in the potential partner record is represented by the seed record () rs The number of unique pairs shared by the seed record and the partner record divided by the seed record ( rs The total number of unique pairs in the array is used to represent this.

[0061] In determining each seed record ( rs After the first q partner records of the seed record, the double crossover exchanger 250 will process the N seed records. rs Mutate each of the seed records. rs The dual crossover switch 250 will determine the seed record ( rs ) and q partner records ( rp The positions of all matching pairs between each of the pairs in the seed. If at least two matching pairs are found, the subsequences between these matching pairs are considered to be potential sequences to be swapped, where the subsequences in the seed are called... ss And the subsequences in the partners are called sp Then define the threshold. t threshold t An integer count representing the road length or a numerical count representing the minutes spent on the road, or both (where separate thresholds can be defined for the road and the time spent). t If the subsequence ss and spThe difference between the time spent or the road length (or both) is at a threshold. t Inside, it is believed sp yes ss Potential interchangeable subsequences. If in the seed record ( rs ) and Partner Records ( rp There are more than one potential subsequence among them. ss and sp To swap, randomly select a pair of subsequences from the potential subsequences and swap them.

[0062] If targeting the seed record ( rs ) and partner records ( rp A pair of commutative subsequences was found. ss and sp Then use the partner record ( rp ) in the subsequence ( sp Replace seed record ( rs ) subsequence in ss The seed record of the new mutation is represented as... rs '. Figure 3 An example of a subsequence being exchanged within a double cross-connector 250 is described. Figure 3 In the middle, the seed record ( rs ) is marked as "seed", and the partner record ( rp ) is marked as "Contributor". In the seed record ( rs ) and Partner Records ( rp Between ), the double cross-connect exchange 250 determines two pairs of matching tuples, marked with dashed boxes. Subsequence ss The length is 4, represented by a striped road pattern, while the subsequence sp The length is 2, represented by a solid pattern road. If the threshold t If there are 5 roads, then this pair of subsequences is considered... ss and sp They are a potential swap pair because the difference between the subsequence lengths is 2, which is less than 5. If this subsequence pair is chosen... ss and sp As the subsequence to be swapped, the double crossover exchange 250 will... ss and sp Swap, and a new seed record ( rs ') will look like Figure 3 The records marked as "mutant seeds" are consistent.

[0063] Obtain N new records from the dual crossover switch 250. rs'), these records will then be passed to the next modification engine 260 for further mutation. Inside the modification engine 260, the partner finder 270 first determines all potential partner records ( rs ') for each input seed record ( rp '), where the set of v partner records ( rp ') includes all records among the N records r whose sequences start with the same hexbin as the sequence in the seed record ( rs ').

[0064] Then, for each seed record ( rs '), the single - crossover start swapper 280 determines the positions of all the tuples in the seed record ( rs ') that have matching tuples in at least one of the v partner records ( rp '). The subsequence ss ' represents the part of the seed record ( rs ') from the start of the sequence to the position of the tuple containing the matching pair in the partner record ( rp '). The potential contributing subsequence sp ' represents the part of the partner record ( rp ') from the start of the sequence to the position of the tuple containing the matching pair with the seed record ( rs '). Then, the single - crossover start swapper 280 iteratively searches all pairs of subsequences rs ' and rp ' for the seed record ( ss ') and all v partner records ( sp '), and retains only a single pair rs ' and ss ' for each seed record ( sp '), in which the difference between the time spent in the subsequences in ss ' and sp ' is the smallest or the difference between the total number of roads is the smallest. If the difference in the road length or the time spent between the subsequences ss ' and sp ' is within a previously defined threshold ss ' and sp ', then t is used to replace sp ' inside the seed record ( rs ') to create a new mutated seed record ( ss ''). rs ''

[0065] Figure 4 describes an example of the subsequences swapped inside the single - crossover start swapper 280. In Figure 4 , the seed record (rs ') is marked as "seed", and the partner record ( rp ') is marked as "Contributor". In the seed record ( rs ') and Partner Records ( rp Between '), a single cross-start exchange 280 determines a pair of matching tuples, outlined in a dashed box. Seed sequence ss The length from the start of the sequence to the matching pair is 1, while the contributing subsequence sp The length of ' is also 1. If the threshold t If there are 5 roads, then this pair of subsequences is considered... ss 'and sp ' is a potential swap pair because the difference between the subsequence lengths is 0, which is less than 5. The single-cross starter exchange 280 will... ss 'and sp 'Swap, and a new seed record ( rs '') will be with Figure 4 The records marked as "mutant seed" are consistent.

[0066] Obtain N new records from single-cross starter switch 280 ( rs After that, these records are passed to the partner finder 290 to determine the seed record for each input. rs All potential partner records ('') rp ''), of which w are partner records ( rp The set includes N records r, with the sequence and seed record ( rs The end of the sequence in '') is the same as all records in hexbin.

[0067] Then, for each seed record ( rs ''), Single crossover end exchange 300 determines seed record ( rs In '') there are w partner records ( rp At least one of the subsequences contains the positions of all pairs of matching pairs. ss '' indicates the seed record ( rs From the records with partners ('') rp The portion of the sequence containing the position of the matched pair of tuples up to the end of the sequence. Potential contributing subsequences. sp '' indicates partner record ( rp From the seed record ('') rs The part containing the position of the matching pair of tuples to the end of the sequence. Then, the single crossover end exchanger 300 targets the seed record ( rs '') and all w partner records ( rp Iteratively search all subsequence pairs. ss ''and sp '', and only retain each seed record ( rs A single pair of '') ss ''and sp '', in this single pair ss ''and sp In '' ss ''and sp The subsequence in '' has the smallest difference in time spent or the smallest difference in the total number of roads. If the subsequence ss ''and sp The difference in road length or travel time is within a previously defined threshold. t Inside, then use sp Replace seed record ( rs '') ss '', to create a seed record for the new mutation ( rs ''').

[0068] Figure 5 An example of a subsequence being exchanged within a single-crossover end exchanger 300 is described. Figure 5 In the middle, the seed record ( rs '') is marked as "seed", and the partner record ( rp The '' mark is labeled as "Contributor". In the seed record ( rs '') and Partner Records ( rp Between '', a single crossover end exchange 300 determines a pair of matching tuples, outlined by a dashed box. Seed sequence ss The length from the matching pair to the end of the sequence is 1, while the contributing subsequence sp The length of '' is 3. If the threshold t If there are 5 roads, then this pair of subsequences is considered... ss ''and sp '' is a potential swap pair because the difference between the subsequence lengths is 2, which is less than 5. The single-cross terminator 300 will terminate the swap. ss ''and sp ''Swap, and a new seed record ( rs ''') will be with Figure 5 The records marked as "mutant seed" are consistent.

[0069] N new records were obtained from the single-cross terminator 300. rs After that, these records are passed to the next modification engine 310 for further mutation (see '''). Figure 13The modification engine 310 can perform time swapping based on date or individual objects. Within the modification engine 310, the density generator 320 first calculates the count of hexbin pairs, where a hexbin pair represents a subsequence of two hexbin assignments—for example, if an individual or vehicle passes through hexbin A first and then through hexbin B, this would represent the occurrence of the pair (hex A, hex B). If the modification engine 310 modifies based on date, the density generator 320 will perform time swapping on all N records ( rs The density generator 320 calculates the hexbin tuple count for each unique date found in the '''(). If the engine 310 is modified based on individual objects, the density generator 320 will calculate the hexbin tuple count for each unique object ID (or vehicle ID).

[0070] After density generator 320 calculates the density count of hexbin pairs, similarity calculator 330 calculates the cosine similarity score between individual density records. If modification engine 310 modifies based on date, similarity calculator 330 calculates the similarity score between each pair of unique dates. If modification engine modifies based on unique objects, similarity calculator 330 calculates the similarity score between each pair of unique object IDs (or vehicle IDs).

[0071] After the similarity calculator 330 calculates the cosine similarity score between all date pairs or unique object pairs, the time exchanger 340 will perform the exchange using the pair with the highest similarity score. If the modification engine 310 modifies based on date, the time exchanger 340 will replace all N records with the date having the highest similarity score. rs The date on '''. For example, if date B has the highest similarity to date A, then all records ( rs Date A in ''' will be replaced with date B. If the modification engine 310 modifies based on objects, the time exchanger 340 will replace N records with the start time of the object with the highest similarity score. rs The start date of all object IDs in the record. For example, if object ID B has the highest similarity to object ID A, the start date of object A in the record will be replaced with the start date from object B in the record. The new mutation record generated by the time exchanger 340 is called... rt .

[0072] Modifying the GPS jitter modifier 360 in Engine 350 will obtain the final N mutation seed records ( rt ) set, and by recording the seed record ( rtNoise is applied by multiplying the distance to each subsequent GPS point in the dataset by a noise ratio, which is expressed as a value between 0 and 1, inclusive. The final modified seed record is output from the GPS jitter modifier 360 and combined into the synthetic dataset (X') 370.

[0073] In addition to the above description and figures, the implementation scheme may be based at least in part on the following pseudocode for a method of synthesizing vehicle positioning system mobility data:

[0074] 1. Original dataset X; X consists of tables, where each table contains records of N individual vehicle location system points, including latitude and longitude coordinates, date and timestamp, road name and hexbin assignment.

[0075] 2. Merge all the tables of N individuals into a dataset Y, such that each row contains the object ID (or vehicle ID), latitude coordinates, longitude coordinates, road name, and hexbin assignment.

[0076] 3. Create road event sequences for N objects, thereby generating N records defining the road sequences of N individuals. r The set of road events is ordered by the start date and time of an object (i.e., a vehicle) on the road.

[0077] 4. For N records ( r Perform double crossover mutation on each record in )

[0078] For r i i. Create a dictionary of binary tuples

[0079] ii. For seed records ( rs ), from N records ( r Find q partner sets in )

[0080] iii. For q partner records ( rp (j) in the set:

[0081] 1. If rs and rp There must be at least two matching pairs between them:

[0082] a. If the sequence ss and sp The difference in road sequence length or duration is at a threshold t Inside:

[0083] i. Subsequences in the seed ss Subsequences in partners sp Swap to obtain a new seed record ( rs ').

[0084] 5. For N records ( rs Perform a single crossover mutation on each record in ').

[0085] a. Regarding rs 'in i :

[0086] i. Record the seed ( rs Find v partner sets that match the starting hexbin.

[0087] ii. For v partner records ( rp ') concentrated j:

[0088] 1. If rs 'and rp There must be at least one matching pair between them:

[0089] a. will ss Consider it as a seed subsequence from the start of the sequence to the matching pair. sp Consider it as a partner subsequence from the start of the sequence to the matching tuple.

[0090] b. If the sequence ss 'and sp The difference in road sequence length or duration is at a threshold t Inside:

[0091] i. Subsequences in the seed ss 'Subsequences in Partners' sp 'Swap to obtain a new seed record ( rs '')

[0092] iii. Record the seed ( rs Find v partner sets that have a matching end hexbin.

[0093] iv. For v partner records ( rp '') concentrated j:

[0094] 1. If rs ''and rp There is at least one matching tuple between '':

[0095] a. will ss '' is considered as a seed subsequence from the position of the matching pair to the end of the sequence, and will sp '' is considered as a partner subsequence from the position of the matching pair to the end of the sequence.

[0096] b. If the sequence ss ''and sp The difference in road sequence length or duration is at a threshold. t Inside:

[0097] i. Subsequences in the seed ss ''and subsequences in the partner sp 'Swap to obtain a new seed record ( rs ''').

[0098] 6. For N records ( rs Each record in ''') undergoes an execution time mutation:

[0099] a. Regarding rs i in ''':

[0100] i. If the mutation occurs based on the date:

[0101] 1. For N records ( rs For each unique date in ''', compute the numeric attribute (count / density) of the n-tuple (e.g., hexbin or road sequence), where n is an integer value greater than or equal to 1.

[0102] 2. Use a similarity calculator to calculate the similarity score (e.g., cosine similarity) for the attributes of each pair of dates.

[0103] 3. For N records ( rs For each date in ''', swap date A with date B, which has the highest similarity score, to obtain N records. rt )

[0104] ii. If a mutation occurs based on the object:

[0105] 1. Regarding the records ( rs For each of the N unique objects in a given set of n-tuples (e.g., hexbin or road sequences), compute the numerical properties (count / density) of the n-tuple.

[0106] 2. Use a similarity calculator to calculate the similarity score (e.g., cosine similarity) for each pair of objects' attributes.

[0107] 3. For N records ( rs For each object ID (or vehicle ID) in ''', swap the start date of object A with the start date of object B, which has the highest similarity score, to obtain N records. rt ).

[0108] 7. For N records ( rt Each record in the record performs vehicle positioning system jitter:

[0109] a. Regarding rt In i :

[0110] i. Apply noise to the vehicle positioning system coordinates by multiplying the distance of each subsequent point by the noise ratio to obtain the final seed record.

[0111] b. Combine all modified seed records to obtain the synthetic dataset (X').

[0112] To evaluate the fidelity of the synthetic longitudinal dataset, cross-validation tests were performed to compare the properties of the synthetic dataset with those of the real dataset, thereby assessing the extent to which the latent properties of the real dataset are preserved. As discussed in further detail below, many metrics can be used in cross-validation, such as density error, run-length error, pattern score and error, length error, etc.

[0113] Figures 6 to 12 This is an example of a density distribution plot obtained by cross-validation of a real dataset and a synthetic dataset from the Microsoft T-Drive taxi trajectory dataset, which includes GPS trajectories of a large number of taxis measured over a period of time. Figure 7 The density distribution of the total distance traveled in kilometers for each object ID is shown in the comparison between the real dataset and the synthetic dataset. Figure 6 The density distribution of the total number of GPS points captured for each object ID is shown in the comparison between the real dataset and the synthetic dataset. Figure 9 The line graph shows a comparison of the frequency of each hexbin assignment in the real dataset from the Microsoft T-Drive dataset with the frequency of each hexbin assignment in the synthetic dataset. For this graph, points close to the dashed line represent cases where the hexbin frequencies are very similar. Figure 9 In the middle, plot the frequency r 2 The value is 0.97, which is close to the ideal value of 1, indicating that the synthetic data has a high fidelity to the real data.

[0114] Figure 10 and Figure 11 This is an example of a time polar plot for time cross-validation of a real dataset and a synthetic dataset from the Microsoft T-Drive dataset. Figure 10 This shows the temporal distribution of all object IDs across all GPS points over a 24-hour period. Figure 11 The density maps of GPS records / points over time are shown for all paths in the real and synthetic datasets of the T-drive dataset.

[0115] Figure 15 This example shows a summary metrics table used for cross-validating real and synthetic datasets from the Microsoft T-Drive dataset. The "Metrics" column includes a description of the metric, the "Real Data" column represents the metric results on the real dataset, and the "Synthetic Data" column represents the metric results on the synthetic dataset.

[0116] Figure 8 This is an example chart showing the latitude and longitude coordinates of all points from the real and synthetic datasets from the Microsoft T-Drive dataset. The chart shows all GPS points from the source and synthetic datasets plotted side-by-side as hexbins, where darker hex cells represent denser hexbins.

[0117] Figure 14 A density error metric is defined, which represents the Jensen-Shannon divergence (JSD) between the density distributions across hexbins in the real and synthetic datasets. For the density error metric, a lower JSD value indicates a lower error, and therefore indicates that the hexbin distributions of the real and synthetic datasets are closer. Figure 14 A travel error metric is also defined, which represents the JSD between the start and end hexbin assignments of each trajectory in the real dataset and the synthetic dataset. Figure 14 The travel error value is 0.071. The smaller the JSD value of the travel error, the more similar the distributions of the start and end hexbins are between the real dataset and the synthetic dataset.

[0118] In addition to the metrics shown in the attached figures above, pattern score, pattern error, and length error metrics can also be used for cross-validation of real and synthetic datasets, such as... Figure 14 As shown. For pattern score and pattern error, a pattern is defined as an ordered sequence of consecutive hexbins traversed by the GPS trajectory. The pattern score is calculated by selecting the most frequently occurring head in the dataset. n Each dataset contains several head patterns, and an F1 score is calculated. A higher pattern score indicates a higher similarity between the head patterns in the real and synthetic datasets. The pattern error is calculated by selecting the most frequently occurring head patterns in the dataset. nThe algorithm identifies several patterns and calculates the relative differences between patterns appearing in the real and synthetic datasets. A smaller pattern error indicates a higher similarity between the head patterns in the real and synthetic datasets. The length error is calculated by summing the Euclidean distances between consecutive points to obtain the estimated run length. Then, the JSD is calculated between the distributions of the estimated run lengths in the real and synthetic datasets to obtain the run length error. A smaller run error value indicates a more similar distribution of run lengths in the real and synthetic datasets.

[0119] Figure 16 This is a flowchart of a method 800 for generating a synthetic dataset based on road vehicle location data according to a disclosed embodiment. The method includes: determining a sequence of road events for each of the N road vehicles based at least in part on trajectory records of N road vehicles, each road event corresponding to the travel of a corresponding vehicle on a specific road, thereby generating N records, where N is an integer (805). The method further includes: identifying the occurrence of tuples in a unique set of tuples generated from the N records in each of the N records, each tuple corresponding to a transition between road events (810); iteratively selecting seed records from the N records (815), and performing the following operations on each of the selected seed records.

[0120] The iterative portion of the method includes: for a selected seed record, determining a set of candidate partner records from the N records based at least in part on the number of matching tuples between the selected seed record and potential partner records from the N records (820). The method further includes: for each candidate partner record, determining a subsequence of each candidate partner record and a corresponding subsequence of the selected seed record based at least in part on the corresponding positions of the matching tuples (825); selecting a candidate partner record from the set of candidate partner records based at least in part on the determination that the length difference between the subsequence of the selected seed record and the corresponding subsequence of each candidate partner record in terms of time, distance, or number of road events is less than a defined threshold (830); replacing the subsequence of the selected seed record with the subsequence of the selected candidate partner record to generate a mutated seed record (835). The method further includes: after iteratively selecting all seed records from the N records, combining the mutated seed records with any seed records without substituted subsequences to generate a synthetic dataset containing N records (840).

[0121] Various aspects of the present invention may be implemented in the form of a system, a computer program product, or a method. Similarly, various aspects of the present invention may be implemented as hardware, software, or a combination of both. Various aspects of the present invention may be implemented as a computer program product stored in the form of computer-readable program code on one or more computer-readable media.

[0122] A computer-readable medium can be a computer-readable storage medium. A computer-readable storage medium can be, for example, an electronic, optical, magnetic, electromagnetic, infrared, or semiconductor system, device, or apparatus, or any combination thereof.

[0123] The computer program code in these embodiments can be written in any suitable programming language. The program code can run on a single computer or multiple computers. The computer may include a processing unit that communicates with a computer-usable medium, wherein the computer-usable medium contains an instruction set, and wherein the processing unit is designed to execute the instruction set.

[0124] The foregoing discussion is intended to illustrate the principles and various embodiments of the invention. Once the foregoing disclosure is fully understood, many variations and modifications will become apparent to those skilled in the art. The appended claims are intended to be construed as encompassing all such variations and modifications.

Claims

1. A method for generating a synthetic dataset based on road vehicle positioning data, the method comprising: The road event sequence of each of the N road vehicles is determined at least in part based on the trajectory records of the N road vehicles, each road event corresponding to the situation of the corresponding one of the N road vehicles traveling on a specific road, thereby generating N records, where N is an integer; In each of the N records, identify the occurrence of a pair of tuples in a unique set of pairs generated from the N records, where each pair of tuples corresponds to a transition between road events; Iteratively select a seed record from the N records, and perform the following for each selected seed record: For a selected seed record, a set of candidate partner records is determined from the N records based at least in part on the number of matching pairs between the selected seed record and potential partner records among the N records; For each candidate partner record, at least in part based on the corresponding position of the matching tuple, a subsequence of each candidate partner record and a corresponding subsequence of the selected seed record are determined; Candidate partner records are selected from the set of candidate partner records, at least in part, based on the fact that the length difference between the subsequence of the selected seed record and the corresponding subsequence of each candidate partner record in terms of time, distance, or number of road events is less than a defined threshold; and Replace the subsequence of the selected seed record with the subsequence of the selected candidate partner record to generate a mutated seed record; as well as After iteratively selecting all seed records from the N records, the mutated seed records and any seed records without replacement subsequences are combined to generate a synthetic dataset containing N records.

2. The method according to claim 1, further comprising: Receive a raw dataset including trajectory records of the N road vehicles, the trajectory records comprising variable-length sequences, each variable-length sequence comprising ordered or timestamped geographic coordinates; Assign each geographic coordinate to a geographic hexbin; Perform reverse geocoding on the trajectory records to obtain the road name for each geographic coordinate; as well as The trajectory records are combined to form a composite dataset, each record in which includes vehicle identification, latitude coordinates, longitude coordinates, date and timestamp, road name, and hexbin assignment.

3. The method according to claim 1, further comprising: A dictionary of tuples is generated by associating the tuples in the N records to generate the unique set of tuples, and the frequency of occurrence of each tuple in the unique set is determined.

4. The method according to claim 1, further comprising: Exchange the points of interest (POIs) within the trajectory record with POIs within a defined distance.

5. The method according to claim 1, wherein, For each candidate partner record, the subsequence of the selected seed record and the corresponding subsequence of each candidate partner record are defined as: the set of tuples located between the position of the first selected matching tuple and the position of the second selected matching tuple in at least two matching tuples.

6. The method according to claim 5, wherein, The determination of the candidate partner record set includes selecting q partner record sets from the N records that are most similar to the selected seed record, and the similarity is based at least in part on the number of matching pairs between the selected seed record and the potential partner records among the N records, where q is an integer.

7. The method according to claim 6, wherein, When selecting q partner records, the similarity is based at least in part on the number of matching pairs between the selected seed record and the potential partner records among the N records divided by the total number of unique pairs within the selected seed record.

8. The method of claim 5, further comprising: Seed records are iteratively selected from the N records of the synthetic dataset, and for each selected seed record, the following is performed: For a selected seed record, a set of candidate partner records is determined from the N records based at least in part on the number of matching pairs between the selected seed record and potential partner records among the N records; For each candidate partner record, at least in part based on the corresponding position of the matching tuple, a subsequence of each candidate partner record and a corresponding subsequence of the selected seed record are determined, wherein, for each candidate partner record, the subsequence of the selected seed record and the corresponding subsequence of each candidate partner record are defined as: (i) a set of tuples located between the start position of the seed record or each candidate partner record and the position of the matching tuple of one or more matching tuples, or (ii) a set of tuples located between the position of the matching tuple of one or more matching tuples and the end position of the seed record or each candidate partner record; Candidate partner records are selected from the set of candidate partner records, at least in part, based on the fact that the length difference between the subsequence of the selected seed record and the corresponding subsequence of each candidate partner record in terms of time, distance, or number of road events is less than a defined threshold; and Replace a subsequence of the selected seed record with a subsequence of the selected candidate partner record to generate a mutated seed record; and After iteratively selecting all seed records from the N records, the mutated seed records and any seed records without replacement subsequences are combined to generate a mutated synthetic dataset containing N records.

9. The method according to claim 1, wherein, The determination of the candidate partner record set includes selecting a set of partner records with the same starting hexbin from the N records, and For each candidate partner record, the subsequence of the selected seed record and the corresponding subsequence of each candidate partner record are defined as: a set of tuples located between the starting position of the seed record or the starting position of each candidate partner record and the position of the selected matching tuple of one or more matching tuples.

10. The method according to claim 1, wherein, The determination of the candidate partner record set includes selecting a set of partner records with the same ending hexbin from the N records, and For each candidate partner record, the subsequence of the selected seed record and the corresponding subsequence of each candidate partner record are defined as: a set of pairs located between the position of the selected matching pair of one or more matching pairs and the end position of the seed record or the end position of each candidate partner record.

11. The method according to claim 1, wherein, When selecting a record from the candidate partner record set, if the length difference between more than one candidate partner record is less than the defined threshold, then one record from the candidate partner record is randomly selected.

12. A system for generating synthetic datasets based on road vehicle location data, comprising: One or more processors, the one or more processors communicating with a memory, the memory storing instructions executable by the one or more processors to perform the method of any one of claims 1-11.

13. A non-transitory computer-readable medium comprising instructions that, when executed by one or more processors of a computer, cause the one or more processors to perform the method of any one of claims 1-11.

Images