Quality evaluation method for spatio-temporal trajectory big data

Abstract

The invention discloses a spatio-temporal trajectory big data-oriented quality evaluation method, which comprises the following steps of: acquiring a trajectory data set, and clustering and redistributing trajectories to obtain a plurality of clusters with a uniform number; and selectively sampling the trajectory data in each cluster to obtain a trajectory subset, performing quality evaluation on the trajectory subset from four dimensions of data validity, integrity, consistency and fairness, and outputting counts of violation indexes as a quality evaluation result of the data set. The method supports constraint check covering four dimensions (validity, integrity, consistency and fairness), supports off-line and on-line evaluation of batch historical trajectory data and streaming real-time trajectory data, and adopts an evaluation optimization strategy to improve evaluation efficiency.

Description

Technical Field

[0001] The present invention belongs to the technical field of data management and quality assessment, and in particular relates to a quality assessment method for spatiotemporal trajectory big data. Background Art

[0002] With the popularity of GPS (Global Positioning System) and the widespread use of related services, trajectory data plays an important role in many fields such as transportation and smart cities. However, due to problems such as inaccurate GPS measurements, low sampling rates, and transmission interruptions, trajectory data often contains errors, resulting in a decline in trajectory data quality, which in turn has a negative impact on downstream services. Therefore, evaluating the quality of trajectory data is a crucial but tedious task, which provides guidance for subsequent data cleaning and analysis.

[0003] In order to solve related problems, some studies have involved general data quality assessment, which can be divided into two categories: task-independent and task-related. Among them, task-independent data quality assessment, such as the literature [Su Yunxiang, Gong Yikun, Song Shaoxu. 2023. Time series data validity. International Computer Association Transactions on Data Management. 2023.1.1, No. 85: 1-85: 26], uses data features to quantify various quality dimensions; task-related data quality assessment, such as the literature [Cedric Lengri, Luca Rimanic, Luca Corral, Wu Wentao, Zhang Ce. 2023. Automatic feasibility study of machine learning based on data quality analysis: A case study of label noise. In the International Conference on Data Engineering. Pages 218-231] considers data quality in the context of specific tasks, especially in the field of machine learning.

[0004] However, existing technologies mainly focus on general data quality assessment problems, and therefore are not good at handling data quality assessment tasks specific to the trajectory level. First, trajectory data contains spatiotemporal dependency characteristics, and existing technologies only focus on patterns in general data, lacking the ability to effectively capture violations of spatiotemporal patterns in trajectory data; second, trajectory data usually exhibits specific group behaviors, such as morning and evening rush hours, and existing technologies only focus on detecting errors in a single univariate time series, lacking the ability to consider interdependencies; in addition, trajectory data is constrained by specific geographical environments, such as capturing the trajectory of car movement in urban areas must be consistent with the urban road network, and existing technologies ignore auxiliary information such as road networks and cannot capture the potential relationship between terrain context and trajectory; in addition, trajectory data is often generated in the form of data streams and used in real time, and existing technologies only focus on static data and cannot be effectively expanded to process streaming data; although some technologies support incremental quality assessment, they still face efficiency challenges when it comes to complex constraint checks. The above problems make it difficult for existing data quality assessment technologies to be applied to real-world trajectory spatiotemporal data quality assessment scenarios. Summary of the invention

[0005] In view of the above, the present invention provides a quality assessment method for spatiotemporal trajectory big data, which supports constraint checks covering four dimensions (validity, completeness, consistency, and fairness), supports offline and online evaluation of batch historical trajectory data and streaming real-time trajectory data, and adopts an evaluation optimization strategy to improve evaluation efficiency.

[0006] A quality assessment method for spatiotemporal trajectory big data includes the following steps: (1) Obtain a trajectory dataset, cluster the trajectories in the trajectory dataset, and redistribute them to obtain multiple clusters of uniform number; (2) Selectively sample a subset of trajectories from all clusters; (3) Quality assessment of trajectory subsets from the perspective of data validity, including checking the correctness of trajectory point timestamp constraints and range constraints; (4) Quality assessment of trajectory subsets from the perspective of data integrity, including checking missing points and missing values ​​of trajectory points; (5) Quality assessment of trajectory subsets from the perspective of data consistency, including checking trajectory smoothness constraints, trajectory length constraints, abnormal trajectories, abnormal trajectory stop points, and trajectory road network constraints; (6) Evaluate the quality of the trajectory subset from the perspective of data fairness, including the calculation of the spatial density of trajectory points in a specific area and the calculation of the temporal density of trajectory points in a specific period of time; (7) The number and proportion of trajectories and trajectory points detected according to steps (3) to (6) are counted together with the spatial density and temporal density calculated in step (7) and output as the final quality assessment result.

[0007] Furthermore, the trajectory data set is collected by GPS equipment to obtain a large number of trajectories, each trajectory is represented by a number of trajectory points, and each trajectory point contains index, timestamp, longitude and latitude information; the trajectory data set is a batch trajectory data set or a streaming trajectory data set, the batch trajectory data set uses an offline storage method to store historical trajectory data, and the streaming trajectory data uses an online storage method to store real-time trajectory data.

[0008] Furthermore, the specific process of clustering and redistributing the trajectories in the trajectory dataset in step (1) is as follows: 1.1 First, the entire two-dimensional geographic space is divided into several grid cells of equal size, and then each trajectory point in the trajectory is replaced by the grid cell to which it belongs, so that the trajectory in the trajectory dataset is converted into a grid representation form; 1.2 Arrange the trajectories in the trajectory dataset in descending order by length, take the first trajectory (i.e. the trajectory with the largest length), and initialize a cluster with this trajectory as the cluster center; 1.3 Take the next track in sequence T G , compare it with the cluster centers of all clusters, if there is a cluster whose cluster center is T G have k If the grid cells overlap, T G Add to this cluster, k is the set threshold, and if T G The length of is greater than the length of the cluster center, then T G As the new cluster center of the cluster; if there is no such cluster, T G Create a new cluster for the cluster center; 1.4 Cluster the trajectories in the trajectory dataset into multiple clusters by the method of step 1.3, and randomly select some trajectories from the large cluster with more than the average number of trajectories and assign them to the small cluster with less than the average number of trajectories to ensure that the number of trajectories in each cluster is uniform.

[0009] Furthermore, the specific implementation of step (2) is as follows: 2.1 Randomly select a cluster, divide the trajectories in the cluster into original trajectories and allocated trajectories according to their sources, and determine the proportion of these two types of trajectories in the cluster. The original trajectories are the trajectories obtained by clustering, and the allocated trajectories are the trajectories obtained from other clusters by redistribution. 2.2 Sample a random number from a uniform distribution between 0 and 1. If the random number is greater than a , then any trajectory from the original trajectory of the cluster is selected to be included in the trajectory subset, otherwise any trajectory from the assigned trajectory of the cluster is selected to be included in the trajectory subset, a is the proportion of the original trajectory in the cluster; 2.3 Repeat steps 2.1 to 2.2 until the number of trajectories in the trajectory subset reaches the set number requirement. This sampling method can ensure that the quality evaluation results on the trajectory subset are similar to the quality evaluation results on the complete trajectory dataset, reducing the computational cost.

[0010] Furthermore, the timestamp constraint in step (3) requires that the timestamps of the track points on the same track increase monotonically over time, and the range constraint requires that the longitude and latitude of each track point be within the longitude and latitude limit range of the specified area.

[0011] Furthermore, in step (4), for any two consecutive trajectory points on the trajectory, if the time interval between the two points is t Greater than 2Δ t , then it is determined that there are missing points in the trajectory, and the number of missing points between the two trajectory points is , Δ t is the sampling period of the trajectory, that is, the minimum time interval between all two consecutive points on the trajectory; if any trajectory point has no longitude value, latitude value or timestamp, it is determined that the trajectory point has a missing value.

[0012] Furthermore, the trajectory smoothness constraint in step (5) is to detect the position deviation through the Kalman filter, requiring that there should be no sudden position movement of the trajectory points in the trajectory; the trajectory length constraint is to require that the Haversine distance accumulation value of all two consecutive points in the trajectory is ≥ L th , L th is the trajectory length threshold and is set at three times the standard deviation of the mean trajectory length; The judgment criteria for abnormal trajectories are: for any trajectory in the trajectory subset T , if the trajectory T The number of outliers in is greater than the given threshold ρ , then determine T is an abnormal trajectory; for the trajectory T Any track point p ,like pThe number of neighbor points of is less than the given threshold η , then it is determined p is an outlier point; for any trajectory point on other trajectories in the trajectory subset q , if dist( p , q )≤ σ , then it is determined q for p Neighboring points, dist( p , q )for p and q Haversine distance, σ is a given distance threshold; The judgment criteria of abnormal trajectory stop points are as follows: first, all possible stop points are identified by using the TrajDBSCAN (Trajectory Density Clustering Spatial Noise Application) algorithm of a small neighborhood for all trajectories in the trajectory subset, and then cluster analysis is performed on the identified stop points, and the stop points that do not belong to any cluster are determined as abnormal trajectory stop points; The trajectory network constraint requires that the trajectory must match the path in the road network. The OHMM (based on online hidden Markov model) network matching algorithm is used to find the correspondence between each trajectory point in the trajectory and the road section in the road network. If the corresponding relationship does not exist, the corresponding trajectory point and its trajectory are judged to violate the trajectory network constraint.

[0013] Furthermore, in step (6), for a specific area Q S , the spatial density of trajectory points in this area ,in N is the total number of trajectory points in the trajectory subset, N ( Q S ) is the trajectory subset located in a specific area Q S The number of trajectory points in Area is the spatial distribution area of ​​the trajectory subset, Area ( Q S ) is a specific area Q S area; for a specific period of time Q t , the time density of trajectory points in this period , N ( Q t ) is the trajectory subset in a specific period Q t The number of trajectory points in Time is the overall time span of the trajectory subset, Time ( Q t ) is a specific period of time Q t time span.

[0014] A computer device includes a memory and a processor, wherein the memory stores a computer program, and the processor is used to execute the computer program to implement the above-mentioned quality assessment method for spatiotemporal trajectory big data.

[0015] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the quality assessment method for spatiotemporal trajectory big data.

[0016] The present invention designs a quality assessment method for spatiotemporal trajectory big data, which supports constraint checks covering four dimensions (validity, completeness, consistency, and fairness), supports offline and online evaluation of batch historical trajectory data and streaming real-time trajectory data, and adopts an evaluation optimization strategy to improve the evaluation efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 The figure is a flow chart of the quality assessment method for spatiotemporal trajectory big data according to the present invention.

[0018] Figure 2 The figure is a schematic diagram of the operation flow of clustering and redistributing trajectories in the present invention.

[0019] Figure 3 Schematic diagram of trajectory data redistribution and characterization trajectory selection in an embodiment of the present invention.

[0020] Figure 4 The figure is a schematic diagram of the operation flow of sampling the trajectories in the cluster to obtain the trajectory subset according to the present invention. DETAILED DESCRIPTION

[0021] In order to describe the present invention more specifically, the technical solution of the present invention is described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0022] like Figure 1 As shown, the quality assessment method for spatiotemporal trajectory big data of the present invention is applied in a terminal and includes the following steps: Step S11: Obtain a trajectory data set, which consists of a number of trajectories, each of which is represented by a number of trajectory points, wherein each trajectory point contains a trajectory index, a timestamp, a longitude, and a latitude; for the trajectory data stored in batches, an offline storage method is used for storage, and for the trajectory data flowing in in real time, an online storage method is used for storage.

[0023] Specifically, the trajectory dataset is mainly obtained through GPS and consists of several trajectories. Each trajectory consists of several GPS sampling points, which become a data record, containing information such as the longitude, latitude and timestamp of the sampling point, that is, . Historical batch trajectory data refers to trajectory data that has been collected and stored. This data can be processed periodically or once. For this type of data, an offline storage method can be used, that is, the data is stored in a database or data warehouse, and then analyzed and processed through batch jobs. The offline storage method allows complex queries and analysis of large amounts of data, but is not suitable for scenarios that require real-time response. Real-time streaming trajectory data refers to data collected continuously from mobile objects. This data needs to be processed immediately to support real-time applications. For streaming data, an online storage method is used, that is, the data is stored in memory or a high-speed storage system and can be processed and analyzed in real time. The online storage method supports fast data access and processing and is suitable for applications that require instant feedback. For offline storage, traditional relational database management systems (RDBMS) or object-oriented databases such as PostgreSQL (Postgresql) and MySQL (My Structured Query Language) can be used. For online storage, you might want to use an in-memory database like Redis (a remote dictionary service), or a stream processing framework like Apache Kafka, Apache Flink, or Spark Streaming.

[0024] Step S12: Cluster and redistribute the trajectories to obtain a uniform number of trajectory clusters. The specific operation process is as follows: Figure 2 As shown: S21: Convert each original trajectory data into a grid representation form. This process involves dividing the two-dimensional geographic space into a number of grid cells of equal size, and representing each trajectory point with the grid cell identifier to which the trajectory point belongs.

[0025] Specifically, the trajectory represented in the free geographic space is transformed into the grid space for representation. Given a two-dimensional free space, we divide the space into 2 θ ×2 θ equal-sized grid cells to construct a grid index G , 2 θ The free geospatial trajectory is converted to a grid-space trajectory by replacing each point in the trajectory with the ID of the grid cell it belongs to. After applying the above method, the trajectory is simplified to a sequence of discrete grid cells.

[0026] S22: Arrange the gridded trajectories in descending order of trajectory length and perform a clustering process: traverse each trajectory to check whether the number of grid cells overlapping with the cluster center trajectory of a certain trajectory cluster exceeds a certain threshold. If so, add the trajectory to the corresponding cluster, otherwise use the trajectory as a cluster center to form a new cluster.

[0027] Specifically, given a grid space trajectory dataset, we first sort the trajectories in descending order of length, and then initialize an empty list to store the cluster center of each cluster. In the actual clustering process, the longest trajectory in the cluster is regarded as the cluster center. We traverse the sorted grid space trajectories and select the currently visited trajectory. T G , check whether it has at least k grid cells intersect with any current cluster center. T G At least k Grid cells and cluster centers c overlap, add it to the c The corresponding cluster of the cluster center C And determine whether the cluster center needs to be modified, otherwise T G Initialize a new cluster for the cluster center C new ,This process continues until all trajectories have been visited.

[0028] S23: Randomly select some trajectories from the large cluster with more than the average number of trajectories and assign them to the small cluster with less than the average number of trajectories to ensure that the number of trajectories in each cluster is consistent.

[0029] Specifically, the data size of each cluster is expressed by Normalization is performed, where | C i | indicates a cluster i The number of trajectories contained, K represents the number of clusters, the denominator represents the number of all trajectories, and the average number of trajectories of all clusters after normalization is 1; Figure 3 In a specific embodiment shown, the normalized sizes of the clusters are 0.4, 1.2, 0.8 and 1.6 respectively. The reallocation method first creates two queues Q ＞1 and Q ＜1 , Q ＞1 The cluster IDs with a storage volume greater than 1, and Q ＜1 Store cluster IDs with data volume less than 1 and initialize an empty dictionary ATo record the distribution relationship between data distributors and data receivers. When both queues are not empty, the algorithm starts Q ＞1 Take a cluster from l , while from Q ＜1 Take a cluster from s , then A Recorded in s Receive from l The trajectory data of the cluster is updated to meet the requirement of 1 data volume. l The size of the cluster is determined based on the updated size. l Join again Q ＞1 The process continues until the sizes of all clusters are balanced to 1 or the queue becomes empty, and finally uniformly sized clusters are obtained. Figure 3 Cluster 4 (red blocks) donates redundant trajectories to cluster 1 (yellow blocks), and cluster 2 (green blocks) donates redundant trajectories to cluster 3 (blue blocks).

[0030] Step S13: Selectively sample the trajectory data in each uniform trajectory cluster to obtain a trajectory subset. The sampling ensures that the quality evaluation result of the complete trajectory data set is similar to the quality evaluation result on the trajectory subset. The specific operation process is as follows: Figure 4 As shown: S31: Randomly select a specific cluster and sample a trajectory from the cluster to include it in the trajectory subset.

[0031] Specifically, after the clustering process is completed, n clusters, using a random number generation algorithm to generate a random integer i ∈[1, n ], and then i The clusters are sampled.

[0032] S32: The trajectories in a cluster can be divided into the original part (from the current cluster) and the allocated part (from other clusters) according to their sources. The proportion of the number of trajectories in the original part to the total number of trajectories in the cluster is p (The proportion of the distribution part is 1- p ), sample a random number from a uniform distribution between 0 and 1, if the random number is greater than p Then a trajectory from the original part is sampled, otherwise a trajectory from the assigned part is sampled.

[0033] Specifically, the Alias ​​sampling process first generates a random number uniformly distributed between 0 and 1. r ,like r Less than the selected ID is i The normalized size of the cluster isd i , then from the cluster C i Sample a trajectory from , otherwise from the data distributor-receiver relationship A The corresponding cluster C Ai A trajectory is sampled in , and the process is repeated until the sampled representative trajectories meet the quantity requirements; this method ensures that the sampled representative trajectory can be consistent with the overall trajectory data distribution. Figure 3 In the specific embodiment shown, cluster 1 is selected in the first case because the random number r = 0.3 is smaller than the normalized size of cluster 1, 0.4, so a trajectory is sampled from the yellow part (original data) of cluster 1; cluster 3 is selected in the second case because the random number r = 0.9 is larger than the normalized size of cluster 3, 0.8, so a trajectory is sampled from the green portion (assigned data) of cluster 3. This redirection mechanism ensures that both the original and assigned portions of the cluster data are appropriately sampled even after trajectory reallocation.

[0034] S33: Repeat the above sampling process until the number of trajectories in the trajectory subset reaches a preset number.

[0035] Step S14: perform a quality assessment on the trajectory subset from the perspective of data validity, including checking the correctness of individual trajectory points such as timestamp constraints and range constraints.

[0036] Specifically, validity refers to the extent to which trajectory data conforms to the underlying spatiotemporal pattern, which imposes timestamp and range constraints on individual points in the trajectory. The timestamp constraint ensures that the timestamps of trajectory points are monotonically increasing over time, defined as ,in p Represents a trajectory point, which is composed of a triplet ( x , y , t ). The range constraint limits the location range of the GPS point and is defined as: ,in x min and x max denote the minimum and maximum longitudes of the considered area, respectively, y min and y max Respectively represent the minimum and maximum latitude of the considered area. If the area range is not specified, it is determined by the latitude and longitude value range itself. In both offline and online evaluation settings, the GPS points in the dataset are traversed to check the timestamp constraint and range constraint.

[0037] Step S15: perform quality assessment on the trajectory subset from the perspective of data integrity, including checking missing data such as trajectory missing points and trajectory missing values.

[0038] Specifically, integrity refers to the completeness and information richness of trajectory data, considering two situations: missing points in trajectories and missing values ​​of trajectory points. Missing points in trajectories are defined as , where Δ t Indicates the sampling period of the trajectory. If the time interval between two consecutive points is abnormally long, there is a missing point. The number of missing points in the trajectory measures the sampling stability of the trajectory data. The more missing points in the trajectory, the more uneven the data sampling. The sampling period of a trajectory is represented by the time interval with the highest frequency between all consecutive GPS points in the trajectory. In both offline and online evaluation settings, the GPS points in the dataset are traversed to count the missing points of the trajectory. For any consecutive points p i+1 and p i , the number of missing points between the two is estimated to be The missing value of a trajectory point is defined as If a GPS point does not have a longitude value, latitude value, or timestamp, we consider it a missing value. The missing value of a trajectory point measures the missingness of the longitude, latitude, and timestamp data of the trajectory point. In both offline and online evaluation settings, we traverse the GPS points in the dataset to count the missing values ​​of the trajectory points.

[0039] Step S16: perform quality assessment on the trajectory subset from the perspective of data consistency, including checking the trajectory smoothness constraint, trajectory length constraint, abnormal trajectory, abnormal trajectory stop point, and trajectory road network constraint, etc., which violate the association relationship between trajectory points or between trajectories.

[0040] Specifically, consistency refers to the degree to which trajectory data maintains a set of semantic rules, considering five situations: trajectory smoothness constraint, trajectory length constraint, abnormal trajectory, trajectory abnormal stop point and trajectory road network constraint.

[0041] The trajectory smoothness constraint limits the position change of trajectory points and requires that there should be no sudden position movement of trajectory points in the trajectory data; Kalman filters are deployed in both offline and online evaluation settings to detect position deviations.

[0042] The trajectory length constraint is defined as ,in dist The distance function measures the Haversine distance between two consecutive trajectory points. L th is the trajectory length threshold. Based on the three sigma rule, the trajectory length threshold L thDefined as three times the standard deviation of the mean trajectory length. In the offline evaluation setting, the trajectories in the dataset are traversed to check the trajectory length constraint; in the online evaluation setting, only GPS points in the sliding window are visited. k If there is no GPS point flowing into the time window, the trajectory flow is considered to have ended and the trajectory length constraint is checked.

[0043] The definition of abnormal trajectory is: from the trajectory T The trajectory point p i The neighbor is defined as ,in σ is a given distance threshold, if p i The number of neighbors is less than or equal to the given threshold. ,but p i It is considered an outlier if the number of outliers in a trajectory exceeds a given threshold. ρ , then the trajectory is an abnormal trajectory. In the offline evaluation setting, TRAOD (Trajectory Outlier Detection) based on the above idea is used as the abnormal trajectory detection method. This method first segments the trajectory and then detects abnormal sub-trajectory segments based on distance; in the online evaluation setting, PTMDS (Probabilistic Trajectory Matching and Detection System), an extension of TRAOD on streaming trajectory data, is used as the abnormal trajectory detection method.

[0044] A stop point refers to an area where a mobile object temporarily stays. Stop points with specific geographical significance in different trajectories (such as service areas) can be well clustered, but abnormal stop points caused by unexpected situations (such as equipment failure) often appear randomly. An abnormal trajectory stop point is defined as an outlier data point (noise) generated after the stop point clustering. An abnormal trajectory stop point is a stop behavior caused by an abnormal situation. In the offline evaluation setting, the TrajDBSCAN method with a small neighborhood is first applied to all trajectories in the trajectory dataset to identify all possible stop points. Then, the identified stop points are clustered using DBSCAN (Density-Based Spatial Clustering of Applications with Noise) to find abnormal trajectory stop points that cannot be attributed to any cluster; in the online evaluation setting, the dynamic DBSCAN algorithm is used to replace the TrajDBSCAN method in the offline evaluation setting to adapt to streaming trajectory data.

[0045] Given a road network represented as a directed graph G =( V , E ), where the node v ∈ V Represents an intersection or the end of a road, edge e ∈ E Represents a road or path P is a sequence of connected road segments, i.e. The trajectory network constraints stipulate that each trajectory T Must be with G A path in P The trajectory network constraint measures the degree to which the trajectory follows the actual road network. In both offline and online evaluation settings, the OHMM (based on the online hidden Markov model) road network matching algorithm is used to match each trajectory point with a road segment in the road network. Once it is found that there is no correspondence between a trajectory point and a road segment, then this point and the trajectory it belongs to violate the road network constraint.

[0046] Step S17: perform quality assessment on the trajectory subset from the perspective of data fairness, including calculation of data distribution such as spatial density and temporal density.

[0047] Specifically, fairness refers to the extent to which biased information exists in the trajectory dataset, which will affect the accuracy of machine learning results in subsequent applications. Fairness considers both spatial density and temporal density. Spatial density is defined as ,in Q S is a rectangular area, For the dataset Located in Q S The set of trajectory points in is the total number of trajectory points in the dataset, Area is the spatial distribution area of ​​the dataset, Area ( Q S )for Q S Spatial distribution area. Spatial density compares the density of GPS points in a specific area with the overall density, measuring the data deviation in the spatial dimension. In both offline and online evaluation settings, quadtrees are constructed for all GPS points in the dataset to accelerate range queries through breadth-first search to find The time density is defined as ,in Q t =[ t min ,t max ] is the time span specified by the user, For Q t The set of trajectory points within the time span, is the time span of the entire dataset. Time density compares the density of GPS points in a specific time period with the overall density, measuring the data deviation in the time dimension. In both offline and online evaluation settings, we construct a segment tree for each leaf node of the quadtree to efficiently query the time period; in order to find GPS points in a given time period, we traverse and prune the segment tree through breadth-first search.

[0048] Step S18: Output the count of violation indicators as the evaluation result of the given trajectory data set.

[0049] Specifically, according to steps S14, S15, S16 and S17, a quality assessment is performed on a data subset representing the trajectory dataset from four data dimensions: data validity, completeness, consistency and fairness. Finally, the number and proportion of trajectories and trajectory points that violate constraints in each data dimension are output respectively. The assessment result of the data subset representing the trajectory dataset can effectively reflect the quality of the original trajectory dataset.

[0050] The above description of the embodiments is to facilitate the understanding and application of the present invention by those skilled in the art. It is obvious that those skilled in the art can easily make various modifications to the above embodiments and apply the general principles described herein to other embodiments without creative work. Therefore, the present invention is not limited to the above embodiments. Improvements and modifications made by those skilled in the art to the present invention based on the disclosure of the present invention should be within the protection scope of the present invention.

Claims

1. A quality assessment method for spatiotemporal trajectory big data, comprising the following steps: (1) Obtain a trajectory dataset, cluster the trajectories in the trajectory dataset, and redistribute them to obtain multiple clusters of uniform number; (2) Selectively sample a subset of trajectories from all clusters; (3) Quality assessment of trajectory subsets from the perspective of data validity, including checking the correctness of trajectory point timestamp constraints and range constraints; (4) Quality assessment of trajectory subsets from the perspective of data integrity, including checking missing points and missing values ​​of trajectory points; (5) Quality assessment of trajectory subsets from the perspective of data consistency, including checking trajectory smoothness constraints, trajectory length constraints, abnormal trajectories, abnormal trajectory stop points, and trajectory road network constraints; (6) Evaluate the quality of the trajectory subset from the perspective of data fairness, including the calculation of the spatial density of trajectory points in a specific area and the calculation of the temporal density of trajectory points in a specific period of time; (7) The number and proportion of trajectories and trajectory points detected according to steps (3) to (6) are counted together with the spatial density and temporal density calculated in step (7) and output as the final quality assessment result.

2. The quality assessment method for spatiotemporal trajectory big data according to claim 1 is characterized by: The trajectory data set is collected by GPS equipment to obtain a large number of trajectories. Each trajectory is represented by a number of trajectory points, and each trajectory point contains index, timestamp, longitude and latitude information. The trajectory data set is a batch trajectory data set or a streaming trajectory data set. The batch trajectory data set uses an offline storage method to store historical trajectory data, and the streaming trajectory data uses an online storage method to store real-time trajectory data.

3. The quality assessment method for spatiotemporal trajectory big data according to claim 1 is characterized by: The specific process of clustering and redistributing the trajectories in the trajectory dataset in step (1) is as follows: 1.1 First, the entire two-dimensional geographic space is divided into several grid cells of equal size, and then each trajectory point in the trajectory is replaced by the grid cell to which it belongs, so that the trajectory in the trajectory dataset is converted into a grid representation form; 1.2 Arrange the trajectories in the trajectory dataset in descending order by length, take the first trajectory (i.e. the trajectory with the largest length), and initialize a cluster with this trajectory as the cluster center; 1.3 Take the next track in sequence T G , compare it with the cluster centers of all clusters, if there is a cluster whose cluster center is T G have k If the grid cells overlap, T G Add to this cluster, k is the set threshold, and if T G The length of is greater than the length of the cluster center, then T G As the new cluster center of the cluster; If there is no such cluster, T G Create a new cluster for the cluster center; 1.4 Cluster the trajectories in the trajectory dataset into multiple clusters by the method of step 1.3, and randomly select some trajectories from the large cluster with more than the average number of trajectories to assign to the small cluster with less than the average number of trajectories to ensure that the number of trajectories in each cluster is uniform.

4. The quality assessment method for spatiotemporal trajectory big data according to claim 1, characterized in that: The specific implementation of step (2) is as follows: 2.1 Randomly select a cluster, divide the trajectories in the cluster into original trajectories and allocated trajectories according to their sources, and determine the proportion of these two types of trajectories in the cluster. The original trajectories are the trajectories obtained by clustering, and the allocated trajectories are the trajectories obtained from other clusters by redistribution. 2.2 Sample a random number from a uniform distribution between 0 and 1. If the random number is greater than a , then any trajectory from the original trajectory of the cluster is selected to be included in the trajectory subset, otherwise any trajectory from the assigned trajectory of the cluster is selected to be included in the trajectory subset, a is the proportion of the original trajectory in the cluster; 2.3 Repeat steps 2.1 to 2.2 until the number of trajectories in the trajectory subset reaches the set number requirement.

5. The quality assessment method for spatiotemporal trajectory big data according to claim 1 is characterized by: The timestamp constraint in step (3) requires that the timestamps of the track points on the same track increase monotonically over time, and the range constraint requires that the longitude and latitude of each track point be within the longitude and latitude limit range of the specified area.

6. The quality assessment method for spatiotemporal trajectory big data according to claim 1, characterized in that: In step (4), for any two consecutive trajectory points on the trajectory, if the time interval between the two points is t Greater than 2Δ t , then it is determined that there are missing points in the trajectory, and the number of missing points between the two trajectory points is , Δ t is the sampling period of the trajectory, that is, the minimum time interval between all two consecutive points on the trajectory; if any trajectory point has no longitude value, latitude value or timestamp, it is determined that the trajectory point has a missing value.

7. The quality assessment method for spatiotemporal trajectory big data according to claim 1, characterized in that: The trajectory smoothness constraint in step (5) is to detect the position deviation through the Kalman filter, requiring that there should be no sudden position movement of the trajectory points in the trajectory; the trajectory length constraint is to require that the cumulative value of the Haversine distance between all two consecutive points in the trajectory is ≥ L th , L th is the trajectory length threshold and is set at three times the standard deviation of the mean trajectory length; The judgment criteria for abnormal trajectories are: for any trajectory in the trajectory subset T , if the trajectory T The number of outliers in is greater than a given threshold ρ , then determine T is an abnormal trajectory; for the trajectory T Any track point p ,like p The number of neighbor points of is less than the given threshold η , then it is determined p is an outlier point; for any trajectory point on other trajectories in the trajectory subset q , if dist( p , q )≤ σ , then it is determined q for p Neighboring points, dist( p , q )for p and q Haversine distance, σ is a given distance threshold; The judgment criteria of abnormal trajectory stop points are as follows: first, all possible stop points are identified for all trajectories in the trajectory subset using the TrajDBSCAN algorithm with a small neighborhood, and then cluster analysis is performed on the identified stop points, and the stop points that do not belong to any cluster are determined to be abnormal trajectory stop points; The trajectory network constraint requires that the trajectory must match the path in the road network. The OHMM network matching algorithm is used to find the correspondence between each trajectory point in the trajectory and the road section in the road network. If the correspondence does not exist, the corresponding trajectory point and its trajectory are judged to violate the trajectory network constraint.

8. The quality assessment method for spatiotemporal trajectory big data according to claim 1, characterized in that: In step (6), for a specific area Q S , the spatial density of trajectory points in this area ,in N is the total number of trajectory points in the trajectory subset, N ( Q S ) is the trajectory subset located in a specific area Q S The number of trajectory points in Area is the spatial distribution area of ​​the trajectory subset, Area ( Q S ) is a specific area Q S area; for a specific period of time Q t , the time density of trajectory points in this period , N ( Q t ) is the trajectory subset in a specific period Q t The number of trajectory points in Time is the overall time span of the trajectory subset, Time ( Q t ) is a specific period of time Q t time span.

9. A computer device comprising a memory and a processor, wherein a computer program is stored in the memory, wherein: The processor is used to execute the computer program to implement a quality assessment method for spatiotemporal trajectory big data as described in any one of claims 1 to 8.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, it implements a quality assessment method for spatiotemporal trajectory big data as described in any one of claims 1 to 8.

Images