An ais data route extraction method based on improved density peak clustering

Abstract

The application discloses a ship route extraction algorithm based on improved density peak clustering. The method comprises the following steps: (1) using a feature point extraction algorithm with an adaptive threshold, obtaining AIS data points with large changes in ground heading and ground speed as feature points; (2) performing spatial partitioning on the feature points according to the longitude and latitude of the feature points by using a quadtree, thereby dividing a plurality of partitions; (3) calculating the similarity distance between ship trajectories by using a FastDTW algorithm; (4) performing local clustering in the partitions by using an adaptive density peak clustering algorithm; (5) merging the clustering results of the plurality of partitions, thereby obtaining actual ship routes; and (6) parallelizing the processes of steps (3), (4) and (5) by using a Spark platform. The method can adapt to trajectory feature extraction under different trajectory forms and different orders of magnitude, has the characteristics of high clustering accuracy, high execution efficiency and suitability for large-scale AIS data mining.

Description

Technical Field

[0001] This invention belongs to the field of marine science and technology, and specifically relates to a method for extracting AIS data routes based on improved density peak clustering. Background Technology

[0002] With the development of economic globalization, maritime transport has become increasingly important, and the massive traffic volume has increased the difficulty of traffic management. Using Automatic Identification Systems (AIS) and adhering to the regulations of the International Maritime Organization (IMO) can yield abundant vessel navigation data. Extracting experienced routes from AIS data helps understand vessel behavior patterns within a region, the distribution patterns of vessel traffic flows, and identify common ports and route nodes. However, existing AIS data route extraction methods neglect the correlation factors between vessel traffic flows, resulting in insufficient mining of vessel AIS data, low efficiency, and difficulty in extracting effective routes from massive amounts of AIS data. Summary of the Invention

[0003] To overcome the problem that existing AIS data route extraction methods struggle to efficiently extract ship routes from massive AIS datasets, this invention provides an AIS data route extraction method based on improved density peak clustering. This method first preprocesses the acquired AIS data to filter out outliers. Then, an adaptive threshold feature point extraction algorithm is used to identify AIS data points with significant variations in ground heading and speed as feature points. Subsequently, quadtree spatial partitioning is performed based on the latitude and longitude of the feature points, resulting in multiple partitions. For each partition, the Fast Dynamic Time Wrapper (FastDTW) algorithm is used to calculate the similarity distance between ship trajectories. Next, an adaptive density peak clustering algorithm is used for local clustering within each partition. Finally, the clustering results from multiple partitions are merged to obtain the ship's empirical route. This method features high computational efficiency and applicability to large-scale AIS data.

[0004] The technical solution adopted by this invention to solve its technical problem is: a method for extracting AIS data flight routes based on improved density peak clustering, characterized by the following steps: Step 1: Preprocess the collected AIS data.

[0005] Step 2: Feature Point Extraction with Adaptive Threshold. First, the trajectory data is grouped according to the Maritime Mobile Service Identity (MMSI). Then, all AIS points for a single trajectory are sorted according to the UNIX timestamp, and the CRC and CRS of adjacent AIS points are calculated. Subsequently, the CRC and CRS are sorted, and differences are removed. Finally, the average sum of the CRC and CRS for each single trajectory is calculated as a threshold, and AIS points with values ​​greater than the average CRC and CRS are selected as feature points. Here, CRC is the rate of change of heading of adjacent AIS data points over a single trajectory time period, and CRS is the rate of change of velocity of adjacent AIS data points over a single trajectory time period. Their calculation formulas are shown below: in, The same ship at different timestamps Location point, and These represent the ship's heading relative to the ground and its speed, respectively.

[0006] Step 3: Perform quadtree spatial partitioning based on the latitude and longitude of the feature points.

[0007] Step 4: For each partition, use the FastDTW algorithm to calculate the similarity distance between ship trajectories. The specific steps are as follows: First, perform data abstraction on the original time series, iterating multiple times; then, run the Dynamic Time Wrapper (DTW) algorithm on the time series at a coarse-grained level; finally, further refine the grid of the wrought path obtained at the coarse-grained level onto a new time series, and calculate the similarity distance between ship trajectories.

[0008] Step 5: Perform local clustering within the partition using the adaptive density peak clustering algorithm. The specific steps are as follows: First, calculate the local density of data at each point within the partition based on the partition data. and relative distance The specific calculation method is as follows: in, For specific data partitioning, For the sample and distance, The cutoff distance is calculated as follows: in, , , It is a distance matrix between all data points, and the results are sorted in ascending order. This represents the number of all data points. Indicates the percentage.

[0009] Then, for each partition, the decision value is calculated based on the local distance and relative density of each point. The decision values ​​of all data within the partition are sorted in descending order, and the critical point is determined according to the following method. : in, Indicates the first The point and the first The slope of the line segment between points, The average slope difference between two adjacent points after sorting is calculated as follows: in, This represents the sum of the slope differences between any two adjacent points after sorting. The total number of samples.

[0010] Step 6: Merge the clustering results of multiple partitions to obtain the ship experience route. The specific steps are as follows: First, based on the data clustering results of each partition, all expansion points to each data partition are identified, and it is determined whether each point is a cluster center. These expansion points are then added to the merge candidate set. Next, the distance between points in two merge candidate sets is determined based on whether it is less than twice the expansion threshold. The system decides whether to merge local clusters and reassign cluster centers, thereby redistributing the remaining non-center points; finally, the local cluster merging is completed, and the final clustering result is obtained.

[0011] Step 7: Use the Spark platform to parallelize steps 4, 5, and 6, distribute the clustering task to multiple machines, reduce the memory consumption of a single machine, and cluster multiple partitions simultaneously to complete the clustering task together.

[0012] The beneficial effects of this invention are as follows: Addressing the difficulty in threshold selection during ship trajectory feature point extraction, this method discloses an adaptive trajectory feature point extraction method. This method can adapt to trajectory feature extraction under different trajectory shapes and orders of magnitude, reducing trajectory point density and improving analysis efficiency while ensuring the integrity of the trajectory shape. Furthermore, this method can process millions of data points in parallel and effectively improves the accuracy and speedup of clustering results, possessing broad application potential and practical value.

[0013] from Figure 2 It can be seen that the method disclosed in this invention is significantly better than traditional methods in terms of data extraction efficiency.

[0014] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments. Attached Figure Description

[0015] Figure 1 This is the execution flowchart of the present invention.

[0016] Figure 2 This is a comparison chart of the running time of the algorithm of this invention and the original density peak clustering algorithm (original DPC) on a vehicle GIS dataset. Detailed Implementation

[0017] like Figure 1 As shown, the specific steps of the AIS data route extraction method based on improved density peak clustering of the present invention are as follows: Step 1: Preprocess the AIS data to be tested, and filter out abnormal AIS data such as abnormal location, non-standard data, abnormal trajectory segment, and abnormal time.

[0018] Step 2: Feature Point Extraction with Adaptive Threshold. First, the trajectory data is grouped according to the MMSI identifier code. Then, all AIS points for a single trajectory are sorted according to the UNIX timestamp, and the CRC and CRS of adjacent AIS points are calculated. Subsequently, the CRC and CRS are sorted, and differences are removed. Finally, the average sum of the CRC and CRS for each single trajectory is calculated as a threshold, and AIS points with values ​​greater than the average CRC and CRS are selected as feature points. Here, CRC is the rate of change of heading of adjacent AIS data points over a single trajectory time segment, and CRS is the rate of change of velocity of adjacent AIS data points over a single trajectory time segment. Their calculation formulas are shown below: in, The same ship at different timestamps Location point, and These represent the ship's heading relative to the ground and its speed, respectively.

[0019] Step 3: Perform quadtree spatial partitioning based on the latitude and longitude of the feature points.

[0020] Step 4: For each partition, use the FastDTW algorithm to calculate the similarity distance between ship trajectories. The specific steps are as follows: First, perform data abstraction on the original time series, iterating multiple times; then, run the Dynamic Time Wrapper (DTW) algorithm on the time series at a coarse-grained level; finally, further refine the grid of the wrought path obtained at the coarse-grained level onto a new time series, and calculate the similarity distance between ship trajectories.

[0021] Step 5: Perform local clustering within the partition using the adaptive density peak clustering algorithm. The specific steps are as follows: First, calculate the local density of data at each point within the partition based on the partition data. and relative distance The specific calculation method is as follows: in, For specific data partitioning, For the sample and distance, The cutoff distance is calculated as follows: in, , , It is a distance matrix between all data points, and the results are sorted in ascending order. This represents the number of all data points. Indicates the percentage.

[0022] Then, for each partition, the decision value is calculated based on the local distance and relative density of each point. The decision values ​​of all data within the partition are sorted in descending order, and the critical point is determined according to the following method. : in, Indicates the first The point and the first The slope of the line segment between points, The average slope difference between two adjacent points after sorting is calculated as follows: in, This represents the sum of the slope differences between any two adjacent points after sorting. The total number of samples.

[0023] Step 6: Merge the clustering results of multiple partitions to obtain the ship experience route. The specific steps are as follows: First, based on the data clustering results of each partition, all expansion points to each data partition are identified, and it is determined whether each point is a cluster center. These expansion points are then added to the merge candidate set. Next, the distance between points in two merge candidate sets is determined based on whether it is less than twice the expansion threshold. The system decides whether to merge local clusters and reassign cluster centers, thereby redistributing the remaining non-center points; finally, the local cluster merging is completed, and the final clustering result is obtained.

[0024] Step 7: Use the Spark platform to parallelize steps 4, 5, and 6, distribute the clustering task to multiple machines, reduce the memory consumption of a single machine, and cluster multiple partitions simultaneously to complete the clustering task together.

[0025] from Figure 2 It can be seen that the method disclosed in this invention is significantly better than traditional methods in terms of data extraction efficiency.

Claims

1. A method for extracting flight routes from AIS data based on improved density peak clustering, characterized in that... Includes the following steps: Step 1: Preprocess the AIS data to be tested; Step 2: Feature point extraction with adaptive threshold; the specific steps are as follows: group the trajectory data according to the Waterborne Mobile Service Identifier; sort all AIS points of a single trajectory according to the UNIX timestamp, and calculate the CRC and CRS of adjacent AIS points; sort the CRC and CRS and remove the differences; calculate the average value of the CRC and CRS of a single trajectory as the threshold, and select AIS points with a value greater than the average value of CRC and CRS as feature points; where CRC is the rate of change of heading of adjacent AIS data points in a single trajectory time segment, and CRS is the rate of change of velocity of adjacent AIS data points in a single trajectory time segment, and their calculation formulas are as follows: in, The same ship at different timestamps Location point, and These represent the ship's heading relative to the ground and its speed, respectively. Step 3: Perform quadtree spatial partitioning based on the latitude and longitude of the feature points; Step 4: For each partition, use the fast dynamic time-rounding algorithm to calculate the similarity distance between ship trajectories. The specific steps are as follows: perform data abstraction on the original time series, and perform data abstraction iteratively multiple times; run the dynamic time-rounding algorithm on the time series at a coarse-grained level; further refine the squares traversed by the rounded path obtained at the coarse-grained level to a new time series, and calculate the similarity distance between ship trajectories. Step 5: Perform local clustering within the partition using the adaptive density peak clustering algorithm; the specific steps are: calculate the local density of data at each point within the partition based on the partition data. and relative distance The specific calculation method is as follows: in, For specific data partitioning, For the sample and distance, The cutoff distance is calculated as follows: in, , , This is a distance matrix between all data points, and the results are sorted in ascending order. This represents the number of all data points. Indicates the percentage; For each partition, the decision value is calculated based on the local distance and relative density of each point. The decision values ​​of all data within the partition are sorted in descending order, and the critical point is determined according to the following method. : in, Indicates the first The point and the first The slope of the line segment between points, The average slope difference between two adjacent points after sorting is calculated as follows: in, This represents the sum of the slope differences between any two adjacent points after sorting. The total number of samples; Step 6: Merge the clustering results of multiple partitions to obtain the ship experience route; the specific steps are as follows: based on the data clustering results of each partition, determine all extension points to each data partition, determine whether they are cluster centers, and add them to the merge candidate set; determine whether the distance between points in two merge candidate sets is less than twice the extension threshold. The decision is made on whether to merge local clusters and reassign cluster centers, thereby redistributing the remaining non-center points, completing the local cluster merging, and obtaining the final clustering result. Step 7: Use the Spark platform to parallelize steps 4, 5, and 6, distribute the clustering task to multiple machines, reduce the memory consumption of a single machine, and cluster multiple partitions simultaneously to complete the clustering task together.

Images