A method, device, and storage device for ship trajectory clustering based on AIS data

Abstract

This invention provides a method, device, and storage device for ship trajectory clustering based on AIS data, comprising: extracting historical trajectory data; preprocessing outliers and redundant values; resampling missing data using a linear interpolation algorithm; and segmenting the processed trajectories; extracting the start and end points of each trajectory segment; clustering the start and end points using an improved DBSCAN algorithm, and grouping trajectories with the same start and end point cluster into one class; for trajectories in the same class, calculating the similarity between trajectories using an improved DTW algorithm to further classify the trajectory categories; extracting feature trajectories for each class of trajectories; calculating the similarity of the trajectories using the improved DTW algorithm to obtain the final clustering result. Compared with traditional DBSCAN or DTW algorithms, this invention improves the efficiency and accuracy of trajectory clustering, can handle dense datasets well, and, through trajectory similarity calculation, can better adapt to clustering results for routes of arbitrary shapes.

Description

Technical Field

[0001] This invention relates to the field of ship navigation trajectory analysis, and in particular to a ship trajectory clustering method, device and storage device based on AIS data. Background Technology

[0002] With the continuous development of the domestic and international economy, the number of ships is constantly increasing, showing a trend towards larger size, higher speed, and greater intelligence. The density of ship traffic in coastal ports and estuaries is constantly increasing, and the navigation environment in these waterways is becoming more complex, placing higher demands on navigation management capabilities. The Automatic Identification System (AIS), as a carrier of ship motion information, provides massive amounts of ship positioning data for related research. However, the trajectory data located by AIS equipment can only reflect information such as the ship's latitude and longitude, positioning time, instantaneous speed, and heading, but cannot reflect the ship's navigational characteristics. Yet, this information is crucial for the planning, management, and control of maritime traffic. Therefore, mining and clustering ship trajectory information from AIS data can provide intelligent decision support for maritime supervision and ship traffic.

[0003] Clustering, as an important research direction in data mining technology, can achieve the reasonable classification of raw data, thereby discovering the inherent laws of things. Summary of the Invention

[0004] To address the aforementioned problems, this invention provides a ship trajectory clustering method, device, and storage device based on AIS data. The ship trajectory clustering method based on AIS data mainly includes:

[0005] S1: Extract ship trajectory data from the AIS database and preprocess abnormal data;

[0006] S2: The trajectory data of each preprocessed vessel is segmented and the navigation trajectory is extracted;

[0007] S3: The improved DBSCAN algorithm is used to cluster the start and end points of the navigation trajectory to obtain multiple clusters;

[0008] S4: Calculate the similarity distance for each cluster trajectory using the DTW algorithm to determine whether further clustering is needed;

[0009] S5: Extract feature trajectories for each cluster;

[0010] S6: Calculate the similarity distance of feature trajectories using the improved DTW algorithm, and obtain the final set of clusters.

[0011] Furthermore, in step S1, the abnormal data includes missing values, outliers, and redundancies in the data, and the preprocessing includes correcting latitude and longitude errors, cleaning up duplicate points, and filtering drift points in the abnormal data.

[0012] Further, in step S2, the segmentation process is as follows: based on the ship's position and positioning time, calculate the maximum positioning time within the specified distance neighborhood of all positions. If both times exceed the time threshold, it is considered that the ship is staying at that position. The trajectory is segmented according to the stopping point, and the navigation trajectory is extracted.

[0013] Furthermore, in step S3, the specific process of obtaining multiple clusters is as follows:

[0014] S31: Extract the starting point and ending point of the ship's navigation trajectory, construct a Ball-Tree for the point sets of the starting point and the ending point respectively, and obtain dataset D to improve the scanning speed of boundary points in the neighborhood of the core point;

[0015] S32: Input dataset D, scan radius R, and minimum number of points Pmin;

[0016] S33: Take a point P that has not been queried in the target dataset, and determine whether the number of points within the scanning radius of point P is greater than or equal to Pmin. If not, mark it as a noise point and repeat step S33; if yes, determine whether point P already belongs to a certain cluster C. If not, create a new cluster.

[0017] S34: Repeat step S33 for all unqueried points in cluster C until the last point is calculated, and then output the set of all clusters.

[0018] Furthermore, in step S4, the specific process of calculating the similarity distance using the improved DTW algorithm is as follows:

[0019] S41: Assume that the sequences of the two trajectories are A = {a1, a2, ..., an} and B = {b1, b2, ..., bm};

[0020] S42: Calculate the distance matrix Ei = {d(ai,b1),d(ai,b2),……,d(ai,bm)}, i = {1,2,……,n}, where Ei is the i-th row of the distance matrix E, d is the distance between two points, n is the length of sequence A, and m is the length of sequence B;

[0021] S43: Based on the distance matrix E, accumulate the distances from the top left corner to the bottom right corner, and minimize the accumulated distance. Then divide by the accumulated number of steps, which is the average accumulated distance. Use this distance as the similarity distance between the two trajectory sequences.

[0022] S44: Based on the similarity distance between all obtained trajectory sequences, set an appropriate threshold and determine whether it is necessary to further divide the trajectory clusters.

[0023] Furthermore, in step S5, for the trajectory similarity distances calculated in step S4, find the trajectory that has the smallest sum of distances to the other trajectories, and this trajectory is the feature trajectory.

[0024] Furthermore, in step S6, the similarity distance between all feature trajectories is calculated using the feature trajectories extracted in step S5 to obtain the final clustering result.

[0025] A storage device that stores instructions and data for implementing a ship trajectory clustering method based on AIS data.

[0026] A ship trajectory clustering device based on AIS data includes: a processor and a storage device; the processor loads and executes instructions and data in the storage device to implement a ship trajectory clustering method based on AIS data.

[0027] The beneficial effects of the technical solution provided by this invention are as follows: This invention extracts historical trajectory data from the AIS database, preprocesses the missing values, outliers, redundant values, and data formats, and segments the processed trajectories. It extracts the start and end points of each trajectory segment. Instead of using the DBSCAN algorithm to cluster ship trajectories within the research scope, this invention uses an improved DBSCAN algorithm to cluster the start and end points of the trajectories, resulting in trajectory clusters. This significantly reduces the space complexity of the algorithm and can effectively cluster large-scale data. Trajectories with the same start and end point clustering results are then grouped together based on the clustering results. For trajectories in the same category, the improved DTW algorithm is used to calculate the similarity distance between trajectories, further dividing the trajectory categories. Feature trajectories of each category are extracted, and the clustering results of the DBSCAN algorithm are corrected by calculating the similarity distance of the trajectories using the improved DTW algorithm, thus improving the accuracy of the clustering results. This results in a trajectory clustering method with fast clustering speed and low implementation complexity, which can effectively identify ship deviations from the navigation channel and can be applied to ship trajectory monitoring, maritime traffic planning, and other fields. Attached Figure Description

[0028] The present invention will be further described below with reference to the accompanying drawings and embodiments. In the accompanying drawings:

[0029] Figure 1 This is a flowchart of a ship trajectory clustering method based on AIS data in an embodiment of the present invention.

[0030] Figure 2This is a schematic diagram illustrating the principle of ship trajectory dwell point extraction in an embodiment of the present invention.

[0031] Figure 3 This is a schematic diagram illustrating the principle of ship navigation trajectory extraction in an embodiment of the present invention.

[0032] Figure 4 This is a schematic diagram of the DBSCAN algorithm in an embodiment of the present invention.

[0033] Figure 5 This is a schematic diagram illustrating the construction principle of the Ball-Tree for trajectory points in an embodiment of the present invention.

[0034] Figure 6 This is a schematic diagram illustrating the principle of the improved DTW distance calculation in this embodiment of the invention.

[0035] Figure 7 This is a schematic diagram of the improved DTW distance calculation principle in an embodiment of the present invention.

[0036] Figure 8 This is a schematic diagram illustrating the principle of feature trajectory extraction in an embodiment of the present invention.

[0037] Figure 9 This is a schematic diagram of the hardware device working in an embodiment of the present invention. Detailed Implementation

[0038] To provide a clearer understanding of the technical features, objectives, and effects of the present invention, specific embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0039] The embodiments of the present invention provide a method, device and storage device for ship trajectory clustering based on AIS data.

[0040] Please refer to Figure 1 , Figure 1 This is a flowchart of a ship trajectory clustering method based on AIS data according to an embodiment of the present invention, specifically including:

[0041] S1. After extracting historical ship trajectory data from the AIS database, preprocessing is performed on missing values, outliers, and redundant values ​​in the data. Missing values ​​are determined as follows: the ship's speed exceeds a speed threshold between two consecutive positioning intervals. Outliers are determined as follows: for three consecutive positioning points, if the speed between the second point and the first and third points exceeds the speed threshold, but the speeds of the first and third points are within the speed threshold, then the second point is considered an outlier. Redundancy is determined as follows: the location and time of two consecutive positioning operations are the same. The preprocessing includes correcting latitude and longitude errors, cleaning up duplicate points, and filtering drift points for missing values, outliers, and redundant values ​​in the data.

[0042] S2. Based on the ship's position and positioning time, traverse all trajectory points, segment the preprocessed trajectory data of each ship, find the trajectory dividing point, and extract the navigation trajectory. This includes the following two methods: (1) Calculate the maximum positioning time within the specified neighborhood of all positions. If both times exceed the time threshold, the ship is considered to be stationary at that position. The first point entering the neighborhood is marked as the stationary point, and the first point leaving the neighborhood is marked as the starting point of the next trajectory. Figure 2 As shown; (2) as Figure 3 As shown, the time difference between two adjacent positioning points is calculated. If it exceeds a time threshold, these two points are considered as trajectory segmentation points and are marked as the end point of the previous trajectory and the start point of the next trajectory, respectively. The ship stops at this position, and the trajectory is segmented based on the stopping point to extract the navigation trajectory.

[0043] S3. For the navigation trajectory data obtained in step S2, extract the start and end points of all trajectories, and then cluster the sets of start and end points using the improved DBSCAN algorithm. The specific process is as follows:

[0044] (1) Because the DBSCAN algorithm scans the neighborhood points of a trajectory point by traversing the distances from all other points, the time complexity of this method is high. Therefore, before executing the DBSCAN algorithm, a Ball-Tree is constructed for the trajectory start and end points to obtain the dataset D. Figure 4 As shown, some sample points are constructed using the Ball-Tree algorithm and displayed on a two-dimensional plane. Figure 4 In the diagram, the large circle contains the entire dataset, the medium circle divides the data in the large circle in half, and the small circle divides the data in each medium circle in half. The reason why a small circle is a single point is that at least two points are needed to form a circle. The medium circle only has three points, so dividing it into two parts can only result in 2+1. Therefore, two data points can form a circle, and a single data point is itself.

[0045] (2) Input dataset D, scan radius R and minimum number of points Pmin;

[0046] (3) Take a point P that has not been queried in the target dataset, and determine whether the number of points within the scanning radius of point P is greater than or equal to Pmin. If not, mark it as a noise point and repeat step (3). If yes, determine whether point P already belongs to a certain cluster C. If not, create a new cluster.

[0047] (4) Repeat step (3) for all unqueried points in cluster C until the last point is calculated, and then output the set of all clusters.

[0048] like Figure 5As shown, point P is selected as the first point in dataset D. The number of points contained within the scan radius R (within the gray circle) is calculated. If the number of points is greater than or equal to the minimum number of points Pmin (which is 3), it means that point P (the core point) belongs to a certain cluster C. Then, for other points in cluster C, such as point Q (density reachable), the above operation is repeated until the set of all points (density reachable) belonging to cluster C is found. For point Q2, the number of points within its scan radius R is less than Pmin, so Q2 is a noise point.

[0049] S4. For the trajectories obtained in step S2 and the clustering results of the start and end points obtained in step S3, if the start and end points of one or more trajectories are in the same cluster, they are grouped into one cluster. Then, for the trajectories in each cluster, the trajectory similarity distance between every two trajectories is calculated. The specific process for calculating the trajectory similarity distance is as follows:

[0050] (1) Assume that the sequences of the two trajectories are A={a1,a2,……,an} and B={b1,b2,……,b=m};

[0051] (2) The distance matrix Ei is calculated as follows: Ei = {d(ai,b1),d(ai,b2),……,d(ai,bm)}, i = {1,2,……,n}, where Ei is the i-th row of the distance matrix E, d is the distance between two points, n is the length of sequence A, and m is the length of sequence B.

[0052] (3) Based on the distance matrix E, accumulate the distances from the top left to the bottom right corner, minimizing the accumulated distance, and then divide by the accumulated number of steps, i.e., the average accumulated distance. This distance is used as the similarity distance between the two trajectory sequences. Since matching cannot skip any point in either sequence, a point in one sequence may match multiple points in the other, thus reducing the DTW distance between two low-similarity sequences. A cross-point matching method is proposed to avoid this situation, such as... Figure 6 As shown.

[0053] (4) Based on the similarity distance between all the obtained trajectories, set an appropriate threshold and determine whether it is necessary to further divide the trajectory clusters.

[0054] like Figure 7 As shown, for two ship trajectory sequences A and B, the distance from all points in sequence A to all points in sequence B is calculated. Then, all points in sequence A are traversed to find the shortest distance from all points in sequence B (dashed line in the figure). These distances are accumulated to obtain the average accumulated distance, which is the similarity distance between sequence A and sequence B.

[0055] S5. For the trajectory similarity distances calculated in step S4, find the trajectory whose sum of distances to all other trajectories is the feature trajectory. For example... Figure 8 As shown, by calculating the distances d(A1,A2), d(A1,A3), and d(A2,A3) between the three trajectory sequences, we can find that the total distance from sequence A2 to sequences A1 and A3 is the smallest. Therefore, sequence A2 is the feature trajectory.

[0056] S6. For the feature trajectories extracted in step S5, calculate the similarity between all feature trajectories, set an appropriate threshold, and obtain the final clustering result.

[0057] Please see Figure 9 , Figure 9 This is a schematic diagram of the hardware device in operation according to an embodiment of the present invention. The hardware device specifically includes: a ship trajectory clustering device 401 based on AIS data, a processor 402, and a storage device 403.

[0058] A ship trajectory clustering device 401 based on AIS data: used to implement the ship trajectory clustering method based on AIS data.

[0059] Processor 402: The processor 402 loads and executes the instructions and data in the storage device 403 to implement the ship trajectory clustering method based on AIS data.

[0060] Storage device 403: The storage device 403 stores instructions and data; the storage device 403 is used to implement the ship trajectory clustering method based on AIS data.

[0061] The beneficial effects of this invention are as follows: This invention extracts historical trajectory data from the AIS database, preprocesses it to remove missing values, outliers, redundant values, and changes in data format, and segments the processed trajectories. It extracts the start and end points of each trajectory segment. Instead of using the DBSCAN algorithm to cluster ship trajectories within the research scope, this invention uses an improved DBSCAN algorithm to cluster the start and end points of the trajectories, resulting in trajectory clusters. This significantly reduces the space complexity of the algorithm and allows for effective clustering of large-scale data. Trajectories with the same start and end point clustering results are then grouped together based on the clustering results. For trajectories in the same category, the improved DTW algorithm is used to calculate the similarity distance between trajectories, further classifying the trajectory categories. Feature trajectories of each category are extracted, and the DBSCAN algorithm's clustering results are corrected by calculating the similarity distance using the improved DTW algorithm, thus improving the accuracy of the clustering results. This results in a trajectory clustering method with fast clustering speed and low implementation complexity, which can effectively identify ship deviations from their course and can be applied to ship trajectory monitoring, maritime traffic planning, and other fields.

[0062] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method of clustering vessel trajectories based on AIS data, characterized in that: The method comprises the following steps: S1: extracting ship trajectory data from an AIS database and pre-processing abnormal data; S2: segmenting the trajectory data of each ship after pre-processing and extracting a sailing trajectory; S3: clustering the start and end points of the sailing trajectory by using an improved DBSCAN algorithm to obtain multiple clusters; In step S3, the specific process of obtaining multiple clusters is as follows: S31: extracting the start and end points of the ship sailing trajectory, constructing a Ball-Tree for the point set of the start and end points respectively and obtaining a data set D to improve the scanning speed of the boundary points in the core point neighborhood; S32: inputting the data set D, a scanning radius R and a minimum number of contained points Pmin; S33: taking an unqueried point P in the target data set, judging whether the number of points in the scanning radius of the point P is greater than or equal to Pmin, if not, marking the point as a noise point and repeating step S33; if yes, judging whether the point P belongs to a certain cluster C, if not, establishing a new cluster; S34: repeating step S33 for all unqueried points in the cluster C until the last point is calculated, and then outputting a set of all clusters; S4: calculating the similarity distance of the trajectory in each cluster by using an improved DTW algorithm, and dividing the trajectories with large similarity distance into different clusters; S5: extracting a feature trajectory from each cluster; S6: calculating the similarity distance of the feature trajectory by using an improved DTW algorithm and obtaining a final set of clusters.

2. The AIS data based ship trajectory clustering method of claim 1, wherein: In step S1, the abnormal data includes missing values, abnormal values and redundancies in the data, and the pre-processing includes longitude and latitude error correction, repeated point cleaning and drift point filtering.

3. The AIS data based ship trajectory clustering method of claim 1, wherein: In step S2, the process of the segmentation is as follows: according to the position and positioning time of the ship, the maximum positioning time in the distance neighborhood of all positions is calculated, if the time exceeds a time threshold, it is considered that the ship stays at the position, the trajectory is segmented according to the stay point, and the sailing trajectory is extracted.

4. The AIS data based ship trajectory clustering method of claim 1, wherein: In step S4, the specific process of calculating the similarity distance by using the improved DTW algorithm is as follows: S41: assuming that the sequences of two trajectories are A={a1, a2, …, an} and B={b1, b2, …, bm}; S42: calculating a distance matrix Ei={d(ai, b1), d(ai, b2), …, d(ai, bm)}, i={1, 2, …, n}, wherein Ei is the ith row of the distance matrix E, d is the distance between two points, n is the length of the sequence A, and m is the length of the sequence B; S43: according to the distance matrix E, the accumulated distance is accumulated from the upper left corner to the lower right corner, and the accumulated distance is minimized, and then the accumulated step number is divided, that is, the average accumulated distance is taken as the similarity distance between the two trajectory sequences; S44: according to the similarity distance between all trajectory sequences, a suitable threshold is set and it is judged whether the trajectory cluster needs to be further divided.

5. The AIS data based ship trajectory clustering method of claim 1, wherein: In step S5, for the trajectory similarity distance calculated in step S4, one of the trajectories with the smallest distance sum with other trajectories is found, which is the feature trajectory.

6. The AIS data based ship trajectory clustering method of claim 1, wherein: In step S6, the similarity distance between all feature trajectories is calculated using the feature trajectories extracted in step S5, and a final clustering result is obtained.

7. A storage device, characterized by: The storage device stores instructions and data for implementing the AIS data-based ship trajectory clustering method of any one of claims 1-6.

8. An AIS data based ship trajectory clustering device, characterized by: Comprise: A processor and a storage device; the processor loads and executes instructions and data in the storage device for implementing the AIS data-based ship trajectory clustering method of any one of claims 1-6.

Images