Ship trajectory clustering method

Abstract

The invention discloses a ship trajectory clustering method, which comprises the following steps of: cleaning AIS data to obtain a plurality of navigation trajectories of a ship in a set time period; the Hausdorff distance between any two navigation tracks is calculated to serve as the calculation basis of a DBSCAN algorithm; a core point prediction model is constructed, high-probability data points in the navigation trajectory are screened out to serve as candidate core points, and neighborhood query and clustering expansion are performed on the candidate core points through a DBSCAN algorithm; a Bayesian optimization algorithm is improved through an adaptive acquisition function, and hyper-parameters of a DBSCAN algorithm are optimized based on the improved Bayesian optimization algorithm. The DBSCAN algorithm has the beneficial effects that the DBSCAN algorithm is improved aiming at the problem of relatively low operation efficiency of the density clustering algorithm under the condition of large data volume, and the operation efficiency of the algorithm is improved while the calculation precision is ensured. The Bayesian optimization algorithm is introduced to automatically optimize hyper-parameters of the DBSCAN algorithm, and the effect and stability of clustering analysis are improved.

Description

Technical Field

[0001] The present application relates to the technical field of ship trajectory analysis, and in particular to a ship trajectory clustering method. Background Art

[0002] With the development of the shipping economy and the widespread use of the Automatic Identification System (AIS), a large amount of ship movement trajectory data has been generated, which is of great significance to the analysis and supervision of global shipping. By mining ship AIS data, valuable information can be provided for analyzing ship movement patterns and behavioral characteristics, as well as track prediction, and cluster analysis plays a key role in this.

[0003] Ship trajectories have an uncertain density distribution, and density methods can be used to cluster ship trajectories. The DBSCAN algorithm is a widely used algorithm for clustering ship AIS trajectories. Its core is to define a neighborhood of ship trajectories and a threshold for limiting the density of trajectory distribution. It then examines the density of trajectories within the neighborhood to achieve clustering of ship trajectories. However, this type of algorithm suffers from low computational efficiency when dealing with large amounts of data and is sensitive to two hyperparameters: the minimum neighborhood radius (Eps) and the minimum number of points in the neighborhood (MinPts). Therefore, traditional DBSCAN methods for clustering ship trajectories often struggle to achieve good and stable recognition results. Summary of the Invention

[0004] One of the objectives of the present application is to provide a ship trajectory clustering method that can solve at least one of the defects in the above-mentioned background technology.

[0005] To achieve at least one of the above purposes, the technical solution adopted in this application is: a ship trajectory clustering method, comprising the following steps: S100: Clean the AIS data and group them according to MMSI, arrange the track points in each group according to the collection time, and obtain the navigation tracks of multiple ships within the set time period; S200: Calculate the Hausdorff distance between any two navigation trajectories to obtain a similarity measurement matrix of all navigation trajectories, and use it as a calculation basis for the DBSCAN algorithm; S300: Build a core point prediction model and select high-probability data points in the navigation trajectory as candidate core points. Use the DBSCAN algorithm to perform neighborhood queries on the candidate core points and perform cluster expansion. S400: improving the Bayesian optimization algorithm through an adaptive acquisition function, and optimizing the hyperparameters of the DBSCAN algorithm based on the improved Bayesian optimization algorithm, so that clustering results are output through the optimized DBSCAN algorithm.

[0006] Preferably, in step S300, the construction of the core point prediction model includes the following processes: clustering a small amount of representative data through the DBSCAN algorithm to generate core point training data; constructing the core point prediction model through a lightweight model algorithm, and training it with the core point training data; extracting the local statistical features of each data point in the navigation trajectory as the input of the core point prediction model, predicting the probability of each data point as a core point, and taking the data points whose predicted probability exceeds the set threshold as candidate core points.

[0007] Preferably, the local statistical characteristics of each data point in the navigation trajectory include the mean of the k-nearest neighbor distance, the local grid density, the distance distribution variance, and the relative density ratio; when predicting the candidate core points, it is suitable to calculate the prediction confidence for the data point that has completed the prediction; based on the calculated prediction confidence distribution, the set threshold of the probability required for judging the candidate core points is adaptively adjusted; wherein, the specific value of the set threshold for judging the candidate core points is 0.7~0.9.

[0008] Preferably, during the cluster expansion process, if the newly added neighboring point is marked as a low-probability core point in the core point prediction model, a neighborhood query is still performed on the neighboring point; based on the neighborhood query result, if the neighboring point is a true core point, the neighboring point and its neighborhood statistical features are used as new training samples for the core point prediction model.

[0009] Preferably, a sliding window mechanism is set for the core point prediction model so that the core point prediction model is retrained at every set time interval or data volume.

[0010] Preferably, the process of improving the Bayesian optimization algorithm in step S400 is: using a Gaussian process function as a proxy function and giving a prior distribution, and then using an adaptive acquisition function to perform multiple iterations of the objective function until the maximum number of iterations is reached; the process of each iteration is: according to the dynamic weight mechanism of the adaptive acquisition function, the search intensity of the updated data points in the search space is automatically adjusted during the iteration process; the new data points are evaluated in the objective function and the corresponding results are obtained, and the proxy function is updated according to the new data points and the corresponding results to obtain a posterior distribution, which is used as the prior distribution of the next iterative process.

[0011] Preferably, the adaptive acquisition function α t The expression of (x) is: ; ; ; ; Where UCB(x) represents the upper confidence limit function, EI(x) represents the expected improvement function, η(t) represents the Sigmoid decay function, t represents the current number of iterations, T represents the total number of iterations, μ(x) represents the mean of the objective function predicted by the Gaussian process, σ(x) represents the standard deviation of the Gaussian process prediction, and β represents the exploration coefficient; x * represents the currently known optimal solution, and f(x) represents the objective function.

[0012] Preferably, the quality of the clustering results of the ship's navigation trajectory is represented by the silhouette coefficient; in step S400, the silhouette coefficient of the navigation trajectory is used as the objective function of the improved Bayesian optimization algorithm; the hyperparameters to be optimized of the DBSCAN algorithm are adjusted within the corresponding parameter variation range, and clustering is performed once during each adjustment process; the clustering results are substituted into the improved Bayesian optimization algorithm for iteration and the corresponding silhouette coefficient is output; the hyperparameter corresponding to the maximum value of the silhouette coefficient returned by the improved Bayesian optimization algorithm is used as the final parameter of the DBSCAN algorithm.

[0013] Preferably, the expression of the silhouette coefficient S used to measure the quality of the clustering result of the navigation trajectory is: ; ; Among them, N is the total number of data points in the cluster, s(i) is the silhouette coefficient of a single data point, a(i) is the intra-cluster closeness, that is, the average distance from data point i to other points in the same cluster, and b(i) is the inter-cluster separation, that is, the average distance from data point i to all points in the nearest other clusters.

[0014] Preferably, when performing clustering using the DBSCAN algorithm, it is suitable to classify some outlier trajectory segments as noise data; for the noise data, it is suitable to repeat the clustering analysis using the DBSCAN algorithm at least once; and the noise data obtained again is identified as a true abnormal driving trajectory and recorded as data for subsequent abnormal trajectory analysis.

[0015] Compared with the prior art, the present invention has the following advantages: (1) In order to solve the problem that the density clustering algorithm has low computational efficiency when the data volume is large, the DBSCAN algorithm is improved to improve the algorithm's operating efficiency while ensuring the computational accuracy.

[0016] (2) The Bayesian optimization algorithm is introduced to automatically optimize the two hyperparameters that need to be manually set in the DBSCAN algorithm, thereby improving the effect and stability of clustering analysis. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 This is a schematic diagram of the overall workflow of this application. DETAILED DESCRIPTION

[0018] Below, the present application is further described in conjunction with specific implementation methods. It should be noted that, in the description of this specification, the reference terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" and the like are intended to mean that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms should not be understood as necessarily referring to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification.

[0019] In the description of this application, it should be noted that for directional words, such as the terms "center", "horizontal", "longitudinal", "length", "width", "thickness", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", "clockwise", "counterclockwise" and so on, indicating the orientation and position relationship are based on the orientation or position relationship shown in the accompanying drawings, which is only for the convenience of describing this application and simplifying the description, and does not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and cannot be understood as limiting the specific scope of protection of this application.

[0020] It should be noted that the terms "first", "second", etc. in the description and claims of this application are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0021] In this application, unless otherwise expressly specified or limited, terms such as "mounted," "connected," "connect," and "fixed" should be understood in a broad sense. For example, they may refer to connection, detachable connection, or integration; mechanical connection or electrical connection; direct connection or indirect connection through an intermediate medium; and internal communication between two components or interaction between two components. Those skilled in the art will understand the specific meanings of the above terms in this application based on specific circumstances.

[0022] In this application, unless otherwise expressly specified or limited, a first feature being "above" or "below" a second feature may include the first and second features being in direct contact, or may include the first and second features being in contact not directly but through another feature between them. Moreover, a first feature being "above," "above," and "above" a second feature may include the first feature being directly above or obliquely above the second feature, or may simply mean that the first feature is higher in level than the second feature. A first feature being "below," "below," and "below" a second feature may include the first feature being directly below or obliquely below the second feature, or may simply mean that the first feature is lower in level than the second feature.

[0023] The terms "comprises" and "having" and any variations thereof in the specification and claims of this application are intended to cover non-exclusive inclusions. For example, a process, method, system, product or apparatus that includes a series of steps or units is not necessarily limited to those steps or units expressly listed, but may include other steps or units not expressly listed or inherent to such process, method, product or apparatus.

[0024] One of the preferred embodiments of this application is as follows: Figure 1 As shown, a ship trajectory clustering method includes the following steps: S100: Clean the AIS data and group them according to MMSI, arrange the track points in each group according to the collection time, and obtain the navigation tracks of multiple ships within a set time period.

[0025] S200: Calculate the Hausdorff distance between any two navigation trajectories to obtain a similarity measurement matrix of all navigation trajectories, and use it as a calculation basis for the DBSCAN algorithm.

[0026] S300: Build a core point prediction model and select high-probability data points in the navigation trajectory as candidate core points. Use the DBSCAN algorithm to perform neighborhood query and cluster expansion on the candidate core points.

[0027] S400: improving the Bayesian optimization algorithm through an adaptive acquisition function, and optimizing the hyperparameters of the DBSCAN algorithm based on the improved Bayesian optimization algorithm, so that clustering results are output through the optimized DBSCAN algorithm.

[0028] It is understandable that by cleaning AIS data, some data on trajectories that completely fail to meet the requirements can be eliminated, thereby reducing the computational effort of the subsequent clustering algorithm and improving clustering speed. The Hausdorff distance is introduced to measure the similarity between two trajectories, and its calculation results are more robust than other methods. To address the low computational efficiency of density clustering algorithms when dealing with large amounts of data, this embodiment improves the DBSCAN algorithm by constructing a core point prediction model using a small amount of data. The constructed core point prediction model is then used to determine the core points of the data points and cluster the data identified as core points. This effectively reduces the computational data volume of the clustering algorithm, thereby improving the algorithm's operational efficiency while ensuring computational accuracy. The Bayesian Optimization algorithm is introduced to automatically optimize two hyperparameters that require manual setting in the DBSCAN algorithm: the minimum neighborhood radius (Eps) and the minimum number of points in the neighborhood (MinPts), thereby improving the effectiveness and stability of clustering analysis.

[0029] In this embodiment, the cleaning of AIS data in step S100 mainly includes two parts: deletion of abnormal trajectories and extraction of trajectory data. For ease of understanding, these two parts will be described in detail below.

[0030] 1. Deletion of abnormal trajectories.

[0031] Since ship AIS signals may be interrupted or abnormal during the transmission, transmission and reception process, the original AIS data generally contains data anomalies and redundancy. Therefore, some measures need to be taken to delete the obviously abnormal trajectory data in the original AIS data to obtain valid trajectory data.

[0032] Specifically, data with a non-9-digit MMSI is deleted. As you can see, since a ship's MMSI is a 9-digit unique identifier, data that doesn't meet this requirement is deleted. Data with a normal navigation status is also selected. Track data with a duration less than a set first threshold is also deleted.

[0033] It should be noted that when the duration of a track is too short, its spatial characteristics are not obvious and will become a interference item in the subsequent clustering algorithm, so it should be deleted. The specific value of the first threshold can be selected according to the actual needs of technical personnel in the field, for example, it can be 20 minutes or half an hour. At the same time, tracks with a maximum speed over the ground less than a set second threshold are deleted. It should be noted that the specific value of the second threshold can be determined according to the actual needs of technical personnel in the field. For example, the second threshold can be set at 5 knots. If the maximum speed is less than 5 knots, it can be considered that the ship is not in a state of motion.

[0034] 2. Extract trajectory data after deleting abnormal trajectories.

[0035] By dividing all trajectory data into multiple groups based on MMSI and arranging the track points in each group in ascending order of acquisition time, we can obtain a trajectory dataset for ships represented by the same MMSI over a period of time. When the time difference or spacing between two adjacent track points of the same ship exceeds a set third threshold, the two adjacent track points are considered the endpoints and starting points of different tracks, respectively, thus dividing the ship's trajectory into multiple sub-tracks.

[0036] It should be noted that the specific value of the third threshold can be selected based on the actual needs of those skilled in the art. For example, when judging based on the time difference between two adjacent trajectory points, the third threshold can be half an hour; when judging based on the distance between two adjacent trajectory points, the third threshold can be 120 meters. It should be noted that after obtaining multiple sub-trajectories, sub-trajectories with a duration less than the first threshold are deleted to avoid interfering with the subsequent clustering process.

[0037] After AIS data cleaning is completed, the trajectories are grouped according to the MMSI. When the time difference or spacing between two adjacent trajectory points corresponding to the same ship is greater than the set third threshold, the two adjacent trajectory points are regarded as the end and starting points of different trajectories, respectively, and the navigation trajectory of a ship is divided into multiple sub-trajectories.

[0038] In this embodiment, the ship's trajectory is a set of coordinate points arranged in chronological order, and the length of the set varies. Therefore, the Hausdorff distance is used to measure the similarity between any two trajectories to form the required distance matrix, i.e., the trajectory similarity matrix. Specifically, when calculating the Hausdorff distance, the entire trajectory can be treated as a high-dimensional point. The Hausdorff distance converts the distance between trajectories into simple numbers. The DBSCAN algorithm only needs to know the distance between points to achieve clustering. Therefore, the distance matrix calculated using the Hausdorff distance can be directly used as the distance input for the DBSCAN algorithm. More simply, the Hausdorff distance preserves the shape information of the entire trajectory while providing a single-valued distance. Therefore, it can directly transform the trajectory point set corresponding to the trajectory into a "point-distance" problem that the DBSCAN algorithm can handle, making it a natural calculation basis for the DBSCAN algorithm. For ease of understanding, the specific calculation process of the Hausdorff distance is described in detail below.

[0039] Specifically, the Hausdorff distance is the maximum distance from one set to the nearest point in another set. Generally, the one-way Hausdorff distances between two sets are not equal. Therefore, the maximum value of the two-way Hausdorff distances can be taken as the Hausdorff distance of any two navigation trajectories in this embodiment.

[0040] Two trajectories T can be given r1 =(p1, p2, ..., p n ) and T r2 =(q1,q2,……,q m ); where p and q represent the trajectory points corresponding to the two trajectories, and n and m represent the lengths of the two trajectories, i.e. the total number of trajectory points. Then the Hausdorff distance H(T r1 , T r2 ) is calculated as follows: .

[0041] .

[0042] .

[0043] Among them, h(T r1 , T r2 ) and h(T r2 , T r1 ) represent two one-way Hausdorff distances respectively.

[0044] It should be noted that for N ship navigation trajectory samples, the Hausdorff distance between each of the trajectories is calculated to obtain the Hausdorff distance matrix corresponding to all samples, that is, the similarity measurement matrix between different trajectory data within the sample; this matrix data can be directly used as the calculation basis for the subsequent DBSCAN algorithm cluster analysis.

[0045] In this embodiment, the construction of the core point prediction model in step S300 includes the following steps: A small amount of representative data is clustered using the DBSCAN algorithm to generate core point training data. The core point prediction model is constructed using a lightweight model algorithm and trained using the core point training data. Local statistical features of each data point in the navigation trajectory are extracted as input to the core point prediction model. The probability of each data point being a core point is predicted, and data points whose predicted probability exceeds a set threshold are selected as candidate core points.

[0046] It's understandable that, according to the principles of the DBSCAN algorithm, the clustering process requires querying the neighborhood of each sample to ensure that every sample point is labeled. However, in reality, the neighborhoods of many nodes overlap. When the dataset is very large, this leads to repeated queries of many nodes, reducing the algorithm's efficiency. It is precisely this repeated query issue that increases the time overhead of the traditional DBSCAN algorithm. In this embodiment, however, the core point prediction model constructed using a lightweight model algorithm can preemptively filter out samples that are clearly not core points. This allows the subsequent DBSCAN algorithm to perform precise neighborhood queries only on high-probability candidate core points, significantly reducing the number of repeated node queries and thus improving the algorithm's efficiency.

[0047] It should be known that there are many specific types of lightweight model algorithms, such as random forest algorithm, decision tree algorithm, logistic regression algorithm, etc., which can be selected according to the actual needs of technical personnel in this field. The specific principles and working processes of the algorithms are well-known technologies to technical personnel in this field, so they will not be elaborated here.

[0048] It should also be noted that the local statistical characteristics of each data point in the navigation trajectory include the mean k-nearest neighbor distance, local grid density, distance distribution variance, and relative density ratio. The k-nearest neighbor distance mean represents the average value of the distances calculated from point x to its k neighbors and is used to reflect local density. The local grid density is the sum of the number of points in the grid where point x is located and the number of points in its neighboring grids, which is calculated by dividing the data into coarse-grained grids. The distance distribution variance represents the variance of the distances from point x to its k neighbors and is used to capture the uniformity of the local data distribution. The relative density ratio represents the ratio of the density of the grid where point x is located to the overall density.

[0049] It is understandable that the specific value of the set threshold for determining candidate core points can be selected according to the actual needs of technical personnel in this field. For example, the value can be 0.7~0.9, that is, the data point is marked as a candidate core point only when the probability of the data point being a core point exceeds 0.7~0.9.

[0050] It should be noted that when predicting candidate core points based on the core point prediction module, there is a possibility of overfitting, that is, high-probability candidate core points are actually noise, which can lead to invalid neighborhood queries in the subsequent DBSCAN algorithm. Therefore, when predicting candidate core points, the prediction confidence can be calculated for the currently predicted data point. Then, based on the calculated prediction confidence distribution, the set threshold for the probability required for candidate core point judgment can be adaptively adjusted. For example, if the prediction confidence is low, the set threshold for the probability required for candidate core point judgment can be increased from 0.7 to 0.8. The specific calculation process of the prediction confidence is common knowledge to those skilled in the art and will not be elaborated here. That is, the confidence calculation is performed based on the probability of whether the candidate core point predicted by the core point prediction model is a true core point to obtain the required prediction confidence.

[0051] Those skilled in the art will appreciate that the DBSCAN algorithm's trajectory point clustering process primarily starts with a candidate core point, connecting all density-reachable points into a cluster, with the remaining data points considered noise. Specifically, a neighborhood radius ε and a minimum number of neighbors, MinPts, are set. When a candidate core point has more than MinPts neighbors within the neighborhood radius ε, it is considered a true core point. Its neighbors are then continuously incorporated into clusters, ultimately outputting a cluster set and a noise set. Based on the aforementioned clustering process, the neighborhood radius ε and the minimum number of neighbors, MinPts, are the two most important hyperparameters of the DBSCAN algorithm. Traditional methods require manual configuration, which is incapable of adapting to algorithm changes, resulting in low clustering accuracy. This embodiment will optimize these two hyperparameters using an improved Bayesian optimization algorithm in subsequent solutions, thereby improving DBSCAN's clustering accuracy.

[0052] In this embodiment, after obtaining candidate core points, a neighborhood query and cluster expansion are initiated; the candidate core point list can be dynamically updated during the cluster expansion process. Specifically, if a newly added neighboring point is a low-probability core point marked in the core point prediction model, a neighborhood query is still performed on that neighboring point, thereby avoiding misjudgments by the core point prediction model. Based on the neighborhood query results, if the neighboring point is a true core point, the neighboring point and its neighborhood statistical features are used as new training samples for the core point prediction model. Based on these new training samples, a mini-batch gradient descent algorithm is used to dynamically update the parameters of the core point prediction model, allowing the core point prediction model to gradually adapt to changes in the data distribution and thereby improve the subsequent prediction accuracy of the core point prediction model.

[0053] It should be noted that the navigation trajectory data of this application is streaming data. The core point prediction model may experience data drift when predicting the core points of streaming data, that is, the data distribution changes over time, which will cause the core point prediction model to fail. Therefore, it is necessary to monitor the prediction error rate of the core point prediction model for streaming data, and retrain the model when the error rate reaches a set value. The amount of computation required to directly calculate the prediction error rate is large. This embodiment replaces the calculation of the prediction error rate by setting a sliding window mechanism, that is, the core point prediction model is retrained at every set time or data volume.

[0054] It is understandable that, as can be seen from the above content, the DBSCAN algorithm is particularly sensitive to two hyperparameters. The traditional method requires a large number of manual parameter adjustments to achieve a relatively ideal clustering effect. However, this method is time-consuming and labor-intensive, lacks stability, and does not necessarily achieve the optimal clustering results. Therefore, the technical solution of this application introduces a Bayesian optimization algorithm to achieve adaptive adjustment of the hyperparameters of the DBSCAN algorithm, thereby greatly improving the clustering analysis effect and computational efficiency of the DBSCAN algorithm.

[0055] The Bayesian optimization algorithm uses probabilistic models based on Bayes' theorem to model and infer the objective function. Specifically, it constructs a surrogate model (usually a Gaussian process) to fit an existing function to evaluate points. This surrogate model captures the uncertainty of the objective function. Based on this surrogate model, the Bayesian optimization algorithm uses acquisition functions (probability improvement (PI), expected improvement (EI), and upper confidence limit (UCB)) to determine the next point to evaluate, thereby finding the global optimal solution within a small number of evaluations.

[0056] Traditional acquisition functions each have their own advantages and disadvantages, which leads to a trade-off problem. Among them, PI is intuitive in concept, simple to calculate, and can converge quickly, but it may converge to a local optimal solution too early because it is sensitive to the probability of improvement and does not pay much attention to the size of the improvement. EI considers not only the probability of improvement, but also the magnitude of improvement. Therefore, it is more robust than PI and can better balance exploration and utilization. However, it has the problem of complex calculation and may fall into suboptimal solutions in multi-peak functions. The UCB algorithm is simple and easy to understand, and can effectively balance exploration and utilization, but it may perform poorly for objective functions with large noise. Therefore, in the technical solution of the present application, the traditional acquisition function can be improved to obtain an adaptive acquisition function. The adaptive acquisition function can be dynamically adjusted according to the operation process of the Bayesian optimization algorithm, so that the Bayesian algorithm has the best iterative effect on the objective function at different operation stages.

[0057] In this embodiment, the Bayesian optimization algorithm is improved in step S400 by using a Gaussian process function as a proxy function and assigning a prior distribution. The adaptive acquisition function is then used to iterate the objective function multiple times until the maximum number of iterations is reached. Each iteration automatically adjusts the search intensity for updated data points within the search space during the iteration based on the dynamic weighting mechanism of the adaptive acquisition function. New data points are evaluated in the objective function and the corresponding results are obtained. The proxy function is then updated based on the new data points and the corresponding results, resulting in a posterior distribution, which serves as the prior distribution for the next iteration.

[0058] It can be understood that the expression of the proxy function F(x) is: .

[0059] ; .

[0060] Where m(x) represents the mean function, K(x, x´) represents the covariance function, and x and x´ represent any two input points, that is, any two hyperparameter combinations (ε, MinPts) with different values.

[0061] Adaptive acquisition function α t The expression of (x) is: .

[0062] ; ; .

[0063] Where UCB(x) represents the upper confidence limit function, EI(x) represents the expected improvement function, η(t) represents the Sigmoid decay function, t represents the current number of iterations, T represents the total number of iterations, μ(x) represents the mean of the objective function predicted by the Gaussian process, σ(x) represents the standard deviation of the Gaussian process prediction, and β represents the exploration coefficient; x * represents the currently known optimal solution, and f(x) represents the objective function.

[0064] In this embodiment, the quality of the clustering results of ship navigation trajectories is represented by the silhouette coefficient. The silhouette coefficient quantifies the distance relationship between the sample and the same cluster and other clusters, reflecting the closeness and separation of the clustering results. In step S400, the silhouette coefficient of the navigation trajectory is used as the objective function of the improved Bayesian optimization algorithm. The hyperparameters to be optimized of the DBSCAN algorithm are adjusted within the corresponding parameter variation range, and clustering is performed once during each adjustment process. The clustering results are substituted into the improved Bayesian optimization algorithm, iterated, and the corresponding silhouette coefficient is output. The hyperparameter corresponding to the maximum silhouette coefficient returned by the improved Bayesian optimization algorithm is used as the final parameter of the DBSCAN algorithm.

[0065] It can be understood that the silhouette coefficient is a parameter with an upper limit of 1. The closer it is to 1, the more compact the clusters and the greater the separation between clusters. The process of tuning DBSCAN algorithm hyperparameters through the Bayesian optimization algorithm can be simply understood as: using a Bayesian optimization algorithm surrogate model to select the most promising parameter values ​​within the parameter variation range for clustering testing. Based on the clustering test results of the DBSCAN algorithm, the Bayesian optimization algorithm can calculate the silhouette coefficient. The parameter value used in the clustering test corresponding to the maximum silhouette coefficient is selected as the optimal hyperparameter and substituted into the DBSCAN algorithm.

[0066] Specifically, the expression of the silhouette coefficient S for measuring the quality of the clustering result of the navigation trajectory is: ; .

[0067] Among them, N is the total number of data points in the cluster, s(i) is the silhouette coefficient of a single data point, a(i) is the intra-cluster closeness, that is, the average distance from data point i to other points in the same cluster, and b(i) is the inter-cluster separation, that is, the average distance from data point i to all points in the nearest other clusters.

[0068] In this embodiment, when performing the above-mentioned cluster analysis, some outlier trajectory segments will appear, that is, this part of the data does not belong to any category, and the program will automatically classify these outlier trajectory data as noise data. Noise data is data containing a large number of abnormal driving trajectories, but at the same time, because the adjustment of hyperparameters is based on a global perspective, it is inevitable that some normal trajectory data will be classified as noise data. Therefore, the DBSCAN algorithm can be used to repeat the cluster analysis of noise data at least once; the noise data obtained again is identified as the real abnormal driving trajectory and recorded. Compared with the traditional method, this application proposes for the first time a method of repeated cluster analysis of noise data in the clustering results, which can effectively extract the real abnormal driving trajectory data and provide data support for subsequent abnormal trajectory analysis.

[0069] The above describes the basic principles, main features, and advantages of the present application. Those skilled in the art should understand that the present application is not limited to the above-described embodiments. The above-described embodiments and the specification merely illustrate the principles of the present application. Various changes and improvements may be made to the present application without departing from the spirit and scope of the present application. These changes and improvements fall within the scope of the present application for which protection is sought. The scope of protection claimed by the present application is defined by the appended claims and their equivalents.

Claims

1. A ship trajectory clustering method, characterized in that: The steps include: S100: Clean the AIS data and group them according to MMSI, arrange the track points in each group according to the collection time, and obtain the navigation tracks of multiple ships within the set time period; S200: Calculate the Hausdorff distance between any two navigation trajectories to obtain a similarity measurement matrix of all navigation trajectories, and use it as a calculation basis for the DBSCAN algorithm; S300: Build a core point prediction model and select high-probability data points in the navigation trajectory as candidate core points. Use the DBSCAN algorithm to perform neighborhood queries on the candidate core points and perform cluster expansion. S400: improving the Bayesian optimization algorithm through an adaptive acquisition function, and optimizing the hyperparameters of the DBSCAN algorithm based on the improved Bayesian optimization algorithm, so that clustering results are output through the optimized DBSCAN algorithm.

2. The ship trajectory clustering method according to claim 1, characterized in that: In step S300, the construction of the core point prediction model includes the following process: Cluster a small amount of representative data using the DBSCAN algorithm to generate core point training data; Build a core point prediction model using a lightweight model algorithm and train it using core point training data; The local statistical features of each data point in the navigation trajectory are extracted as the input of the core point prediction model, and the probability of each data point being a core point is predicted. The data points whose predicted probability exceeds the set threshold are selected as candidate core points.

3. The ship trajectory clustering method according to claim 2, characterized in that: The local statistical characteristics of each data point in the navigation trajectory include the mean of k-nearest neighbor distance, local grid density, distance distribution variance and relative density ratio; When predicting candidate core points, it is suitable to calculate the prediction confidence of the currently predicted data point; Adaptively adjust the threshold value of the probability required for judging candidate core points based on the calculated prediction confidence distribution; The specific value of the threshold used to determine the candidate core points is 0.7~0.

9.

4. The ship trajectory clustering method according to claim 2, characterized in that: During the cluster expansion process, if the newly added neighboring point is marked as a low-probability core point in the core point prediction model, a neighborhood query is still performed on the neighboring point; Based on the neighborhood query results, if the neighboring point is a true core point, the neighboring point and its neighborhood statistical features are used as new training samples for the core point prediction model.

5. The ship trajectory clustering method according to claim 2, characterized in that: A sliding window mechanism is set for the core point prediction model so that the core point prediction model is retrained at every set time or data volume.

6. The ship trajectory clustering method according to any one of claims 1 to 5, characterized in that: The process of improving the Bayesian optimization algorithm in step S400 is as follows: The Gaussian process function is used as the proxy function and a prior distribution is given. Then, the adaptive acquisition function is used to iterate the objective function multiple times until the maximum number of iterations is reached. The process of each iteration is: According to the dynamic weight mechanism of the adaptive acquisition function, the search intensity of the updated data points in the search space is automatically adjusted during the iterative process; Evaluate the new data point in the objective function and obtain the corresponding result. Update the proxy function according to the new data point and the corresponding result to obtain a posterior distribution, which is used as the prior distribution of the next iterative process.

7. The ship trajectory clustering method according to claim 6, characterized in that: Adaptive acquisition function α t The expression of (x) is: ； ； ； ； Where UCB(x) represents the upper confidence limit function, EI(x) represents the expected improvement function, η(t) represents the Sigmoid decay function, t represents the current number of iterations, T represents the total number of iterations, μ(x) represents the mean of the objective function predicted by the Gaussian process, σ(x) represents the standard deviation of the Gaussian process prediction, and β represents the exploration coefficient; x * represents the currently known optimal solution, and f(x) represents the objective function.

8. The ship trajectory clustering method according to claim 6, characterized in that: The continuity and uniformity of the ship's trajectory are expressed by the silhouette coefficient; In step S400, the contour coefficient of the navigation trajectory is used as the objective function of the improved Bayesian optimization algorithm; The hyperparameters to be optimized of the DBSCAN algorithm are adjusted within the corresponding parameter variation range, and clustering is performed once during each adjustment process; Substitute the clustering results into the improved Bayesian optimization algorithm for iteration and output the corresponding silhouette coefficient; The hyperparameter corresponding to the maximum value of the silhouette coefficient returned by the improved Bayesian optimization algorithm is used as the final parameter of the DBSCAN algorithm.

9. The ship trajectory clustering method according to claim 8, characterized in that: The expression of the silhouette coefficient S used to measure the continuity and uniformity of the navigation trajectory is: ； ； Among them, N is the total number of data points in the cluster, s(i) is the silhouette coefficient of a single data point, a(i) is the intra-cluster closeness, that is, the average distance from data point i to other points in the same cluster, and b(i) is the inter-cluster separation, that is, the average distance from data point i to all points in the nearest other clusters.

10. The ship trajectory clustering method according to claim 1, wherein: When performing clustering with the DBSCAN algorithm, it is suitable to classify some outlier trajectory segments as noise data; For noisy data, it is appropriate to repeat the cluster analysis at least once using the DBSCAN algorithm; The noise data obtained again is identified as the real abnormal driving trajectory and recorded as data for subsequent abnormal trajectory analysis.

Images