Inland ship AIS trajectory similarity query method and system based on locality sensitive hashing

Abstract

The invention relates to an inland ship AIS trajectory similarity query method and system based on locality sensitive hashing, and belongs to the technical field of ship space-time trajectory data processing. According to the method, character dimension reduction processing is firstly carried out on spatio-temporal trajectory data, the trajectory data is converted into a character sequence, and the calculation complexity is effectively reduced; then, a MinHash signature and an LSH technology are utilized to classify and retrieve the character sequence, and historical trajectory data similar to the query trajectory is quickly found. And finally, extracting a target track section meeting a similarity threshold value through an LCSS and a sliding window algorithm, and realizing motion change similarity query and spatial position similarity query. The method has the advantages of high calculation efficiency, flexible functions, high applicability and the like, and provides effective support for applications such as inland ship navigation behavior analysis, navigation safety monitoring, channel management optimization and the like.

Description

Technical Field

[0001] The invention belongs to the technical field of ship spatiotemporal trajectory data processing, and relates to an inland river ship AIS trajectory similarity query method and system based on local sensitive hashing. Background Art

[0002] The Automatic Identification System (AIS) is a widely used digital navigation aid system and equipment. Ship dynamic information such as ship position, speed, rate of change of heading and heading, as well as ship static information such as ship name, call sign, draft, dangerous goods, etc. can be broadcast and communicated between AIS stations and AIS base stations via VHF radio waves to ensure maritime traffic safety. At present, there are hundreds of thousands of ships installed with AIS in the world, and the global monthly AIS data volume can reach hundreds of GB. Among them, each AIS dynamic message can be regarded as a time and space trajectory point of the ship. Therefore, these accumulated AIS data contain a large amount of historical ship trajectory data. Through the analysis and mining of historical ship trajectory data, it can provide good solutions for route optimization, abnormal behavior monitoring, false target identification, port throughput statistical analysis, etc.

[0003] Inland AIS spatiotemporal trajectory similarity query is a data analysis technology that can identify navigation trajectories with similar motion patterns or behavioral characteristics from a spatiotemporal trajectory database. This technology uses a predefined similarity metric (similarity criterion) to judge the similarity of two trajectories and find trajectories from the data set that are close to a given sample or query object in specific features. Its main goal is to support applications such as navigation behavior analysis, navigation safety monitoring, and waterway management optimization by revealing the similarities between ships in terms of motion direction, speed, stay time, and path selection.

[0004] At present, the technical methods for spatiotemporal trajectory similarity matching are mainly based on information technologies such as data mining and machine learning. By preprocessing the spatiotemporal trajectory data, extracting features, and calculating similarity, the similarity evaluation between different ship trajectories can be achieved. Among them, similarity calculation is the core link, and commonly used methods include Euclidean distance and dynamic time warping (DTW). These methods can reflect the similarity between trajectories to a certain extent, but they still face some challenges when processing inland ship trajectory data. For example, the navigation speed of inland ships is usually slow, the time span of trajectory data is large, and it is greatly affected by the environment. Therefore, traditional similarity calculation methods may not accurately reflect the actual similarity between trajectories. In addition, the trajectory data of inland ships often contains noise and outliers, which also increases the difficulty of similarity matching. Summary of the invention

[0005] In view of this, the purpose of the present invention is to provide an inland vessel AIS trajectory similarity query method and system based on local sensitive hashing. It is used to process motion change and spatial position similarity query problems in large-scale data. For motion change similarity query, a characterization dimensionality reduction method is adopted to discretize the two-dimensional trajectory data into a character sequence, effectively solving the problems that traditional spatial position encoding and trajectory matching methods fail to cover. This conversion not only reduces the computational complexity, but also significantly improves the query efficiency. In addition, the method can solve the spatial position similarity query problem by constraining the starting point position. Compared with other methods, this method also has a major advantage in that it can realize the query of unequal length spatiotemporal trajectory sequence data, thereby providing more flexible and accurate query results.

[0006] In order to achieve the above object, the present invention provides the following technical solutions:

[0007] A method for similarity query of inland river vessel AIS trajectory based on local sensitive hashing comprises the following steps:

[0008] The inland river AIS spatiotemporal trajectory data is processed by characterization and dimension reduction, discretized into character sequences, and feature extraction is performed;

[0009] Generate a MinHash signature based on the extracted features;

[0010] Using the local sensitive hashing (LSH) technology, the trajectory data is assigned to the corresponding LSH bucket according to the MinHash signature;

[0011] Perform character dimension reduction and feature extraction on the query trajectory data to generate the MinHash signature of the query trajectory;

[0012] Using the MinHash signature of the query trajectory, retrieve similar historical trajectory data and filter out trajectory data that meets the time and space range requirements;

[0013] For the selected trajectory data, the Jaccard similarity calculation method is used to perform similarity threshold filtering to obtain a similar trajectory data set;

[0014] For similar trajectory datasets, the longest common subsequence (LCSS) algorithm and sliding window are used to extract the target spatiotemporal trajectory segments that meet the similarity threshold.

[0015] Output the target spatiotemporal trajectory segments and sort them according to their similarity.

[0016] Furthermore, the characterization dimensionality reduction process includes discretizing the two-dimensional trajectory data into a character sequence.

[0017] Furthermore, the generation of the MinHash signature includes selecting k independent hash functions, each function is responsible for mapping the elements in the set into a larger hash space, and then calculating the hash values ​​of all elements in the two sets respectively, and selecting the minimum value therefrom to form the MinHash signature of each set.

[0018] Furthermore, the similarity threshold filtering includes using the Minhash signature between the query data and the queried data set to perform an unbiased estimation of the Jaccard similarity, and filtering the query results by setting a similarity threshold.

[0019] Furthermore, the target spatiotemporal trajectory segment acquisition includes setting a sliding window, sequentially extracting a sliding window character sequence of a specific length and executing a longest common subsequence (LCSS) algorithm to identify the longest common subsequence.

[0020] An inland river vessel AIS trajectory similarity query system based on local sensitive hashing, comprising:

[0021] A characterization dimension reduction module for characterization dimension reduction processing and feature extraction of inland river AIS spatiotemporal trajectory data;

[0022] MinHash signature module for generating MinHash signatures;

[0023] An LSH bucket allocation module for allocating trajectory data to LSH buckets using a locality sensitive hashing (LSH) technique;

[0024] A query trajectory processing module for characterizing, reducing, and extracting features from query trajectory data;

[0025] A query trajectory retrieval module for retrieving similar trajectory data using the MinHash signature of the query trajectory;

[0026] A similarity threshold filtering module for performing similarity threshold filtering using a Jaccard similarity calculation method;

[0027] A target trajectory segment acquisition module for extracting target spatiotemporal trajectory segments using the longest common subsequence (LCSS) algorithm and a sliding window;

[0028] An output module for outputting and sorting target spatiotemporal trajectory segments;

[0029] The characterization dimension reduction module performs characterization dimension reduction and feature extraction on the input inland river AIS spatiotemporal trajectory data, and outputs the processed data to the MinHash signature module;

[0030] The MinHash signature module receives the data output by the characterization dimension reduction module, generates a MinHash signature, and then outputs the MinHash signature to the LSH bucket allocation module;

[0031] The LSH bucket allocation module receives the MinHash signature output by the MinHash signature module, and uses the local sensitive hashing LSH technology to allocate the trajectory data to the LSH bucket, and then outputs the LSH bucket allocation result to the query trajectory retrieval module;

[0032] The query trajectory processing module performs character dimension reduction processing and feature extraction on the input query trajectory data, generates the MinHash signature of the query trajectory, and then outputs the MinHash signature of the query trajectory to the query trajectory retrieval module;

[0033] The query trajectory retrieval module receives the LSH bucket allocation result output by the LSH bucket allocation module and the MinHash signature of the query trajectory output by the query trajectory processing module, retrieves trajectory data similar to the query trajectory, and filters out trajectory data that meets the time and space range requirements, and then outputs the screening results to the similarity threshold filtering module;

[0034] The similarity threshold filtering module receives the trajectory data output by the query trajectory retrieval module, and performs similarity threshold filtering using the Jaccard similarity calculation method to obtain a similar trajectory data set, and then outputs the similar trajectory data set to the target trajectory segment acquisition module;

[0035] The target trajectory segment acquisition module receives the similar trajectory data set output by the similarity threshold filtering module, and uses the longest common subsequence LCSS algorithm and sliding window to extract the target spatiotemporal trajectory segment, and then outputs the target spatiotemporal trajectory segment to the output module;

[0036] The output module receives the target spatiotemporal trajectory segments output by the target trajectory segment acquisition module, outputs the target spatiotemporal trajectory segments, and sorts them according to the similarity.

[0037] The beneficial effects of the present invention are:

[0038] (1) Effectively reduce the computational complexity and utilize locality-sensitive hashing technology to reduce query consumption in large-scale data sets.

[0039] (2) In terms of query function, it effectively solves the problems that traditional spatial position encoding and trajectory matching methods cannot cover, and realizes unequal length sequence matching of motion change similarity query and spatial position similarity query.

[0040] (3) Without changing the existing storage model and taking into full account the characteristics of the data, a feasible solution is provided for motion change and spatial position similarity queries in large-scale datasets.

[0041] Other advantages, objectives and features of the present invention will be described in the following description to some extent, and to some extent, will be obvious to those skilled in the art based on the following examination and study, or can be taught from the practice of the present invention. The objectives and other advantages of the present invention can be realized and obtained through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] In order to make the purpose, technical solutions and advantages of the present invention more clear, the present invention will be described in detail below in conjunction with the accompanying drawings, wherein:

[0043] Figure 1 It is a flow chart of a trajectory similarity query method based on locality sensitive hashing;

[0044] Figure 2 A query flow chart of a trajectory similarity query method based on local sensitive hashing proposed by the present invention;

[0045] Figure 3 This is a schematic diagram of character dimensionality reduction provided by the present invention. DETAILED DESCRIPTION

[0046] The following describes the embodiments of the present invention by specific examples, and those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the illustrations provided in the following embodiments only illustrate the basic concept of the present invention in a schematic manner, and the following embodiments and features in the embodiments can be combined with each other without conflict.

[0047] Among them, the drawings are only used for illustrative explanations, and they only represent schematic diagrams rather than actual pictures, and should not be understood as limitations on the present invention. In order to better illustrate the embodiments of the present invention, some parts of the drawings may be omitted, enlarged or reduced, and do not represent the size of actual products. For those skilled in the art, it is understandable that some well-known structures and their descriptions in the drawings may be omitted.

[0048] The same or similar numbers in the drawings of the embodiments of the present invention correspond to the same or similar parts; in the description of the present invention, it should be understood that if the terms "upper", "lower", "left", "right", "front", "rear", etc. indicate the orientation or position relationship, they are based on the orientation or position relationship shown in the drawings, which is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operate in a specific orientation. Therefore, the terms describing the position relationship in the drawings are only used for illustrative purposes and cannot be understood as limiting the present invention. For ordinary technicians in this field, the specific meanings of the above terms can be understood according to specific circumstances.

[0049] See also Figure 1 to Figure 3 The technical solution adopted by the present invention is to use character dimension reduction method, LSH technology and matching skills to solve the motion change and spatial position similarity query problems in large-scale data sets.

[0050] First, the spatiotemporal trajectory data is processed by characterization and dimension reduction and feature extraction is completed, in order to convert complex spatiotemporal information into a feature form that is easy to index; next, these features are used to generate MinHash signatures, and LSH buckets are allocated accordingly, in order to improve search efficiency; then, LSH bucket search is performed to retrieve historical spatiotemporal trajectory data sets similar to the query spatiotemporal trajectory; finally, the target trajectory segment that meets the similarity threshold is found by combining LCSS and sliding windows. The specific steps are as follows:

[0051] (1) Characterization, dimensionality reduction and feature extraction of inland river AIS spatiotemporal trajectory data

[0052] (2) Constructing LSH Buckets

[0053] ①MinHash signature generation. When performing MinHash calculations based on the extracted character sequence features, we first select k independent hash functions, each of which is responsible for mapping the elements in the set to a larger hash space. Then, for each hash function, we calculate the hash values ​​of all elements in the two sets and select the minimum value. These minimum values ​​form the MinHash signatures of their respective sets. This step effectively captures the characteristics of the set and provides a basis for comparing the similarity of the sets later.

[0054] ②LSH bucket allocation. Based on the generated MinHash signature, the local sensitive hashing (LSH) technology is applied to classify the data items and assign them to the corresponding LSH buckets. If the MinHash signatures of two data items are similar, they are considered similar and have a high probability of being assigned to the same LSH bucket. In order to ensure efficient retrieval of similar items, a similarity threshold is set. Only when the similarity between data items exceeds this threshold will they be classified into the same LSH bucket. This method aims to optimize the fast search process of similar items and improve retrieval efficiency.

[0055] (3) Extraction of similar data character sets.

[0056] ① Characterization processing and feature extraction of query data.

[0057] ②Query data LSH bucket search. The query spatiotemporal trajectory data is processed by the MinHash algorithm to generate the MinHash signature of the data, and these signatures are used to map the data to the corresponding LSH bucket. Next, by retrieving the data in a specific LSH bucket, candidates similar to the query data are quickly extracted. Finally, according to the set time and space range requirements, qualified data is filtered out from these candidates.

[0058] ③Similarity threshold filtering. Use the Minhash signature between the query data and the queried data set to perform an unbiased estimate of the Jaccard similarity, filter the query results by setting the similarity threshold, arrange the query results in order, and then output them. The expression of Jaccard similarity is:

[0059]

[0060] In the formula, A and B are two sets, |A∩B| represents the number of elements in the intersection of sets A and B, and |A∪B| represents the number of elements in the union of sets A and B. Jaccard similarity is an indicator to measure the similarity between two sets, with values ​​ranging from 0 to 1, where 0 means that the two sets are completely dissimilar and 1 means that the two sets are exactly the same.

[0061] (4) Acquisition of target space-time trajectory segments.

[0062] Assume that the similarity threshold is K, the query character sequence is Q, and the length is n1. The character sequence to be matched is T, and the length is n2 (n2>(2-K)n1, rounded to the nearest integer). The specific steps are as follows:

[0063] ① Set the sliding window to n3 (n3 = (2-K)n1, rounded to the nearest integer). The length n3 is selected mainly because the longest common subsequence between the query sequence and the sequence to be matched is the query character sequence, which ensures similarity from an extreme perspective;

[0064] ② When analyzing the character sequence to be matched, the sliding window character sequence of a specific length is sequentially extracted and the longest common subsequence (LCSS) algorithm is executed to identify the longest common subsequence L, whose length is recorded as n4. According to the matching results, the situation can be divided into two cases:

[0065] 1) Successful match. Determine the starting position X and ending position Y of L in T, and calculate the length of the segment as n4, and calculate the similarity as If the similarity is greater than the similarity threshold K, the data is retrieved and the matching process is completed.

[0066] 2) Unsuccessful match. Determine the starting position X and ending position Y of L in T, and calculate the length of the segment as n4, and calculate the similarity as If the similarity is less than the similarity threshold K, the similarity does not meet the standard, and the first matched character position is skipped, the sliding window character sequence is regenerated from the next position, and the matching process is restarted.

[0067] ③ Extract the target space-time trajectory segment data according to the matched character information, obtain the motion change similarity query results, and arrange the output results in order of similarity.

[0068] ④ If you want to perform a spatial location similarity query, you can set a limited range based on the starting point of the query spatiotemporal trajectory data, and output the results in descending order of similarity.

[0069] The LSH-based spatiotemporal trajectory similarity query method, in terms of query efficiency, discretizes the spatiotemporal trajectory data into character sequences by adopting a characterization dimensionality reduction method, effectively reducing the computational complexity, and using local sensitive hashing technology to reduce the query consumption in large-scale data sets. In terms of query function, it effectively solves the problems that traditional spatial position encoding and trajectory matching methods fail to cover, and realizes unequal length sequence matching for motion change similarity query and spatial position similarity query. This method provides a feasible solution for motion change and spatial position similarity queries in large-scale data sets without changing the existing storage model and fully considering the data characteristics.

[0070] Embodiment 1:

[0071] Query target: Query ships with a similar navigation track to the "Yangtze River" ship in a certain section of the Yangtze River from May 1 to May 10, 2023.

[0072] step:

[0073] Data preparation: Collect AIS trajectory data of all ships in a certain section of the Yangtze River from May 1 to May 10, 2023, including the trajectory data of the "Yangtze River" ship.

[0074] Character dimensionality reduction: All AIS trajectory data is converted into character sequences, for example, longitude and latitude coordinates are converted into corresponding character representations. For example, longitude 118.78° is converted into the character sequence "11878".

[0075] Feature extraction: Extract features from the character sequence, such as the starting point, end point, path length, etc. of the trajectory. For example, the starting point of the extracted trajectory is "11878", the end point is "11879", and the path length is "10".

[0076] MinHash signature generation: MinHash signature generation is performed on the features of each trajectory to obtain a MinHash signature. For example, the trajectory features of the "Yangtze River" ship are MinHash signed to obtain the MinHash signature "abcd".

[0077] LSH bucket allocation: All MinHash signatures of all traces are allocated to corresponding LSH buckets. For example, the MinHash signature "abcd" is allocated to LSH bucket "1".

[0078] Query track processing: The AIS track data of the Yangtze River ship is subjected to character dimension reduction and feature extraction, and the MinHash signature of the query track is generated. For example, the MinHash signature of the query track is obtained as "abcd".

[0079] Similar trajectory retrieval: Use the MinHash signature "abcd" of the query trajectory to retrieve trajectory data similar to the query trajectory in LSH bucket "1".

[0080] Similarity threshold filtering: For the retrieved similar trajectory data, similarity threshold filtering is performed using the Jaccard similarity calculation method to obtain a similar trajectory data set. For example, if the similarity threshold is set to 0.8, the similar trajectory data set contains trajectory data whose MinHash signature has a Jaccard similarity greater than 0.8 with "abcd".

[0081] Target trajectory segment acquisition: For similar trajectory data sets, the longest common subsequence (LCSS) algorithm and sliding window are used to extract the target spatiotemporal trajectory segments. For example, the trajectory segment most similar to the "Yangtze River" ship trajectory is extracted, with a length of 5.

[0082] Result output: Output the target spatiotemporal trajectory segments and sort them according to the similarity. For example, output the trajectory segment with the highest similarity and display the starting point, end point, path length and other information of the trajectory.

[0083] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solution of the present invention can be modified or replaced by equivalents without departing from the purpose and scope of the technical solution, which should be included in the scope of the claims of the present invention.

Claims

1. A method for similarity query of inland river vessel AIS trajectory based on local sensitive hashing, characterized by: The following steps are involved: The inland river AIS spatiotemporal trajectory data is processed by characterization and dimension reduction, discretized into character sequences, and feature extraction is performed; Generate a MinHash signature based on the extracted features; Using the local sensitive hashing (LSH) technology, the trajectory data is allocated to the corresponding LSH bucket according to the MinHash signature; Perform character dimension reduction and feature extraction on the query trajectory data to generate the MinHash signature of the query trajectory; Using the MinHash signature of the query trajectory, retrieve similar historical trajectory data and filter out trajectory data that meets the time and space range requirements; For the selected trajectory data, the Jaccard similarity calculation method is used to perform similarity threshold filtering to obtain a similar trajectory data set; For similar trajectory datasets, the longest common subsequence (LCSS) algorithm and sliding window are used to extract target spatiotemporal trajectory segments that meet the similarity threshold. Output the target spatiotemporal trajectory segments and sort them according to their similarity.

2. According to claim 1, a method for similarity query of inland river vessel AIS trajectory based on local sensitive hashing is characterized by: The characterization dimensionality reduction process includes discretizing the two-dimensional trajectory data into a character sequence.

3. According to claim 1, a method for similarity query of inland river vessel AIS trajectory based on local sensitive hashing is characterized by: The generation of the MinHash signature includes selecting k independent hash functions, each function is responsible for mapping the elements in the set into a larger hash space, and then calculating the hash values ​​of all elements in the two sets respectively, and selecting the minimum value from them to form the MinHash signature of each set.

4. According to claim 1, a method for similarity query of inland river vessel AIS trajectory based on local sensitive hashing is characterized in that: The similarity threshold filtering includes using the Minhash signature between the query data and the queried data set to perform unbiased estimation of the Jaccard similarity, and filtering the query results by setting a similarity threshold.

5. According to claim 1, a method for similarity query of inland river vessel AIS trajectory based on local sensitive hashing is characterized in that: The target spatiotemporal trajectory segment acquisition includes setting a sliding window, sequentially extracting a sliding window character sequence of a specific length and executing a longest common subsequence (LCSS) algorithm to identify the longest common subsequence.

6. A local sensitive hashing based inland river vessel AIS trajectory similarity query system, characterized by: include: A characterization dimension reduction module for characterization dimension reduction processing and feature extraction of inland river AIS spatiotemporal trajectory data; MinHash signature module for generating MinHash signatures; An LSH bucket allocation module for allocating trajectory data to LSH buckets using a locality sensitive hashing (LSH) technique; A query trajectory processing module for characterizing, reducing, and extracting features from query trajectory data; A query trajectory retrieval module for retrieving similar trajectory data using the MinHash signature of the query trajectory; A similarity threshold filtering module for performing similarity threshold filtering using a Jaccard similarity calculation method; A target trajectory segment acquisition module for extracting target spatiotemporal trajectory segments using the longest common subsequence (LCSS) algorithm and a sliding window; An output module for outputting and sorting target spatiotemporal trajectory segments; The characterization dimension reduction module performs characterization dimension reduction and feature extraction on the input inland river AIS spatiotemporal trajectory data, and outputs the processed data to the MinHash signature module; The MinHash signature module receives the data output by the characterization dimension reduction module, generates a MinHash signature, and then outputs the MinHash signature to the LSH bucket allocation module; The LSH bucket allocation module receives the MinHash signature output by the MinHash signature module, and uses the local sensitive hashing LSH technology to allocate the trajectory data to the LSH bucket, and then outputs the LSH bucket allocation result to the query trajectory retrieval module; The query trajectory processing module performs character dimension reduction processing and feature extraction on the input query trajectory data, generates the MinHash signature of the query trajectory, and then outputs the MinHash signature of the query trajectory to the query trajectory retrieval module; The query trajectory retrieval module receives the LSH bucket allocation result output by the LSH bucket allocation module and the MinHash signature of the query trajectory output by the query trajectory processing module, retrieves trajectory data similar to the query trajectory, and filters out trajectory data that meets the time and space range requirements, and then outputs the screening results to the similarity threshold filtering module; The similarity threshold filtering module receives the trajectory data output by the query trajectory retrieval module, and performs similarity threshold filtering using the Jaccard similarity calculation method to obtain a similar trajectory data set, and then outputs the similar trajectory data set to the target trajectory segment acquisition module; The target trajectory segment acquisition module receives the similar trajectory data set output by the similarity threshold filtering module, and uses the longest common subsequence LCSS algorithm and sliding window to extract the target spatiotemporal trajectory segment, and then outputs the target spatiotemporal trajectory segment to the output module; The output module receives the target spatiotemporal trajectory segments output by the target trajectory segment acquisition module, outputs the target spatiotemporal trajectory segments, and sorts them according to the similarity.

Images