Marine space-time big data parallel KNN query processing method based on PID

Abstract

The invention provides a marine space-time big data parallel KNN query processing method based on PID, and relates to the technical field of space-time big data management. According to the method, a PID controller technology which is widely used in the industry is introduced for the first time, and variable step size searching processing based on a feedback mechanism is achieved. The method comprises the following steps: firstly, pre-processing collected marine data, dividing the data by adopting a grid division method, and indexing the pre-processed marine data by adopting a grid indexing technology on the basis; coding each grid unit in a row sorting mode; judging which rows and columns are within the radius range of a circle by using a row sorting grid index, so as to directly judge whether the rows and the columns intersect with the circle or not; when KNN query is carried out, the adjustability of a PID system is utilized, the search range is dynamically adjusted through negative feedback, dynamic prediction of the query radius in KNN query processing is achieved, the number of KNN query times is reduced, and therefore the KNN query processing speed is increased.

Description

technical field [0001] The invention relates to the technical field of spatiotemporal big data management, in particular to a PID-based parallel KNN query processing method for marine spatiotemporal big data. Background technique [0002] Since the 21st century, with the rapid development of information technology and the rapid development of ocean observation technologies such as ocean remote sensing and ocean buoys, the scale of ocean data has exploded and has become a very important type of big data. The ocean field has entered the era of big data. KNN query usually refers to given a spatial data set and a query point, and returns the k results closest to the query point that meet the query conditions. As a very important spatial query operation, KNN query is widely used in space application systems, and also has important applications in marine application systems such as marine detection, marine rescue, and marine information privacy protection. How to perform efficien...

AI Technical Summary

Problems solved by technology

The traditional KNN query processing method usually adopts a centralized data processing method, which is not suitable for the processing of marine big data. However, the existing KNN query processing algorithms in a distributed environment, such as ParallelCircularTrip, etc., are mostly based on the MapReduce framework. MapReduce is Disk-based processing framework, less efficient for iterative processing

At the same time, these existing algorithms usually use index structures such as grid index and R-tree index, and their indexing efficiency is not high.

When performing KNN query processing, the query radius growth step is fixed, resulting in too many queries and subsequent calculations are too large, which affects the efficiency of query processing

Images