Method for migrating massive spatial data in cluster fast

Abstract

The invention discloses a method for migrating massive spatial data in a cluster fast and belongs to the technical field of data migration. The method includes the steps: 1, a cluster storing image data is built; 2, a subdivision index gsot file in each host in the cluster is built; 3, a management node reads the subdivision index gsot file in each host in real time, and an index summary list is built; 4, a receiving host sends a set migration code to the management node; 5, the management node extracts an index code matched with the migration code and a corresponding index item of the index code or a nest index code and a corresponding index item of the nest index code; 6, the management node builds an index sub-list for the extracted index code and the index item of the index code and the nest index code and the index item of the nest index code, and images or image patches are obtained according to the index item of the index sub-list; and 7, the management node transmits the obtained images or image patches to the receiving host. The method is applied to fast migration of massive multi-source heterogeneous data in the cluster.

Description

technical field [0001] The invention relates to the technical field of data migration. Background technique [0002] With the development of satellite observation systems for the earth, people's ability to acquire remote sensing data, especially remote sensing image data, has been continuously improved. Therefore, the remote sensing image data obtained by people gradually presents the characteristics of multi-source, multi-scale, multi-temporal, global coverage and high resolution. , At the same time, the amount of data is growing explosively, and has reached the level of PB data. [0003] The abundance of earth observation methods has led to more and more sources of remote sensing image data, and the data generated by different data source methods are different in size, resolution, and image naming format. [0004] For remote sensing image data with different attributes such as map size and resolution in the cluster storage system, if necessary, all types of data within a ...

AI Technical Summary

Problems solved by technology

[0004] For remote sensing image data with different attributes such as map size and resolution in the cluster storage system, if required, all types of data within a certain region must be migrated. However, due to the heterogeneity of the data, it cannot be Unified benchmarks for query and retrieval must be indexed and searched separately for data of different framing standards according to their own unique codes

Since the data of different framing standards must be collected and sorted according to the corresponding framing specifications, this makes the migration process mostly at the level of manual operation. Manual selection, copying and migration of data will not only cause omissions, errors, etc. In the case of a very large amount of data, it will be very time-consuming

As a routine work, data migration has seriously affected the efficiency of data sharing between remote sensing information management systems.

Images