An efficient organization and management method of massive spatio-temporal data

Abstract

The invention discloses an efficient organization and management method of massive spatio-temporal data. The overall steps of the method are as follows: step 1, uploading data; 2, data processing; 3,data storage; 4, dynamically dividing the spatial data into blocks; a, constructing a vector data Key-Value data model; B. HDFS vector data being stored in blocks. The invention aims at the information access safety in the large concurrent access, high concurrency access, multi-source heterogeneous spatio-temporal information fusion storage and updating, performs research on cloud-based solutions,to realize the effective integrated management of massive heterogeneous data and improve the distribution, access and operation performance of spatial data in the distributed environment, at the sametime, the technology of GIS data distributed dynamic partition is studied to optimize the distribution efficiency of GIS service engine, and to meet the smart city geospatial data efficient storage management and sharing distribution.

Description

technical field [0001] The invention relates to an efficient organization and management method, in particular to an efficient organization and management method for massive spatio-temporal data. Background technique [0002] In recent years, the rapid development of geospatial information technology has provided massive data sources for GIS (Geographic Information System). With the rapid expansion of the amount of spatial data and the deepening of people's dependence on spatial information, the contradiction between the characteristics of spatial information such as large data volume, complex structure, distributed heterogeneous storage and the acquisition efficiency of spatial data entities and their processing functions Increasingly prominent, the service of GIS functions and the intelligence of data and function acquisition provide a feasible idea for solving this contradiction. [0003] At present, high-intensity data storage management, high-performance computing, hig...

AI Technical Summary

Problems solved by technology

[0003] At present, high-intensity data storage management, high-performance computing, high-concurrent access, and high-intensity application technologies have weak support for big data. Storage applications and sharing cannot provide powerful geospatial analysis and computing services, and the scalability is weak. Bearing the pressure of high concurrent access, the application support ability is weak, and it cannot carry richer applications

[0004] However, the different storage methods of multi-source heterogeneous spatio-temporal data make management inconvenient; local (remote) files, management databases and other storage methods are mixed, making it difficult to perform unified and automated management, such as backup, security, data output and retrieval etc.; structured and unstructured data are often stored separately and in different ways, and the semantics, syntax, and capabilities of different spatial databases and file formats vary widely, which increases the difficulty of application development; under high concurrency, centralized data storage is Bottleneck, and the centralized storage method is likely to cause a sharp drop in application performance

Images