GeoWebCache distributed storage method and device based on Hbase

Abstract

The invention provides a GeoWebCache distributed storage method and a GeoWebCache distributed storage device based on an Hbase. The storage method comprises the following steps of storing tile data cached in a GeoWebcache into an Hbase (Hardware Base); performing grouping and merging processing on the tile data stored in the Hbase; and storing the processed tile data into a Moosefs (Moosefs) file system. According to the method, the small tile data are integrated into the Hfile large file data structure to be stored, so that unnecessary storage overhead cannot be generated; required tile data can be quickly obtained from mass data through rowkey, and the requirement for reading speed is met; tile data are stored in the distributed file system through the Hbase, the capacity can be increased by increasing storage nodes, and expandability is provided for storage of the tile data; meanwhile, the distributed file system shares read-write loads, and the read-write performance is improved.

Description

technical field [0001] The invention belongs to the technical field of geographic information, and in particular relates to a distributed storage method and device of GeoWebCache based on Hbase. Background technique [0002] Currently, GeoServer map server supports WCS 1.0, WMS 1.1.1 and WFS 1.0 standards, and integrates GeoWebCache component by default. [0003] GeoWebCache provides a tile cache function for WMS services. When a client requests a map service, GeoWebCache intercepts the WMS call request and returns the generated Tiles cache. If the request does not have a corresponding server cache, GeoServer will generate it for the requested layer. For the corresponding Tiles, for maps with relatively small data, the tile caching function provided by GeoWebCache is completely sufficient, but if it is a map with a relatively large amount of data, the tile caching function provided by GeoWebCache often cannot meet the storage requirements. [0004] Setting the cache directo...

AI Technical Summary

Problems solved by technology

[0003] GeoWebCache provides a tile cache function for WMS services. When a client requests a map service, GeoWebCache intercepts the WMS call request and returns the generated Tiles cache. If the request does not have a corresponding server-side cache, GeoServer generates it for the requested layer. Corresponding Tiles, for maps with relatively small data, the tile caching function provided by GeoWebCache is completely sufficient, but if it is a map with a relatively large amount of data, the tile caching function provided by GeoWebCache often cannot meet the storage requirements

[0004] Setting the cache directory of GeoWebcache to the Hdfs file system can realize large-capacity storage, but in the Hdfs file system, any file, directory or block is represented by a piece of metadata in the memory of the NameNode node, and this is subject to the physical Due to the limitation of memory capacity, the effect of processing small files (files whose size is significantly smaller than the block size on Hdfs (default 128MB)) is very inefficient. A large number of small files will cause a large amount of metadata to be generated, causing large redundancy in storage. In addition, reading the required tiles in a large number of small files has high requirements on the performance of the database, which seriously affects the performance of data reading and writing.

Images