Multi-terminal storage fusion method for geological tile data

Abstract

The invention provides a multi-terminal storage fusion method for geological tile data, which is characterized in that all tile data are stored in a file in a single file form, and MBTiles format data of geological tiles are organized by using an SQLite database. A reasonable database is designed, image data are stored in the database according to a primary key, and query is carried out through the database in the use process. The method has the advantages that management is more convenient, due to the existence of the SQL language, tedious file operation can be replaced by one SQL statement, in addition, the form of a single file is simpler, and the query efficiency can be improved by adding a disk and a memory cache.

Description

Technical Field

[0001] The present invention relates to the field of geographic information processing technology, and in particular to a multi-terminal storage fusion method for geological tile data. Background Art

[0002] At present, most of the systems and software for large-scale geographic information services at home and abroad use the pyramid model to store and manage map tile data, but most file systems and transmission protocols have certain limitations on the number of tiles that can be processed in the millions. When storing more than a certain number of files under different file systems, the response will be slow, and there may even be a limit on the maximum number of files. In addition, the read, write and transmission performance of millions of tile data are not high when deployed or migrated. Geological tile data is often stored in the form of multiple files. The multi-file format means that all tile images are stored in the device storage in the form of independent files. When there are 1,000 tile images, there are also 1,000 corresponding tile image files in the device.

[0003] The tile data of multiple file storage is transferred from PC to multiple devices such as Android devices. The Media Transfer Protocol (MTP) is used to transfer files, and the speed is often unacceptably slow when transferring a large number of files. In addition, due to the differences between devices, when the MTP mode takes too long to transfer files, the device often becomes unresponsive, so the user needs to disconnect the phone and restart the file transfer, which may not be successful even after repeated attempts.

[0004] Developers face a harsh reality with millions of map tiles: most file systems and transfer protocols are not very efficient for handling millions of images. On a FAT32 disk, a folder can contain up to 65536 files, HFS can list up to 32,767 files, and EXT3 becomes very slow when there are more than 20,000 files. Copying millions of tile data over USB or network is inefficient and slow. Summary of the invention

[0005] The purpose of the technical solution of the present invention is to store millions of tile data in one file and support data fusion and transmission on multiple platforms such as PC, WEB and mobile terminals.

[0006] In order to achieve the above-mentioned invention object, the technical solution of the present invention provides a multi-terminal storage fusion method of geological tile data, comprising the following steps:

[0007] PC side: The pyramid model based on the quadtree structure is divided from the upper layer to the lower layer with the same geographical coverage to obtain tiles. The tiles store the level, row number, and column number of the tiles on the two-dimensional plane map generated according to the preset projection to form tile location information;

[0008] Find the corresponding tile according to the tile location information, use the level, row number, column number as the primary key to store the level, row number, column number and tile string identifier to obtain the map table structure, use the tile string identifier as the primary key to store the tile string identifier and binary impact data to obtain the images table structure, and the binary impact data will be converted into binary BLOB format and stored in MBTiles;

[0009] Create a joint index of the map table structure based on the row number and column number, and create an index of the images table structure based on the tile string identifier;

[0010] Get the map tile data of each level of tiles, set the number of tiles to be stored in each MBTiles file, read the tile data in the largest level of the map tile data, get the location information and tile coordinate number range of all tiles in the level corresponding to the tile data in the largest level, and calculate the number of corresponding MBTiles files and the file coordinate number range;

[0011] Create column-by-column directories according to the numerical ranges corresponding to the columns in the file coordinate number range, create empty MBTiles files in the column-by-column directories and name them with row values ​​to establish the organization of MBTiles files in the file system, calculate the corresponding MBTiles files according to the tile location information, and store them in the corresponding empty MBTiles files as SQLite databases;

[0012] Mobile side: Get the SQLite database from the PC side, and build the mobile side SQLite database based on the SQLiteOpenHelper abstract class, SQLiteDatabase database access class, and SQLiteCursor query structure operation class.

[0013] Preferably, the map scale corresponding to the upper layer is smaller than the map scale corresponding to the lower layer, and the ground resolution corresponding to the upper layer is smaller than the ground resolution corresponding to the lower layer.

[0014] Preferably, the level, row number and column number are compiled according to the longitude and latitude of the geographical coordinates corresponding to the tile.

[0015] Preferably, the numbering range of the row and column numbers is formulated according to a rule describing the ground resolution corresponding to the preset projection.

[0016] Preferably, the multi-terminal storage fusion method further includes removing redundant tile data in the images table structure according to the tile string identifier.

[0017] Preferably, the correspondence between the file coordinate numbers is as follows:

[0018]

[0019]

[0020] Among them, min_col is the upper left column number, min_row is the upper left row number, max_col is the lower right column number, max_row is the lower right row number, r is the number of rows, and c is the number of columns.

[0021] The technical solution of the present invention proposes a multi-terminal storage fusion method for geological tile data, proposes to store all tile data in a single file, and uses SQLite database to organize geological tile MBTiles format data. Design a reasonable database, store image data in the database according to the primary key, and query through the database during use. Its advantage is that it is more convenient to manage. Due to the existence of SQL language, a SQL statement can replace cumbersome file operations. In addition, the form of a single file is also more concise, and query efficiency can be improved by increasing disk and memory cache. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 A schematic diagram of a tile pyramid model provided by an embodiment of the present invention;

[0023] Figure 2 A schematic diagram of tile coordinate numbering provided by an embodiment of the present invention;

[0024] Figure 3 A schematic diagram of a file storage organization provided in an embodiment of the present invention;

[0025] Figure 4 A schematic diagram of a tile storage process provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0026] The present invention will be further described below in conjunction with specific embodiments. It should be understood that these embodiments are only used to illustrate the present invention and are not intended to limit the scope of the present invention. In addition, it should be understood that after reading the content taught by the present invention, those skilled in the art can make various changes or modifications to the present invention, and these equivalent forms fall within the scope limited by the appended claims of the application equally.

[0027] An embodiment of the present invention provides a multi-terminal storage fusion method for geological tile data, wherein the multi-terminal storage fusion method stores tile data in the MBTiles file format based on SQLite, and the multi-terminal storage fusion method comprises the following steps:

[0028] (1) Download SQLite on the PC: Download the 32-bit or 64-bit version of SQLite installation package that matches the Windows platform from the SQLite official website, double-click to open the installation package and install it according to the default options. After the installation is complete, add the SQLite installation directory to the system PATH in the advanced environment options of the computer.

[0029] (2) Constructing a tile pyramid based on MBTiles: The pyramid model is based on a quadtree structure and is constructed in a hierarchical and block-based manner. Each layer represents the same geographical coverage. As the layers are divided level by level, from the upper layer to the lower layer, the corresponding map scale becomes larger and larger, and the ground resolution represented becomes higher and higher. Figure 1 The tile location information should be the level, row number, and column number of the tile on a two-dimensional plane map generated by Mercator projection or other projection methods under the global grid system. The longitude and latitude of the corresponding geographic coordinates of the tile should be directly found through its location information, such as Figure 2 The tile coordinate number range refers to the grid coordinate number range corresponding to all tiles at the resolution of the layer corresponding to the projection rule, that is, the range of row numbers and column numbers.

[0030] The tile storage structure based on MBTiles is shown in Table 1 and Table 2. The map table contains four fields: zoom_level, tile_column, tile_row, and tile_id, which store the tile level, location coordinates, and tile string identifier, respectively, with (zoom_level, tile_column, tile_row) as the primary key. The images table contains tile_id and tile_data fields, which store the tile string identifier and tile image data, respectively, with tile_id as the primary key. The tile image data tile_data is converted into binary BLOB format and stored in MBTiles.

[0031] Table 1 map table structure

[0032] Field Name Data Types Notes zoom_level INTEGER Tiles level tile_column INTEGER Tile column number tile_row INTEGER Tile row number tile_id TEXT Tile string identifier

[0033] Table 2 images table structure

[0034] Field Name Data Types Notes tile_id TEXT Tile string identifier tile_data BLOB Binary Impact Data

[0035] Since a large amount of tile data is stored in an MBTiles file, it takes a long time to query the specified tile image data. Indexes are created for the map table and images table to improve query efficiency. For the images table, an index is created for the tile_id field; for the map table, a joint index is created for the tile_column field and the tile_row field. When querying the specified tile image data, conditional queries for tile_column and tile_row are usually performed at the same time in the MBTiles file, so it is necessary to create a joint index for the two fields, which not only shortens the query time but also saves storage space. In addition, in the images table, redundant tile data is removed based on the content of the tile_data field. The index relationship is changed from a one-to-one relationship in an ordinary database to a many-to-one relationship, which can effectively remove redundant tiles and improve tile call efficiency.

[0036] (3) Construct tile storage based on MBTiles: First, obtain the map tile data containing tiles of each level, and set the storage number of tiles in each MBTiles file to c×r, where c represents the number of columns and r represents the number of rows. After that, read the tile data in the largest level of the map tile data, and obtain the location information and tile coordinate number range of all tiles in this level. Calculate the total number of MBTiles files that need to be generated at the current level and the file coordinate number corresponding to each MBTiles file that needs to be generated. Assume that the tile coordinate number range of this layer is a rectangular quadrilateral [min_col, min_row, max_col, max_row], min_col, min_row are the upper left column and row numbers, and max_col, max_row are the lower right column and row numbers. Assuming that the coordinate number range of the MBTiles file generated corresponding to this range is [min_C, min_R, max_C, max_R], the corresponding relationship should satisfy the following formula:

[0037]

[0038] Create a directory for this level under the storage directory, and create directories according to the range of column C values. Create empty MBTiles files in each column directory and name them according to the row R value. That is, create an empty MBTiles file according to the path of "specified directory / current level / column number C / row number R.mbtiles". Figure 3 As shown, the organization of MBTiles files in the file system is established.

[0039] After that, all tiles of the level that have been read are calculated according to the position information of each tile, and the corresponding MBTiles file is stored in the file according to the specification. Assuming that the tile coordinate number of a tile is tile_col column and tile_row row, the corresponding MBTiles file coordinate column C and row R value should satisfy:

[0040]

[0041] Since in tile pyramid maps, map tiles are usually stored together in the order of their position coordinates, we can store each dataset in the corresponding MBTiles file according to the previously set number of tiles stored in the MBTiles file, with each c×r tile as a tile dataset according to the coordinates. Assuming that the position coordinate column number of an MBTiles file is C and the row number is R, the tile dataset that should be stored should be all tiles with position coordinates within the following range, such as Figure 4 As shown:

[0042] C×c, R×r, (C+1)×c-1, (R+1)×r-1

[0043] After all tiles of this level are stored, the next level of tiles is stored. The storage operation of tiles of each level is similar to the above steps until all data is stored.

[0044] (4) Creation and use of SQLite database on mobile devices. Each SQLite database is contained in only one file. There is no external permission system, database background process or configuration. Each .sqlite file is an independent database. You can copy a .sqlite file from the PC to the mobile device, and its rows, tables and indexes can be fully used.

[0045] The SQLite database is mainly composed of three classes to create and operate the database: SQLiteOpenHelper abstract class, which inherits from this class to implement user classes to provide database opening and closing functions. SQLiteDatabase database access class, which is used to insert records and query records in the database. SQLiteCursor query structure operation class, which is used to access records in the query results.

[0046] The mobile terminal divides the scheduling of tile data into three levels: tile cache for displaying the current area, pre-access tile cache, and external SQLite database. The size of the display cache is the same as the number of map tiles that the screen can display, which mainly depends on the screen resolution of the device. The size of the pre-access tile cache is much larger than the display cache capacity. It is mainly used to store map tiles that may be used for display. According to the data pre-reading strategy, tiles from other areas are pre-read into memory. Its capacity is mainly determined by the device memory. The external database is used to store multiple independent map tile data tables. Sorting by different fields can quickly search for tile data that meets the conditions, and facilitate data sharing between different applications. Its capacity is determined by the size of the device's external storage device.

[0047] When the program is initialized, it displays the 0th level tiles according to the map category selected by the user, and then reads the tile data around the current display area and the tile data that needs to be read after zooming in advance. When performing pre-access operations, tiles with the same zoom level and similar positions as the map tiles in the current display area have the highest priority, and tiles in other positions are second; when reading tiles of different zoom levels in advance, low-resolution tiles are read first, because low-resolution tiles take up less space and load faster. If the pre-access space is not full, high-resolution tiles can be loaded into the cache to further enhance the smoothness of map zooming.

[0048] The embodiment of the present invention proposes a multi-terminal storage fusion method for geological tile data to store map tiles. Map tiles are highly portable, have a small number of files, and are easy to back up. Due to the portability of SQLite, files can be directly operated on multiple platforms without setting up an environment or modifying parameters. Experiments show that due to the establishment of efficient indexes in the database, the method in this paper improves the query and reading speed of map tiles.

Claims

1. A multi-terminal storage fusion method for geological tile data, characterized in that: The following steps are involved: PC side: The pyramid model based on the quadtree structure is divided from the upper layer to the lower layer with the same geographical coverage to obtain tiles. The tiles store the level, row number, and column number of the tiles on the two-dimensional plane map generated according to the preset projection to form tile location information; Find the corresponding tile according to the tile location information, use the level, row number, column number as the primary key to store the level, row number, column number and tile string identifier to obtain the map table structure, use the tile string identifier as the primary key to store the tile string identifier and binary impact data to obtain the images table structure, and the binary impact data will be converted into binary BLOB format and stored in MBTiles; Create a joint index of the map table structure based on the row number and column number, and create an index of the images table structure based on the tile string identifier; Get the map tile data of each level of tiles, set the number of tiles to be stored in each MBTiles file, read the tile data in the largest level of the map tile data, get the location information and tile coordinate number range of all tiles in the level corresponding to the tile data in the largest level, and calculate the number of corresponding MBTiles files and the file coordinate number range; Create column-by-column directories according to the numerical ranges corresponding to the columns in the file coordinate number range, create empty MBTiles files in the column-by-column directories and name them with row values ​​to establish the organization of MBTiles files in the file system, calculate the corresponding MBTiles files according to the tile location information, and store them in the corresponding empty MBTiles files as SQLite databases; Mobile side: Get the SQLite database from the PC side, and build the mobile side SQLite database based on the SQLiteOpenHelper abstract class, SQLiteDatabase database access class, and SQLiteCursor query structure operation class.

2. A multi-terminal storage fusion method for geological tile data according to claim 1, characterized in that: The map scale corresponding to the upper layer is smaller than the map scale corresponding to the lower layer, and the ground resolution corresponding to the upper layer is smaller than the ground resolution corresponding to the lower layer.

3. A multi-terminal storage fusion method for geological tile data as claimed in claim 1, characterized in that: The level, row number, and column number are compiled according to the longitude and latitude of the geographic coordinates corresponding to the tile.

4. The multi-terminal storage fusion method of geological tile data according to claim 1, characterized in that: The numbering range of the row and column numbers is formulated according to the rule describing the ground resolution corresponding to the preset projection.

5. The multi-terminal storage fusion method of geological tile data according to claim 1, characterized in that: The multi-terminal storage fusion method also includes removing redundant tile data in the images table structure according to the tile string identifier.

6. A multi-terminal storage fusion method for geological tile data according to claim 1, characterized in that: The corresponding relationship between the file coordinate numbers is as follows: Among them, min_col is the upper left column number, min_row is the upper left row number, max_col is the lower right column number, max_row is the lower right row number, r is the number of rows, and c is the number of columns.

Images