Remote sensing image indexing and storing method based on Uber H3 hexagonal grid

Abstract

The invention belongs to the field of space-time big data, and discloses a remote sensing image indexing and storing method based on a Uber H3 hexagonal grid, which comprises the following steps of: 1, for remote sensing images with different resolutions, cutting the remote sensing images by adopting Uber H3 grids with different resolutions; 2, storing the generated remote sensing image slices into different nodes according to a Hash code (consistent hash ring) and a remote sensing image slice storage algorithm; 3, generating a Main Table, a secondary index table and a B + tree while storing the remote sensing image slices; and 4, in combination with the generated Main Table, the secondary index table and the B + tree, selecting a KNN query algorithm, a spatio-temporal range query algorithm and other algorithms according to query requirements to obtain target data. Compared with a Google S2 remote sensing image storage index method based on a quadrilateral grid and a Uber H3-based remote sensing image storage index method without applying hash, a B + tree and a secondary index table, the storage efficiency of BSH3 is better than that of S2 and H3.

Description

Technical Field

[0001] The present invention belongs to the field of spatio-temporal big data, and particularly relates to the storage and indexing of multi-resolution and large-range time-series remote sensing image data. Background Art

[0002] With the continuous progress of remote sensing technology, the amount of remote sensing image data obtained has increased exponentially. These data have wide application values in the fields of environmental monitoring, resource management, urban planning, and disaster emergency. For example, in environmental monitoring, high-resolution remote sensing data is used to track the diffusion path of pollutants and monitor the changes in the ecological environment; in resource management, remote sensing data can support mineral resource exploration and agricultural resource assessment; in urban planning, remote sensing images are helpful for urban expansion monitoring and infrastructure construction; in disaster emergency, remote sensing data is crucial for disaster assessment and emergency response. However, the unprecedented growth of remote sensing big data has posed severe challenges to the storage of remote sensing image data. How to effectively store, manage, and quickly retrieve such a huge amount of data is an important problem that needs to be solved urgently. NoSQL has advantages such as strong scalability and concurrency when storing remote sensing images. Some experts have conducted research on it. As a typical NoSQL database, HBase can efficiently store structured and unstructured data. Its column family mechanism and storage characteristics make it very suitable for storing remote sensing images. However, the huge amount of remote sensing image data is not only reflected in the total data volume, but also in the data volume of a single image. With the improvement of image resolution, the data volume of a single image has also reached the GB level. Therefore, remote sensing images are often cut into tiles for storage. To maintain the spatial correlation between remote sensing images, researchers often combine space-filling curves to store adjacent slices in the same place. This sliced storage method helps to improve the data reading efficiency and retrieval speed because it can ensure that adjacent image slices in space are stored in physically adjacent positions, thus reducing the disk seek time when reading data. But there are still some problems: it is difficult to uniformly manage multi-resolution remote sensing images; the node load in the cluster is unbalanced; these will all affect the storage and query efficiency of images.

[0003] GeoHash, Google S2, and Uber H3 are the three most commonly used geocoding methods in commerce. GeoHash and Google S2 divide the Earth using quadrilaterals, while Uber H3 uses hexagons as the basic unit to fill the Earth. When storing remote sensing image tiles, researchers focused on designing row keys for remote sensing image tiles, using geocoding such as Google S2 and GeoHash for spatial coding, maintaining the spatial correlation between tiles, and storing them in HBase. For example, CAO et al. used Hilbert coding and variants of Hilbert coding to design the row keys of tiles for efficient tile querying. Wang et al. combined GeoSOT codes with metadata as geocoding to construct the row keys of tiles, thus achieving efficient remote sensing image storage. X. Wang et al. used S2 curves and load balancing strategies to construct the row keys of tiles to achieve efficient storage of remote sensing images. Liu et al. combined geographical coordinates with other attributes to construct the row keys of tiles, etc.

[0004] Geocoding can maintain a certain degree of spatial locality, making nearby points still close after mapping, which is suitable for spatial queries. However, some encodings lack flexibility when dealing with data of different resolutions and are difficult to uniformly manage multi-resolution data.

[0005] In the existing remote sensing image tile indexing methods based on geocoding, the following problems exist:

[0006] (1) Geohash recursively divides the geographical area into smaller grids, and each grid corresponds to a unique string. The advantage of this method is simple coding, but the spatial locality is weak, and the gap between encodings at different levels is too large, making it difficult to manage multi-resolution remote sensing images. Google S2 uses quadrilaterals to divide the Earth, with good hierarchy, but the efficiency of neighborhood queries is not as good as that of hexagons.

[0007] (2) Temporal remote sensing images have high temporal resolution and large accumulation, and efficient storage and management need to be carried out for their time dimension. Summary of the Invention

[0008] The objective of the present invention is that among the common figures for constructing a global discrete grid system, namely triangles, quadrilaterals, and hexagons, the grid index units of triangles and squares have different neighbor distances, while those of hexagon grid index units are different. The distance between the center point of a hexagon and all its adjacent center points is equal, which makes the distances between all neighbors uniform. Hexagons have the characteristics of the highest sampling efficiency, the shape closest to a circle, being conducive to spatial analysis and simulation modeling, and having consistent topological relationships, which is conducive to the implementation of analysis algorithms such as proximity and connectivity. Therefore, this paper proposes a method for storing remote sensing images of the global grid Uber H3 and HBase based on hexagon partitioning, with the following contributions:

[0009] (1) Higher query efficiency and multi-resolution support: Uber H3 has higher spatial query and proximity query efficiency compared to other global discrete grid systems. This is of certain significance for environmental monitoring and geological disaster monitoring, where it is necessary to quickly obtain image data of specific areas for real-time analysis and decision-making. At the same time, it has a globally unified grid system, which can adapt to remote sensing image data of different resolutions and is conducive to the integration and analysis of multi-source data.

[0010] (2) Temporal query: Combine the B+ tree and secondary index to achieve efficient management of the time dimension of remote sensing images.

[0011] To achieve the above objective, the present invention provides the following technical solution: A method for indexing and storing remote sensing images based on the Uber H3 hexagon grid, the method comprising the following steps:

[0012] Step 1: Collect data, and use 98 remote sensing images from Sentinel-2 with resolutions of 10m, 20m, and 60m in Yunnan Province during the period from March 3, 2023 to April 21, 2023 as the data source;

[0013] Step 2: For remote sensing images of different resolutions, select an appropriate resolution Uber H3 grid, and then use the remote sensing slicing algorithm to cut it;

[0014] Step 3: Use the cutting algorithm of the H3 grid to cut remote sensing images of different resolutions. The specific steps are as follows: Obtain the H3 cell boundaries of the corresponding resolution remote sensing images, calculate the H3 index of the cells, then crop according to the H3 cell boundaries, and use the H3 index as the unique identifier to output remote sensing image slices;

[0015] Step 4: For the cut remote sensing image slices, according to the resolution of the grid where they are located, obtain the H3 code of the grid center, and at the same time perform Unix encoding on the time when the remote sensing images are obtained; Combine the H3 code and the Unix code as the code of this remote sensing slice;

[0016] Step 5: Perform hash processing on the encoding of the remote sensing image slice, and place the resulting random 2-digit hash code at the first two positions of this remote sensing image slice. At this time, the encoding of each remote sensing image slice consists of a hash code + H3 code + Unix code; use this encoding as the rowkey to store all remote sensing image slices into Hbase;

[0017] Step 6: While storing the remote sensing image slices, generate a B+ tree and a secondary index table. The specific content is as follows: The RowKey of the secondary index table is H3 index, and the corresponding Value is all remote sensing image slices related to this H3 index, which is used to store the Rowkey information of remote sensing images in the same area in the MainTable; The B+ tree uses the Unix timestamp as the key and the spatial coordinates as the value;

[0018] Step 7: KNN query; The specific implementation details of the KNN query are as follows: The algorithm first converts the given longitude and latitude into an H3 hexagonal index with a specified resolution and obtains the center point of the index cell; then, the algorithm calculates all cells within a specified radius k from the center H3 cell, and forms a set of these cell indices to form a "filled disk" that includes the center cell and all cells within k around it; next, use these H3 indices to query the corresponding row keys, that is, rowkeys, in the secondary index table; for each valid H3 index, the algorithm extracts the relevant row keys from the secondary index table and stores them; finally, the algorithm queries in the main table according to these row keys to obtain the final result;

[0019] Step 8: Spatiotemporal range query; The spatiotemporal range query algorithm designed by this method includes a spatial query algorithm part, and the specific implementation details are shown in Algorithm 2: The spatiotemporal query algorithm filters and queries data through two dimensions of time and space; first, use the B+ tree index to filter out relevant H3 indices within the given time range; then generate a polygon according to the specified geographical boundary, and use the H3 library to fill the polygon into a set of H3 indices; take the intersection of the time and space indices to obtain a set of H3 indices that meet both time and space conditions; query these filtered H3 indices to further filter out the row keys, that is, rowkeys, that meet the time conditions; finally, the algorithm queries in the main table according to these row keys to obtain the final result;

[0020] Step 9: Model performance evaluation; To verify the performance of the remote sensing image indexing and storage model BSH3 based on Uber H3 hexagonal discrete grid proposed in this paper, this method compares the proposed model with the remote sensing image storage and indexing methods based on Uber H3 and Google S2 that do not apply the consistent hash ring, B+ tree, and secondary index table methods.

[0021] Preferably, in step 2, specifically as follows: For remote sensing images with a resolution of 10m, select the UberH3 grid with a resolution of 7.

[0022] Preferably, in step 2, for remote sensing images with a resolution of 20m, select the Uber H3 grid with a resolution of 6.

[0023] Preferably, in step 2, for remote sensing images with a resolution of 60m, select the Uber H3 grid with a resolution of 5.

[0024] Compared with the prior art, the beneficial effects of the present invention are:

[0025] 1. Through experimental comparison with the Google S2 remote sensing image storage and indexing method based on quadrilateral grid and the Uber H3-based remote sensing image storage and indexing method without applying hash, B+ tree, and secondary index table, the storage efficiency of BSH3 is superior to that of S2 and H3, and the spatial range and spatio-temporal query efficiency are also about 4 to 5 times higher than other methods;

[0026] 2. In terms of KNN query efficiency, BSH3 is about 6 times higher than S2 and about 4 times higher than the ordinary H3 system. This model slices and stores remote sensing images based on the Uber H3 system and designs certain solutions for the problems of load balancing and spatio-temporal characteristics of remote sensing images, with good effects. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 It is a distance diagram between adjacent grids of different grids of the present invention;

[0028] Figure 2 It is a technical roadmap of the present invention;

[0029] Figure 3 It is a storage performance comparison diagram of different methods of the present invention;

[0030] Figure 4 It is a range query performance comparison diagram of the present invention;

[0031] Figure 5 It is a KNN query performance comparison diagram of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0032] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0033] A method for indexing and storing remote sensing images based on Uber H3 hexagonal grids, the method comprising the following steps:

[0034] Step 1: Collect data, and use 98 remote sensing images from Sentinel-2 with resolutions of 10m, 20m, and 60m in Yunnan Province from March 3, 2023 to April 21, 2023 as the data source;

[0035] Step 2: For remote sensing images with different resolutions, select appropriate resolution Uber H3 grids, and then use the remote sensing slicing algorithm to cut them, specifically as follows: for remote sensing images with a resolution of 10m, select an UberH3 grid with a resolution of 7; for remote sensing images with a resolution of 20m, select an Uber H3 grid with a resolution of 6; for remote sensing images with a resolution of 60m, select an Uber H3 grid with a resolution of 5;

[0036] Step 3: Use the cutting algorithm of the H3 grid to cut remote sensing images with different resolutions. The specific steps are as follows: obtain the H3 cell boundaries of the remote sensing images with the corresponding resolution, calculate the H3 index of the cells, and then crop according to the H3 cell boundaries. Use the H3 index as the unique identifier and output the remote sensing image slices;

[0037] Step 4: For the cut remote sensing image slices, obtain the H3 code of the grid center according to the resolution of the grid where they are located, and at the same time perform Unix encoding on the acquisition time of the remote sensing images; combine the H3 code and the Unix code as the code of this remote sensing slice;

[0038] Step 5: Perform hash processing on the codes of the remote sensing image slices, and place the obtained random 2-bit hash code at the first two positions of this remote sensing image slice. At this time, the code composition of each remote sensing image slice is hash code + H3 code + Unix code; use this code as the rowkey and store all the remote sensing image slices into Hbase;

[0039] Step 6: While storing the remote sensing image slices, generate a B+ tree and a secondary index table. The specific details are as follows: The RowKey of the secondary index table is the H3 index, and the corresponding Value is all the remote sensing image slices related to this H3 index, which is used to store the Rowkey information of the remote sensing images in the same area in the MainTable; The B+ tree uses the Unix timestamp as the key and the spatial coordinates as the value;

[0040] Step 7: KNN query; The specific implementation details of the KNN query are as follows: The algorithm first converts the given longitude and latitude into an H3 hexagonal index with a specified resolution and obtains the center point of the index cell; Then, the algorithm calculates all the cells within a specified radius k from the central H3 cell as the starting point, and forms a set of these cell indexes to form a "filled disk" that includes the central cell and all cells within the range of k around it; Next, use these H3 indexes to query the corresponding row keys, that is, rowkeys, in the secondary index table; For each valid H3 index, the algorithm extracts the relevant row keys from the secondary index table and stores them; Finally, the algorithm queries in the main table according to these row keys to obtain the final result;

[0041] Step 8: Spatiotemporal range query; The spatiotemporal range query algorithm designed in this method includes a spatial query algorithm part. The specific implementation details are shown in Algorithm 2: The spatiotemporal query algorithm filters and queries data through two dimensions of time and space; First, use the B+ tree index to filter out relevant H3 indexes within a given time range; Then generate a polygon according to the specified geographical boundary, and use the H3 library to fill the polygon into a set of H3 indexes; Take the intersection of the time and space indexes to obtain a set of H3 indexes that meet both the time and space conditions; Query these filtered H3 indexes to further filter out the row keys, that is, rowkeys, that meet the time conditions; Finally, the algorithm queries in the main table according to these row keys to obtain the final result;

[0042] Step 9: Model performance evaluation; In order to verify the performance of the remote sensing image indexing and storage model BSH3 based on the Uber H3 hexagonal discrete grid proposed in this paper, this method compares the proposed model with the remote sensing image storage and indexing methods based on Uber H3 and Google S2 that do not apply the consistent hash ring, B+ tree, and secondary index table methods.

[0043] The experimental results show that compared with the HBase-based storage model of Google S2 divided by quadrilateral grids, this scheme has significant advantages in storage efficiency and query performance, and can effectively support the application requirements of large-scale remote sensing image data.

[0044] It should be noted that in this text, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprising", "including" or any other variant thereof are intended to cover non-exclusive inclusion, such that a process, method, article or device comprising a series of elements not only includes those elements but also includes other elements not expressly listed, or further includes elements inherent to such process, method, article or device.

[0045] Although the embodiments of the present invention have been shown and described, those of ordinary skill in the art can understand that various changes, modifications, substitutions and variations can be made to these embodiments without departing from the principles and spirit of the present invention. The scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A remote sensing image indexing and storage method based on UberH3 hexagonal grid, characterized by: The method comprises the following steps: Step 1: Collect data, using 98 remote sensing images of Yunnan Province with resolutions of 10m, 20m, and 60m from Sentinel-2 from 2023-03-03 to 2023-04-21 as the data source; Step 2: For remote sensing images of different resolutions, select the appropriate resolution Uber H3 grid and then cut it using the remote sensing slicing algorithm; Step 3: Use the H3 grid cutting algorithm to cut remote sensing images of different resolutions. The specific steps are as follows: obtain the H3 cell boundary of the remote sensing image of the corresponding resolution, calculate the H3 index of the cell, and then cut according to the H3 cell boundary, use the H3 index as a unique identifier, and output the remote sensing image slice; Step 4: For the cut remote sensing image slice, obtain the H3 code of the center of the grid according to the resolution of the grid where it is located, and perform Unix coding on the time when the remote sensing image was acquired; combine the H3 code and Unix code as the code of this remote sensing slice; Step 5: Hash the code of the remote sensing image slice and place the random 2-bit hash code obtained after the processing in the first two digits of this remote sensing image slice. At this time, the code of each remote sensing image slice is composed of hash code + H3 code + Unix code. Use this code as rowkey to store all remote sensing image slices in Hbase. Step 6: While storing the remote sensing image slices, a B+ tree and a secondary index table are generated. The specific contents are as follows: The RowKey of the secondary index table is H3 index, and the corresponding Value is all remote sensing image slices related to this H3 index, which is used to store the Rowkey information of remote sensing images of the same area in MainTable; the B+ tree uses Unix timestamp as the key and spatial coordinates as the value; Step 7: KNN query; The specific implementation details of KNN query are as follows: the algorithm first converts the given longitude and latitude into an H3 hexagonal index of the specified resolution and obtains the center point of the index cell; then, the algorithm calculates all cells with an H3 distance within the specified radius k starting from the central H3 cell, and forms a set of these cell indexes to form a "filled disk" containing the central cell and all cells within the k range around it; next, these H3 indexes are used to query the corresponding row keys, i.e., rowkeys, in the secondary index table; for each valid H3 index, the algorithm extracts the relevant row keys from the secondary index table and stores them; finally, the algorithm queries the main table based on these row keys to obtain the final results; Step 8: Spatiotemporal range query; The spatiotemporal range query algorithm designed by this method includes a spatial query algorithm part, and its specific implementation details are shown in Algorithm 2: The spatiotemporal query algorithm filters and queries data through the two dimensions of time and space; first, the B+ tree index is used to filter out the relevant H3 indexes within a given time range; then a polygon is generated according to the specified geographic boundary, and the polygon is filled into a set of H3 indexes using the H3 library; the intersection of the time and space indexes is taken to obtain a set of H3 indexes that meet both the time and space conditions; these filtered H3 indexes are queried to further filter out the row keys that meet the time conditions, namely rowkeys; finally, the algorithm queries the main table based on these row keys to obtain the final result; Step 9: Model performance evaluation; In order to verify the performance of the remote sensing image indexing and storage model BSH3 based on the Uber H3 hexagonal discrete grid proposed in this paper, this method compares the proposed model without applying the consistent hash ring, B+ tree, and secondary index table methods, and the remote sensing image storage and indexing methods based on Uber H3 and Google S2.

2. The remote sensing image indexing and storage method based on the Uber H3 hexagonal grid according to claim 1, characterized in that: In step 2, the details are as follows: For remote sensing images with a resolution of 10m, select the UberH3 grid with a resolution of 7.

3. The remote sensing image indexing and storage method based on the Uber H3 hexagonal grid according to claim 1, characterized in that: In step 2, for the remote sensing image with a resolution of 20 meters, the Uber H3 grid with a resolution of 6 is selected.

4. The remote sensing image indexing and storage method based on the Uber H3 hexagonal grid according to claim 1, characterized in that: In step 2, for the remote sensing image with a resolution of 60 meters, the Uber H3 grid with a resolution of 5 is selected.

Images