Geospatial data distributed storage method for mapping geographic information
By dividing geospatial data into regular grid cells and generating common terrain data, the scalability bottleneck and inefficiency of traditional storage models are solved, achieving efficient and scalable data management and accurate data restoration.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- 江西省地质局第一地质大队
- Filing Date
- 2026-04-01
- Publication Date
- 2026-06-26
AI Technical Summary
Traditional centralized storage models are difficult to adapt to massive geospatial data, and suffer from problems such as storage capacity expansion bottlenecks, low data read and write efficiency, and weak disaster recovery capabilities, which cannot meet the needs of surveying and mapping operations for real-time data retrieval.
The target region is divided into regular grid units, the feature parameters of the terrain region are extracted and grouped into region types based on similarity, common terrain data are generated and stored in the same distributed node, and the target data is restored using individual geographic pointers.
It achieves efficient deduplication and compression of geospatial data, reduces hardware costs, improves query efficiency, ensures data integrity and consistency, and has good scalability and data restoration accuracy.
Smart Images
Figure CN122285667A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of data storage technology, and more specifically to a distributed storage method for geospatial data used in mapping geographic information. Background Technology
[0002] In the field of surveying and mapping geographic information, geospatial data, as a core basic resource, encompasses diverse data types such as high-precision remote sensing imagery, 3D terrain models, and vector geographic features, and its data storage volume is experiencing explosive growth. As surveying and mapping technology iterates towards higher precision and full coverage, the resolution of a single remote sensing image has reached sub-meter level, the data volume of 3D city modeling is often measured in TB levels, and the data from the national geographic census has exceeded PB levels. Traditional centralized storage models are no longer suitable for this data volume.
[0003] Traditional centralized storage relies on a single server or small storage cluster, which suffers from bottlenecks in storage capacity expansion. When faced with massive amounts of geospatial data, it is prone to exhausting storage resources and incurring high expansion costs. Furthermore, the data read / write efficiency of centralized architectures is limited by the performance of a single node. High-concurrency queries and batch data retrieval can easily lead to system lag and response delays, failing to meet the real-time data retrieval needs of surveying and mapping operations. In addition, centralized storage has weak disaster recovery capabilities; hardware failures can easily result in data loss, posing a serious threat to the security of surveying and mapping results. Summary of the Invention
[0004] The purpose of this invention is to provide a distributed storage method for geospatial data for mapping geographic information, and to solve the following technical problems.
[0005] The objective of this invention can be achieved through the following technical solutions: A distributed storage method for geospatial data used in mapping geographic information includes the following steps: Step S1: Obtain the original geographic data of the original region, divide the original region into several grid units with a preset grid size, and each grid unit is an independent terrain region; obtain the terrain feature parameters of each terrain region, and divide each terrain region into several region types according to the terrain feature parameters; Step S2: For any region type, compare and analyze the terrain feature parameters of all terrain regions within the region type, extract common geographic parameters, and generate common terrain data based on the common geographic parameters; obtain the region identifiers of each terrain region, and generate individual geographic pointers for each terrain region relative to the common terrain data; store the common terrain data of all terrain regions belonging to the same region type, as well as the region identifiers and individual geographic pointers of each terrain region, in the same distributed storage node; Step S3: When a user initiates a query request for a target location, the region identifier of the target location is obtained based on the location information of the target location and is recorded as the target region identifier. Based on the target region identifier, the target terrain region is determined among all distributed storage nodes, and the geographical data of the target terrain region is restored based on the common terrain data in the distributed storage nodes where the target terrain region is located and the unique geographical pointer of the target terrain region.
[0006] As a further aspect of the present invention, the process of obtaining the terrain feature parameters of the terrain region includes: acquiring laser point cloud data within a grid cell; obtaining terrain feature parameters of the terrain region within the grid cell based on the laser point cloud data; the terrain feature parameters include several terrain attribute values, including average elevation, slope standard deviation, surface roughness, elevation variation coefficient, and terrain undulation.
[0007] As a further aspect of the present invention: the process of dividing various terrain regions into several region types includes: The topographic feature parameters of each terrain region are constructed into terrain feature vectors. For any two terrain regions, the similarity between the terrain feature vectors of the two terrain regions is obtained. If the similarity is less than the preset similarity threshold, the two terrain regions are determined to be candidate regions of the same type. All terrain regions that are candidate regions of the same type to each other are determined to belong to the same region type.
[0008] As a further aspect of the present invention, the process of extracting common geographic parameters includes: A spatial reference system is established, and laser point cloud data of all terrain regions are acquired. The laser point cloud data of all terrain regions are uniformly registered to the spatial reference system and spatial alignment is performed at the point level. For any point, all terrain attribute values at the point are acquired sequentially, and the range of each terrain attribute value at the point is obtained to obtain the discrete index at the point. If the discrete index of the point exceeds a preset discrete threshold, the point is recorded as a common point, and each common point and its all terrain attribute values are acquired to form common geographic parameters.
[0009] As a further aspect of the present invention: the process of obtaining the discrete index at the network point includes: For any terrain attribute value at a network point, obtain the range of that terrain attribute value among all terrain attribute values at all network points; then obtain the average of the ranges of all terrain attribute values at the network point, and denote it as the discrete index at the point.
[0010] As a further aspect of the present invention, the process of extracting common geographic parameters also includes: If the discrete index of a point is less than or equal to a preset discrete threshold, the point is removed and its location is marked as a blank point. All terrain attribute values of common points adjacent to the blank point are obtained and marked as adjacent terrain attribute values. Based on all adjacent terrain attribute values, all terrain attribute values at the blank point are obtained using spatial interpolation.
[0011] As a further aspect of the present invention, the process of generating common terrain data includes fitting the common geographic parameters to a spatial surface based on spatial interpolation to obtain common terrain data.
[0012] As a further aspect of the present invention: the process of generating personalized geographic pointers for terrain regions relative to common terrain data includes: Acquire laser point cloud data of the terrain region, perform point-by-point difference calculation on the laser point cloud data and the common terrain data to generate a residual data matrix; identify all points in the residual data matrix with a value of 0, record them as indifferent points, remove all indifferent points to obtain sparse residual point cloud data; compress and encapsulate the residual point cloud data to obtain personalized geographic pointers.
[0013] The beneficial effects of this invention are: This invention provides a distributed storage method for geospatial data used in mapping geographic information. By dividing the target area into regular grid units, extracting the terrain feature parameters of each terrain region and grouping them into several region types based on similarity, a common terrain data is generated for each type and stored in the same distributed node. A pointer to the common data and individual residuals is generated for each region, thereby achieving efficient deduplication and compression of geospatial data while ensuring data integrity.
[0014] Compared to the traditional method of storing each grid cell independently, this invention fully utilizes the spatial correlation of geospatial data on terrain, avoiding the repeated storage of a large amount of repetitive terrain data within the same type of area, significantly reducing hardware costs and maintenance burden. Furthermore, because common data and all unique pointers for the same area type are centrally stored on the same node, querying any target area only requires accessing a single node to obtain complete data, avoiding network overhead and latency caused by cross-node data scheduling.
[0015] Furthermore, maintaining only one copy of the common data ensures the consistency of the terrain base across all areas of the same type, eliminating the complexity and inconsistency risks associated with synchronizing multiple copies of data. When new surveying data is added, only its type needs to be determined and individual residuals generated; there is no need to repeatedly store the common data, giving the system excellent scalability. This method also ensures that the reconstructed data is completely consistent with the original data through the encoding method and coordinate system transformation parameters in the individual pointers, achieving high-fidelity data restoration.
[0016] In summary, this invention fundamentally solves the technical problem of excessive geographic data storage while ensuring data accuracy and query efficiency, providing a practical solution for the efficient management of massive amounts of surveying and mapping geographic information. Attached Figure Description
[0017] The invention will now be further described with reference to the accompanying drawings.
[0018] Figure 1 This is a schematic diagram illustrating the steps of the distributed storage method for geospatial data used in mapping geographic information according to the present invention; Figure 2 This is a schematic diagram of the distributed storage method for geospatial data used in the mapping of geographic information according to the present invention. Detailed Implementation
[0019] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0020] Please see Figure 1 As shown, the present invention is a distributed storage method for geospatial data for mapping geographic information, comprising the following steps: Step S1: Obtain the original geographic data of the original region, divide the original region into several grid units with a preset grid size, and each grid unit is an independent terrain region; obtain the terrain feature parameters of each terrain region, and divide each terrain region into several region types according to the terrain feature parameters; In a preferred embodiment of the present invention, the process of setting the grid size includes: Collect several terrain region samples with different terrains, including mountains, plains and lakes; obtain the center point of the terrain region sample and obtain the edge contour line of the terrain region; select several reference points at equal intervals on the edge contour line, obtain the distance between each reference point and the center point, and obtain the average value of all distances, which is recorded as the average distance; obtain the average distance D of all terrain region samples, and set the grid size to D×D. In a preferred embodiment of the present invention, the process of obtaining the terrain feature parameters of the terrain region includes: acquiring laser point cloud data within a grid cell; obtaining the terrain feature parameters of the terrain region within the grid cell based on the laser point cloud data; the terrain feature parameters include several terrain attribute values, including average elevation, slope standard deviation, surface roughness, elevation variation coefficient, and terrain undulation. Specifically, the grid cell is divided into several sub-cells. The average elevation is obtained by calculating the arithmetic mean of the elevation values of all sub-cells within the grid cell. The slope standard deviation is used to characterize the dispersion of the surface slope of all sub-cells within the grid cell. The surface roughness is defined as the ratio of the surface area of all sub-cells within the grid cell to their projected area. The elevation variation coefficient is used to reflect the relative change in elevation of all sub-cells within the grid cell. The topographic relief is the elevation difference between the highest and lowest points of all sub-cells within the grid cell. In a preferred embodiment of the present invention, the process of dividing various terrain regions into several region types includes: The topographic feature parameters of each topographic region are constructed into topographic feature vectors. For any two topographic regions, the similarity between the topographic feature vectors of the two topographic regions is obtained. If the similarity is less than the preset similarity threshold, the two topographic regions are determined to be candidate regions of the same type. All topographic regions that are candidate regions of the same type to each other are determined to belong to the same region type. The process of obtaining the similarity between the terrain feature vectors of two terrain regions includes obtaining the cosine similarity between the terrain feature vectors of the two terrain regions; It is understandable that a higher cosine similarity indicates that the terrain features of the two regions are more similar; Step S2: For any region type, compare and analyze the terrain feature parameters of all terrain regions within the region type, extract common geographic parameters, and generate common terrain data based on the common geographic parameters; obtain the region identifiers of each terrain region, and generate individual geographic pointers for each terrain region relative to the common terrain data; store the common terrain data of all terrain regions belonging to the same region type, as well as the region identifiers and individual geographic pointers of each terrain region, in the same distributed storage node; In a preferred embodiment of the present invention, the process of extracting common geographic parameters includes: A spatial reference system is established, and laser point cloud data of all terrain regions are acquired. The laser point cloud data of all terrain regions are uniformly registered to the spatial reference system and spatial alignment at the point level is performed. For any point, all terrain attribute values at the point are acquired sequentially, and the range of each terrain attribute value at the point is obtained to obtain the discrete index at the point. If the discrete index of the point exceeds the preset discrete threshold, the point is recorded as a common point, and each common point and its all terrain attribute values are acquired to form common geographic parameters. The process of dot-level spatial alignment includes: Obtain the center points of all secondary units in the terrain region, denoted as secondary unit center points, and obtain the coordinates of each secondary unit center point in the spatial reference system, denoted as grid points; select one terrain region from all terrain regions as the reference terrain region, denote the grid coordinate matrix of the reference terrain region as the reference coordinate system, and denote all other terrain regions as regions to be registered; the grid coordinate matrix is composed of the planar coordinates of the center points of each secondary unit; Between each registration area and the reference terrain area, select at least three non-collinear pairs of corresponding control points. The corresponding control point pairs refer to two network points that are in the same location on the ground but have different coordinate values in different data sources. Based on the coordinates of the corresponding control point pairs, a system of coordinate transformation equations is established. The parameters of the coordinate transformation model are solved using the least squares method. The coordinate transformation model adopts an affine transformation model, and its expression is as follows: Where (x, y) are the coordinates of the grid points in the region to be registered, and (X', Y') are the transformed coordinates. , , b0, b1, and b2 are the transformation parameters to be solved. The solved transformation parameters are substituted into the affine transformation model, and the coordinates of all the grid points in the registration area are transformed one by one so that the grid points of all terrain areas correspond in spatial position under the reference coordinate system. The registration area after coordinate transformation is resampled so that the grids of all terrain areas have the same pixel size and spatial resolution. For non-integer coordinate positions that appear during the resampling process, the terrain attribute value of the position is calculated by bilinear interpolation. It is worth noting that at this time, at least 20 checkpoints are randomly selected, the coordinate difference between the reference area and the area to be registered is calculated for each checkpoint, and the root mean square error of all checkpoints is calculated. If the root mean square error is greater than a preset accuracy threshold, which is 1 / 2 of the original data spatial resolution, then the control point pair is reselected until the root mean square error is less than or equal to the accuracy threshold. The process of obtaining discrete indicators at network points includes: For any terrain attribute value at a network point, obtain the range of that terrain attribute value among all terrain attribute values at all network points; then obtain the average of the ranges of all terrain attribute values at the network point, and denote it as the discrete index at the point. The process of extracting common geographic parameters also includes: If the discrete index of a network point is less than or equal to a preset discrete threshold, then the network point is removed and its location is recorded as a blank point; all terrain attribute values of common network points adjacent to the blank point are obtained and recorded as adjacent terrain attribute values; based on all adjacent terrain attribute values and spatial interpolation, all terrain attribute values at the blank point are obtained. In a preferred embodiment of the present invention, the process of generating common terrain data includes fitting the common geographic parameters to a spatial surface based on spatial interpolation to obtain common terrain data. In a preferred embodiment of the present invention, the region identifier of the terrain region is generated based on the latitude and longitude coordinates of the center point of the terrain region; In a preferred embodiment of the present invention, the process of generating personalized geographic pointers for terrain regions relative to common terrain data includes: Acquire laser point cloud data of the terrain region, perform point-by-point difference calculation on the laser point cloud data and the common terrain data to generate a residual data matrix; identify all points in the residual data matrix with a value of 0, record them as indifferent points, remove all indifferent points to obtain sparse residual point cloud data; compress and encapsulate the residual point cloud data to obtain personalized geographic pointers; In a preferred embodiment of the present invention, the process of storing in a distributed storage node includes establishing a pointer index table for the distributed storage node that associates each region identifier with its individual geographic pointer. Step S3: When a user initiates a query request for a target location, the region identifier of the target location is obtained based on the location information of the target location and is recorded as the target region identifier; based on the target region identifier, the target terrain region is determined among all distributed storage nodes, and the geographical data of the target terrain region is restored based on the common terrain data in the distributed storage nodes where the target terrain region is located and the unique geographical pointer of the target terrain region. In a preferred embodiment of the present invention, the location information of the target location is the latitude and longitude coordinates of the target location; In a preferred embodiment of the present invention, the process of determining the target terrain region among all distributed storage nodes includes: Obtain the pointer index table of each distributed storage node, retrieve the Euclidean distance between the target area identifier and each area identifier in each pointer index table, select the area identifier with the closest Euclidean distance and record it as the target area identifier, and then record the distributed storage node where the target area identifier is stored as the distributed storage node where the target terrain area is located. In a preferred embodiment of the present invention, the process of restoring the geographic data of the target terrain region includes: According to the pointer index table, the personalized geographic pointer of the target area identifier is obtained from the distributed storage node and denoted as the target personalized geographic pointer; the target personalized geographic pointer is decoded to restore the sparse residual point cloud data and denoted as the target residual point cloud data; and the common terrain data in the distributed storage node is obtained and denoted as the target common terrain data. The target residual point cloud data is reconstructed into a residual raster matrix with the same size and spatial resolution as the target common terrain data. For the points not covered in the target residual point cloud data, their residual values are set to 0; the residual raster matrix and the target common terrain data are spatially registered, and the registered residual raster matrix and the target common terrain data are overlaid point by point to obtain the reconstructed complete geographic data.
[0021] The foregoing has provided a detailed description of one embodiment of the present invention, but this description is merely a preferred embodiment and should not be construed as limiting the scope of the invention. All equivalent variations and modifications made within the scope of the present invention should still fall within the scope of the present invention.
Claims
1. A distributed storage method for geospatial data used in mapping geographic information, characterized in that, Includes the following steps: Step S1: Obtain the original geographic data of the original region, divide the original region into several grid units with a preset grid size, and each grid unit is an independent terrain region; obtain the terrain feature parameters of each terrain region, and divide each terrain region into several region types according to the terrain feature parameters; Step S2: For any region type, compare and analyze the terrain feature parameters of all terrain regions within the region type, extract common geographic parameters, and generate common terrain data based on the common geographic parameters; obtain the regional identifiers of each terrain region, and generate individual geographic pointers for each terrain region relative to the common terrain data; Common terrain data of all terrain regions belonging to the same region type, as well as regional identifiers and individual geographic pointers of each terrain region, are stored in the same distributed storage node; Step S3: When a user initiates a query request for a target location, the region identifier of the target location is obtained based on the location information of the target location and is recorded as the target region identifier. Based on the target region identifier, the target terrain region is determined among all distributed storage nodes, and the geographical data of the target terrain region is restored based on the common terrain data in the distributed storage nodes where the target terrain region is located and the unique geographical pointer of the target terrain region.
2. The distributed storage method for geospatial data for mapping geographic information according to claim 1, characterized in that, In step S1, the process of obtaining the terrain feature parameters of the terrain region includes acquiring laser point cloud data within the grid cell, and obtaining the terrain feature parameters of the terrain region within the grid cell based on the laser point cloud data. The terrain feature parameters include several terrain attribute values, including average elevation, slope standard deviation, surface roughness, elevation variation coefficient, and terrain undulation.
3. The distributed storage method for geospatial data for mapping geographic information according to claim 1, characterized in that, In step S1, the process of dividing each terrain region into several region types includes: The topographic feature parameters of each terrain region are constructed into terrain feature vectors. For any two terrain regions, the similarity between the terrain feature vectors of the two terrain regions is obtained. If the similarity is less than the preset similarity threshold, the two terrain regions are determined to be candidate regions of the same type. All terrain regions that are candidate regions of the same type to each other are determined to belong to the same region type.
4. The distributed storage method for geospatial data for mapping geographic information according to claim 1, characterized in that, In step S2, the process of extracting common geographic parameters includes: A spatial reference system is established, and laser point cloud data of all terrain regions are acquired. The laser point cloud data of all terrain regions are uniformly registered to the spatial reference system and spatial alignment is performed at the point level. For any point, all terrain attribute values at the point are acquired sequentially, and the range of each terrain attribute value at the point is obtained to obtain the discrete index at the point. If the discrete index of the point exceeds a preset discrete threshold, the point is recorded as a common point, and each common point and its all terrain attribute values are acquired to form common geographic parameters.
5. The distributed storage method for geospatial data for mapping geographic information according to claim 4, characterized in that, In step S2, the process of obtaining the discrete indices at the network points includes: For any terrain attribute value at a network point, obtain the range of that terrain attribute value among all terrain attribute values at all network points; then obtain the average of the ranges of all terrain attribute values at the network point, and denote it as the discrete index at the point.
6. The distributed storage method for geospatial data for mapping geographic information according to claim 5, characterized in that, In step S2, the process of extracting common geographic parameters also includes: If the discrete index of a point is less than or equal to a preset discrete threshold, the point is removed and its location is marked as a blank point. All terrain attribute values of common points adjacent to the blank point are obtained and marked as adjacent terrain attribute values. Based on all adjacent terrain attribute values, all terrain attribute values at the blank point are obtained using spatial interpolation.
7. The distributed storage method for geospatial data for mapping geographic information according to claim 1, characterized in that, In step S2, the process of generating common terrain data includes fitting the common geographic parameters to a spatial surface based on spatial interpolation to obtain common terrain data.
8. The distributed storage method for geospatial data for mapping geographic information according to claim 1, characterized in that, In step S2, the process of generating individual geographic pointers for a terrain region relative to common terrain data includes: Acquire laser point cloud data of the terrain region, perform point-by-point difference calculation on the laser point cloud data and the common terrain data to generate a residual data matrix; identify all points in the residual data matrix with a value of 0, record them as indifferent points, remove all indifferent points to obtain sparse residual point cloud data; compress and encapsulate the residual point cloud data to obtain personalized geographic pointers.
9. The distributed storage method for geospatial data for mapping geographic information according to claim 1, characterized in that, In step S3, the process of determining the target terrain region among all distributed storage nodes includes: Obtain the pointer index table of each distributed storage node, retrieve the Euclidean distance between the target area identifier and each area identifier in each pointer index table, select the area identifier with the closest Euclidean distance and record it as the target area identifier, and then record the distributed storage node where the target area identifier is stored as the distributed storage node where the target terrain area is located.