Similarity query method for polygonal geographic data in spatial database, electronic equipment and readable storage medium

Abstract

The invention discloses a similarity query method for polygonal geographic data in a spatial database, and belongs to the technical field of spatial databases and geographic information. The method aims at solving the problem that in the prior art, when mass data are processed, the shape similarity searching efficiency is low. The core of the method comprises the following steps of: firstly, converting each polygon into a fixed-length digital signature with translation and zoom invariance through normalization, gridding and MinHash coding; secondly, the signature is divided into blocks (Band), an index is built, and the signature is stored in a distributed No-SQL database such as Apache Cassandra; and finally, during query, a candidate set is quickly filtered out by using a Band key of a query signature, accurate geometric similarity calculation and sorting are performed on the candidate set, and Top-K most similar polygons are returned. According to the method, efficient and accurate real-time shape similarity query in a large-scale spatial database is realized through a filtering-refining two-stage strategy, and the query efficiency and the system expandability are remarkably improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of spatial database and geographic information technology, and particularly relates to a similarity query method for polygon geographic data in a spatial database, an electronic device and a readable storage medium. BACKGROUND

[0002] In spatial database and geographic information system, many spatial data are polygon data, and it is an important problem to find similar polygons in the database given a polygon, which has wide application scenarios in smart city management, geographic information retrieval, spatial data mining, map matching and other fields. The existing technologies can be mainly divided into two categories: a method based on accurate geometric calculation and a method based on spatial index.

[0003] The method based on accurate geometric calculation mainly relies on the principle of computational geometry, and measures the similarity by directly calculating the geometric relationship between polygons. The core of this method is to accurately calculate the geometric feature parameters of the polygon, such as area, perimeter, convex hull, etc., and to evaluate the shape similarity by comparing these parameters. Among them, the geometric similarity calculation based on Jaccard coefficient is a typical representative, which quantifies the similarity degree by calculating the ratio of the intersection area to the union area of two polygons. Commonly used GEOS, JTS and other geometric calculation libraries provide stable and reliable polygon Boolean operation functions, which can efficiently process the intersection and union calculation of complex polygons, and provide a technical basis for accurate similarity comparison. The advantage of this method is that the calculation result is accurate, which can faithfully reflect the geometric similarity relationship between polygons.

[0004] The method based on spatial index focuses on accelerating the similarity query process by establishing an efficient spatial data structure. This method usually extracts and abstracts the features of the polygon first, and then uses a specially designed spatial index structure to organize these feature data. R-tree and its variants are the most commonly used spatial index structure, which can quickly exclude irrelevant candidate objects by organizing spatial objects into a hierarchical bounding box structure. In addition, quadtree, grid index and other structures are also widely used in the management and retrieval of spatial data.

[0005] Although the method based on accurate geometric calculation can provide accurate similarity measurement in theory, it faces serious computational efficiency bottleneck in practical engineering applications. The core operation of this method, the polygon intersection and union calculation, has high computational complexity, and when dealing with complex polygons with a large number of vertices, the calculation time increases nonlinearly. For a large city geographic data set with millions of building polygons, if the full-precision calculation method is used for similarity query, the computational overhead will become unbearable. More importantly, this method lacks an effective indexing mechanism to speed up the search process, resulting in the need for geometric calculation with most of the polygons in the database for each query, which makes the system response time increase sharply with the expansion of data size, and cannot meet the real-time requirements of practical applications.

[0006] On the other hand, the method based on spatial index performs well in spatial query efficiency, but its original design is mainly to support spatial query operations based on location relationship, such as range query, proximity query and spatial join, etc. The traditional R-tree, quadtree and other spatial index structures approximate the spatial object by the minimum bounding box. Although this representation method can efficiently filter out the candidate objects with similar spatial positions, it cannot effectively support similarity queries based on shape features. Two polygons may have completely different shape features, but have similar location and bounding box attributes in the spatial index, which makes it difficult to accurately capture the shape similarity relationship between polygons simply by relying on the spatial index. SUMMARY

[0007] The technical solution of the present application to solve the above technical problems is to provide a similarity query method for polygon geographic data in a spatial database, comprising the following steps:

[0008] Polygon encoding: converting each polygon geographic data in the spatial database into a fixed-length MinHash signature, which represents the shape feature of the polygon and has translational and scaling invariance;

[0009] Index storage: dividing the MinHash signature into multiple consecutive Bands, calculating a Band-level hash key for each Band, and storing each polygon according to all its (Band number, Band-level hash key) pairs in a distributed NoSQL database, so that each polygon has multiple index records in the database;

[0010] Top-K similarity query: in response to receiving a query polygon, the following operations are performed:

[0011] Converting the query polygon into a query MinHash signature;

[0012] splitting the query MinHash signature into Bands and computing Band-level hash keys according to the same rule;

[0013] retrieving, from the distributed NoSQL database, a preliminary candidate polygon set using the computed (Band number, Band-level hash key) pairs in a concurrent manner;

[0014] computing the exact geometric similarity between the query polygon and each polygon in the preliminary candidate polygon set;

[0015] returning, from the preliminary candidate polygon set, the K polygons with the highest similarity as the query result according to the exact geometric similarity.

[0016] Further, the polygon encoding step specifically includes:

[0017] Data preprocessing and normalization: computing the geometric center of the polygon and translating it to the origin of the coordinate system, and then scaling the polygon according to its minimum circumscribed rectangle to unify its scale;

[0018] Grid division and feature extraction: constructing a fixed-size square grid centered at the origin to cover the normalized polygon, and computing the ratio of the intersection area of each grid cell and the polygon to the total area of the cell as the fill rate of the cell;

[0019] Feature vector construction: arranging the fill rates of all grid cells in a predetermined order to form a high-dimensional feature vector;

[0020] MinHash signature generation: based on the rejection sampling method, using multiple hash functions to process the feature vector to generate a fixed-length MinHash signature composed of multiple integer hash values.

[0021] Further, in the MinHash signature generation sub-step, for each hash function, a MinHash value is generated through the following process:

[0022] a. Initialize the attempt counter;

[0023] b. Randomly generate a grid cell index and a random number in the range [0, 1);

[0024] c. Increase the value of the attempt counter;

[0025] d. Determine whether the random number is less than the fill rate of the feature vector corresponding to the grid cell index;

[0026] e. If yes, return the value of the current attempt counter as the MinHash value corresponding to the hash function; if no, repeat steps b to e.

[0027] Further, in the index storage step, the distributed NoSQL database is Apache Cassandra, which stores the index table structure with a composite partition key of (Band number, Band level hash key) and a cluster key of polygon ID.

[0028] Further, in the Top-K similarity query step, the exact geometric similarity is a geometric similarity based on Jaccard coefficient, which is obtained by calculating the ratio of the intersection area and the union area of two polygons.

[0029] Further, in the Top-K similarity query step, before calculating the exact geometric similarity, it further includes the step of retrieving the corresponding polygon geometric information from the metadata table storing the original geometric data or complete MinHash signature according to the polygon ID obtained from the index.

[0030] To solve the above technical problems, the present application further provides an electronic device, comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor executes the program to realize the steps of the method described above.

[0031] To solve the above technical problems, the present application further provides a computer readable storage medium having a computer program stored thereon, wherein the program is executed by a processor to realize the steps of the method described above.

[0032] Compared with the prior art, the similarity query method for polygon geographic data in a spatial database provided by the present application has the following remarkable beneficial effects:

[0033] 1. A leap in query efficiency:

[0034] The present application abstracts complex geometric shapes into compact MinHash signatures and uses a Band-based distributed index structure to reduce the complexity of similarity queries from linear complexity to O (log n). In practical applications, for a massive database containing millions of polygons, the initial filtering stage of the query can be completed in milliseconds, quickly excluding up to 85%-95% of irrelevant data. This "coarse filtering" mechanism fundamentally solves the efficiency bottleneck caused by computationally intensive geometric operations in traditional methods, making real-time interactive queries on large-scale data sets possible.

[0035] 2. High precision results at high efficiency:

[0036] The "filter-refine" two-stage strategy adopted by the present application does not sacrifice accuracy while pursuing efficiency. First, the MinHash signature itself is a mature probability model for estimating Jaccard similarity, which can guarantee the theoretical lower bound of candidate set recall rate. Second, and most importantly, the present application only uses approximate signatures in the filtering stage, and performs accurate geometric similarity calculation on the significantly reduced candidate set (usually only 5%-15% of the full data) in the final sorting stage. This design ensures that the final returned Top-K results are close to the results of brute-force accurate calculation on the full database, effectively avoiding the precision loss that may be caused by pure approximation methods, and achieving the best balance between efficiency and accuracy.

[0037] 3. Excellent system scalability:

[0038] The technical solution of the present application is deeply compatible with distributed architecture and has excellent horizontal expansion capability. First, the MinHash signature has a fixed length and is independent of the complexity (number of vertices) of the original polygon, which facilitates the system to uniformly and standardize process massive heterogeneous polygon data. Second, the index strategy based on Band is naturally suitable for the data model of ApacheCassandra and other distributed No-SQL databases. When the data size continues to grow, the system only needs to simply increase the cluster nodes to linearly improve its data storage capacity, write throughput and concurrent query processing capability, and can easily cope with the exponential growth of future data volume.

[0039] 4. Enable real-time interactive applications:

[0040] In summary, the present application first makes it possible to perform second-level response shape similarity Top-K query on large-scale spatial databases at the city level. After the user submits a complex query polygon, the system can return the most similar results within a few seconds, greatly improving the interactive experience and decision-making efficiency of geographic spatial analysis, smart city management, map retrieval, and land tracing, and has great social and economic value.

[0041] 5. Technical versatility and robustness:

[0042] The feature extraction and encoding method in the present application has translational and scaling invariance, and can effectively process polygon data of various sources and scales in reality. This method does not depend on specific business domain knowledge and can be widely applied to various shape-based spatial data retrieval scenarios, showing good versatility and robustness. BRIEF DESCRIPTION OF DRAWINGS

[0043] In order to make the technical solutions of the embodiments of the present application or the prior art clearer, the accompanying drawings needed in the embodiments or prior art description will be briefly introduced. Obviously, the accompanying drawings in the following description only constitute some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained according to the structures shown in the drawings without creative labor.

[0044] Figure 1 The step flow chart of the similarity query method for polygon geographic data in a spatial database according to the present application. DETAILED DESCRIPTION

[0045] The present application provides a similarity query method for polygon geographic data in a spatial database, an electronic device and a readable storage medium, aiming at solving the problem of low shape similarity search efficiency in processing large-scale geographic information polygon data in the prior art, and providing an innovative method capable of efficiently and accurately performing shape similarity search.

[0046] The similarity query method for polygon geographic data in a spatial database according to the present application will be described in specific embodiments as follows:

[0047] Embodiment 1

[0048] A similarity query method for polygon geographic data in a spatial database, as shown in Figure 1 , comprising the following steps:

[0049] S10: Polygon encoding: converting each polygon geographic data in the spatial database into a fixed-length MinHash signature, which represents the shape feature of the polygon and has translational and scaling invariance;

[0050] S20: Index storage: dividing the MinHash signature into a plurality of consecutive Bands, calculating a Band-level hash key for each Band, and storing each polygon according to all its (Band number, Band-level hash key) pairs into a distributed NoSQL database, so that each polygon has multiple index records in the database;

[0051] S30: Top-K similarity query: in response to receiving a query polygon, the following operations are performed:

[0052] Converting the query polygon into a query MinHash signature;

[0053] Dividing the query MinHash signature into Bands according to the same rule and calculating the Band-level hash key;

[0054] retrieve, from the distributed NoSQL database, concurrently, a preliminary candidate polygon set using the calculated (Band number, Band level hash key) pair;

[0055] calculate the exact geometric similarity between the query polygon and each polygon in the preliminary candidate polygon set;

[0056] return, from the preliminary candidate polygon set, the K polygons with the highest similarity as the query result according to the exact geometric similarity.

[0057] Further, the polygon encoding step specifically comprises:

[0058] S11: data preprocessing and normalization: calculate the geometric center of the polygon and translate it to the origin of the coordinate system, and then scale the polygon according to the minimum circumscribed rectangle of the polygon to unify the scale;

[0059] S12: grid division and feature extraction: construct a fixed-size square grid covering the normalized polygon with the origin as the center, calculate the ratio of the area of the intersection region of each grid cell and the polygon to the total area of the cell as the filling rate of the cell;

[0060] S13: feature vector construction: arrange the filling rates of all grid cells in a predetermined order to form a high-dimensional feature vector;

[0061] S14: MinHash signature generation: based on the rejection sampling method, use multiple hash functions to process the feature vector to generate the fixed-length MinHash signature composed of multiple integer hash values.

[0062] Further, in the MinHash signature generation sub-step, for each hash function, a MinHash value is generated through the following process:

[0063] a. initialize the attempt counter;

[0064] b. randomly generate a grid cell index and a random number in the range [0, 1);

[0065] c. increase the value of the attempt counter;

[0066] d. determine whether the random number is less than the filling rate of the feature vector corresponding to the grid cell index;

[0067] e. if yes, return the current value of the attempt counter as the MinHash value corresponding to the hash function; if no, repeat steps b to e.

[0068] Further, in the index storing step, the distributed NoSQL database is Apache Cassandra, which stores the index table structure with (Band number, Band level hash key) as the composite partition key and polygon ID as the clustering key.

[0069] Further, in the Top-K similarity query step, the exact geometric similarity is the geometric similarity based on Jaccard coefficient, which is obtained by calculating the ratio of the intersection area and the union area of two polygons.

[0070] Further, in the Top-K similarity query step, before calculating the exact geometric similarity, the step of retrieving the corresponding polygon geometric information from the metadata table storing the original geometric data or complete MinHash signature according to the polygon ID obtained from the index is further included.

[0071] Embodiment 2:

[0072] A similarity query method for polygon geographic data in a spatial database, assuming that in the geographic information database of a certain large city, about 500,000 residential community contour polygon data are stored. Each community polygon is defined by a series of latitude and longitude coordinates of its boundary. The object of the present application is to quickly find the Top-10 existing communities most similar to the shape of an arbitrary-shaped community contour (for example, a newly planned community plan) from the 500,000 communities when a user inputs the community contour. The method includes the following steps:

[0073] S1, polygon encoding:

[0074] S1.1 Data preprocessing and normalization

[0075] Read all vertex coordinates of a community polygon.

[0076] S1.2 Calculate the centroid: calculate the geometric center of the polygon.

[0077] Translation: translate all vertex coordinates of the polygon so that the centroid is at the coordinate origin (0, 0).

[0078] Scaling: calculate the minimum circumscribed rectangle of the polygon to obtain its width W and height H. Let S = max(W, H). Divide all vertex coordinates of the polygon by S to scale the polygon to a unit square (with a side length of 1). This step ensures the translation and scaling invariance of the polygon.

[0079] S2, grid division and feature extraction:

[0080] Define a 20*20 grid covering the range from (-0.5,-0.5) to (0.5,0.5). The entire normalized polygon will lie within this grid.

[0081] For each cell Cell(i,j) in the grid, compute the intersection area of the cell and the normalized polygon.

[0082] Compute the fill ratio of the cell: FillRatio(i,j) = IntersectionArea / CellArea. The area of each cell is (1 / 20)*(1 / 20) = 0.0025.

[0083] Arrange all the fill ratios of the 400 (20x20) grid cells in row-major order to obtain a feature vector V of length 400.

[0084] S3: Generate MinHash signature

[0085] Use 20 MinHash values as the signature of the polygon.

[0086] Prepare 20 different hash functions H1, H2,..., H 20 . This can be achieved by using a hash function (e.g. MurmurHash) and setting 20 different seeds.

[0087] For each hash function H i , perform the following "reject sampling" process to generate a MinHash value:

[0088] a. Initialize an attempt counter k = 0.

[0089] b. Randomly generate an integer in the range [0, 399] as the grid index idx, and generate a random number r in the range [0, 1).

[0090] c. Increase the counter k by 1.

[0091] d. Check if the random number r is less than the fill ratio V[idx] corresponding to this index in the feature vector. If r < V[idx], accept this sampling and return the current k value as the MinHash value M i . Otherwise, repeat steps b-d until the condition is met.

[0092] Repeat the above process for the 20 hash functions to obtain a MinHash signature [M1, M2,..., M 20 ] containing 20 integers.

[0093] Cassandra database Schema design and storage:

[0094] To store and index the above generated data, two tables are designed in Cassandra as follows:

[0095] 1. Neighborhood Band Index Table (neighborhood_bands)

[0096] This table is the core to achieve fast filtering.

[0097] sql

[0098] CREATE TABLE neighborhood_bands(

[0099] band_id int,--Band number (0 to 3)

[0100] band_hash bigint,--Band hash key

[0101] neighborhood_id uuid,--Neighborhood unique identifier

[0102] full_minhash blob,--Store full 20 MinHash values (serialized)

[0103] PRIMARY KEY ((band_id, band_hash), neighborhood_id)

[0104] ) WITH CLUSTERING ORDER BY (neighborhood_id ASC);

[0105] 2. Neighborhood Metadata Table (neighborhood_metadata)

[0106] This table stores raw or refined data for calculation.

[0107] sql

[0108] CREATE TABLE neighborhood_metadata(

[0109] neighborhood_id uuid PRIMARY KEY,--Neighborhood unique identifier

[0110] wkt_text text,--WKT (Well-Known Text) format data of neighborhood outline

[0111] name text,--Neighborhood name

[0112] -- other business attributes... ) ;

[0114] 3. Data Ingestion Process

[0115] For a cell polygon, the ingestion process is as follows:

[0116] Execute the encoding step above to generate a signature of 20 MinHash values [M1, M2,..., M20]. 20 ].

[0117] Divide the 20 signatures into 4 Bands (Bands), each containing 5 MinHash values.

[0118] Band 0: [M1, M2, M3, M4, M5]

[0119] Band 1: [M6, M7, M8, M9, M10] 10 ].

[0120] Band 2: [M11, M12, M13, M14, M15] 11 , 12 , 13 , 14 , 15 ].

[0121] Band 3: [M16, M17, M18, M19, M20] 16 , 17 , 18 , 19 , 20 ].

[0122] Hash the 5 values contained in each Band to compute a Band hash key (e.g., hash(tuple(M1, M2, M3, M4, M5))).

[0123] Insert 4 records into the neighborhood_bands table, each record corresponding to a (band_id, band_hash) partition key and containing the cell's ID and the full MinHash signature.

[0124] Insert 1 record into the neighborhood_metadata table containing the cell ID and the WKT-formatted data of its original footprint.

[0125] Query Process

[0126] When a user submits a query cell footprint Q:

[0127] 1. Generate query signature: Perform the exact same encoding steps on Q as in the ingestion, resulting in a query signature Q_sig of 20 MinHash values.

[0128] 2. Compute query Bands: Divide Q_sig into 4 Bands as well, and compute the hash key for each Band.

[0129] 3. Parallel Band queries: Execute 4 queries concurrently to Cassandra:

[0130] sql

[0131] -- Example: Candidates for query Band 0

[0132] SELECT neighborhood_id,full_minhashFROM neighborhood_bands

[0133] WHERE band_id=0AND band_hash=<Q_Band0_Hash>;

[0134] The database will process these queries in parallel on different nodes, and return all matching candidate neighborhood IDs.

[0135] 4. Refine and rank:

[0136] a. From the returned candidate records, we can get the IDs of the candidate neighborhoods.

[0137] b. From the neighborhood_metadata table, we can batch fetch the original WKT polygon data for these candidate neighborhoods by their IDs.

[0138] c. Using a geometry library such as GEOS, we can compute the exact geometric Jaccard similarity (intersection area / union area) between the query neighborhood Q and each candidate neighborhood one by one.

[0139] d. Rank all candidate neighborhoods by similarity in descending order, and return the top-10 as the final result.

[0140] Embodiment 3:

[0141] An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, the processor implementing the steps of the method as claimed in embodiment 2 when executing the program.

[0142] Embodiment 4:

[0143] A computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements the steps of the method as claimed in embodiment 2.

[0144] The above description is merely that of the preferred embodiments of the application, but the protection scope of the application is not limited thereto, and any changes or substitutions easily conceived by those skilled in the art within the technical scope disclosed by the application should be encompassed within the protection scope of the application. Therefore, the protection scope of the application should be subject to the protection scope of the claims.

Claims

1. A method for similarity query of polygonal geographic data in a spatial database, characterized in that, The method comprises the following steps: polygon encoding: converting each polygon geographic data in the spatial database into a fixed-length MinHash signature, which represents the shape characteristics of the polygon and is invariant to translation and scaling; index storage: dividing the MinHash signature into multiple consecutive Bands, calculating a Band-level hash key for each Band, and storing each polygon in the distributed NoSQL database according to all its (Band number, Band-level hash key) pairs, so that each polygon has multiple index records in the database; Top-K similarity query: in response to receiving a query polygon, the following operations are performed: convert the query polygon into a query MinHash signature; divide the query MinHash signature into Bands according to the same rule and calculate the Band-level hash key; use the calculated (Band number, Band-level hash key) pair to retrieve the preliminary candidate polygon set from the distributed NoSQL database concurrently; calculate the exact geometric similarity between the query polygon and each polygon in the preliminary candidate polygon set; according to the exact geometric similarity, return the K polygons with the highest similarity from the preliminary candidate polygon set as the query result.

2. The method of claim 1, wherein, The polygon encoding step specifically comprises: data preprocessing and normalization: calculate the geometric center of the polygon and translate it to the origin of the coordinate system, then scale the polygon according to its minimum bounding rectangle to unify its scale; grid division and feature extraction: construct a fixed-size square grid centered at the origin to cover the normalized polygon, calculate the area ratio of the intersection area between each grid cell and the polygon to the total area of the cell as the filling rate of the cell; feature vector construction: arrange the filling rates of all grid cells in a predetermined order to form a high-dimensional feature vector; MinHash signature generation: based on the rejection sampling method, use multiple hash functions to process the feature vector to generate a fixed-length MinHash signature composed of multiple integer hash values.

3. The method of claim 2, wherein, In the MinHash signature generation sub-step, for each hash function, a MinHash value is generated through the following process: a. initialize the attempt counter; b. randomly generate a grid cell index and a random number in the range [0, 1); c. increase the value of the attempt counter; d. determine whether the random number is less than the filling rate of the feature vector corresponding to the grid cell index; e. if yes, return the current value of the attempt counter as the MinHash value corresponding to the hash function; if not, repeat steps b to e.

4. The method of claim 1, wherein, In the index storage step, the distributed NoSQL database is Apache Cassandra, and the table structure for storing indexes is (Band number, Band-level hash key) as the composite partition key and polygon ID as the clustering key.

5. The method of claim 1, wherein, In the Top-K similarity query step, the exact geometric similarity is a geometric similarity based on Jaccard coefficient, which is obtained by calculating the ratio of the intersection area and the union area of two polygons.

6. The method of claim 1, wherein, In the Top-K similarity query step, before calculating the exact geometric similarity, it further includes the step of retrieving the corresponding polygon geometric information from the metadata table storing the original geometric data or complete MinHash signature according to the polygon ID obtained from the index.

7. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor implements the steps of the method of any one of claims 1 to 6 when executing the program.

8. A computer-readable storage medium having stored thereon a computer program, characterized in that, The program is executed by the processor to implement the steps of the method of any one of claims 1 to 6.

Images