A method and system for rapid retrieval and display of multi-level administrative division elements

Abstract

The present invention discloses a method and system for rapid retrieval and display of multi-level administrative division elements, which belongs to the field of geographic information technology, and includes the following steps: pre-processing the graphic data through topological sparseness technology, data compression technology and spatial index technology to achieve high-quality compression and high-speed access of the graphic data; adopting a flat storage structure design to optimize the administrative division attribute data, create an attribute index, and achieve one-step retrieval hit; creating a full-text retrieval table through FTS natural language-based search technology, using a simple library for Chinese word segmentation to support full-text retrieval based on the name of the administrative division; implementing cache optimization for attribute retrieval through a secondary cache strategy combining memory with a Redis key-value storage system to improve concurrent performance. The present invention can effectively reduce the technical cost and hardware resource requirements of the software system that realizes this capability, meet the needs of business users to build personalized, low-cost applications based on their own data, and run efficiently.

Description

Technical Field

[0001] The invention belongs to the field of geographic information technology, and specifically relates to a method and system for quickly searching and displaying multi-level administrative division elements. Background Art

[0002] With the rapid development and popularization of geographic information technology, the retrieval and display of multi-level administrative division elements have become increasingly important and have become an important supporting technology in the fields of geographic information systems (GIS), online map services, etc. In geographic information systems (GIS) and online map services, users can quickly obtain detailed administrative division information or locate the administrative division to which the current location belongs, which is crucial to meeting the needs of different terminals and scenarios, such as navigation, urban planning, disaster response and other applications on mobile devices.

[0003] At present, although the commercial geographic information system GIS products on the market at home and abroad can provide the retrieval and display functions of multi-level administrative division elements, they are all provided in the form of full-featured server-side large-scale software, with high software costs and hardware resource requirements; this makes it difficult for small enterprises and individual users to afford. In addition, large IT Internet companies also have corresponding products and services, but the products and services of these large IT Internet companies are all based on self-collected data and self-operated infrastructure, which makes it difficult for these products and services to meet the special needs of other business application parties for privatization, self-construction and data personalization.

[0004] Therefore, it is of great practical value to develop a method and system for rapid retrieval and display of multi-level administrative division elements that can effectively reduce software costs and hardware resource requirements while meeting the privatization, self-construction and data personalization needs of business applications. Summary of the invention

[0005] The present invention aims at the above-mentioned problems, makes up for the deficiencies of the prior art, and provides a method and system for rapid retrieval and display of multi-level administrative division elements. The present invention can not only effectively reduce software costs and hardware resource requirements, but also meet the privatization, self-construction and data personalization requirements of business application parties; and has high operating performance. Specifically, it proposes a solution that can meet the needs of business application parties based on their own data, can be updated by themselves, is controllable and safe, and has good efficiency.

[0006] To achieve the above purpose, the present invention adopts the following technical solution.

[0007] In a first aspect, the present invention provides a method for quickly searching and displaying multi-level administrative division elements, comprising the following steps:

[0008] S1. Preprocess the graphic data through topological sparseness technology, data compression technology and spatial indexing technology to achieve high-quality compression and high-speed access of graphic data;

[0009] S2. Adopt a flat storage structure design, optimize the administrative division attribute data, create an attribute index, and achieve one-step search hits;

[0010] S3. Create a full-text search table through FTS natural language-based search technology, and use the simple library for Chinese word segmentation to support full-text search based on administrative division names;

[0011] S4. Through the secondary cache strategy that combines memory and Redis key-value storage system, cache optimization is implemented for attribute retrieval to improve concurrency performance.

[0012] Furthermore, the topological thinning technology in step S1 specifically includes: unifying the topological relationship of the surface graphics of the administrative divisions to form arcs and nodes, and achieving the topological consistency of the restored administrative division surface by thinning the arcs and reconstructing the topology.

[0013] Furthermore, the data compression technology in step S1 specifically includes: calculating the outer enclosing rectangle of all points in the point coordinate array of each graphic, taking the lower left corner point of the rectangle as the origin coordinate {0, 0} to establish a Gaussian rectangular coordinate system, forming a local coordinate system for the graphic, and subtracting the coordinates of each point in the point coordinate array of the graphic from the coordinates of the origin to obtain a relative deviation value {dx, dy}, and converting this relative deviation value into an INT2 integer data type for storage.

[0014] Furthermore, the spatial indexing technology in step S1 specifically includes: storing the graphic data preprocessed by topological rarefaction technology and data compression technology in a data file, and establishing an RTree index for the graphic based on the spatial indexing principle, so as to achieve fast retrieval hits based on spatial location.

[0015] Furthermore, the step S2 specifically includes: first, standardizing the administrative division data structure, processing the attribute data and pre-processed graphic data into tree data with a standard structure according to the administrative division level; then, adopting a flat storage structure design, the administrative division code, name, center point, boundary range data and the current level graphic data in the target format, all parent attributes and graphic information, and lower-level graphics and attribute information are structured and stored together; finally, the above-mentioned structured attribute data is stored in a data file, and an attribute index is created based on the characteristics of the attribute index.

[0016] Furthermore, the step S3 also includes: acquiring target data according to the association between the full-text search data and the entity data.

[0017] Furthermore, the step S4 specifically includes: first, when retrieving the attribute conditions of the administrative division code, name, and ID, define a cache manager to manage the distributed cache; when retrieving through attributes, read, store, and expiration management of the cache is performed; then, the cache key-value pair storage structure is used to store the administrative division attribute conditions as keys and the query results as values; finally, the secondary cache is implemented by combining memory with the Redis key-value storage system, and the hot data is cached in the memory according to the hot ranking of the most recently accessed data, and the remaining data is cached in the Redis key-value storage system.

[0018] In a second aspect, the present invention also provides a fast retrieval and display system for multi-level administrative division elements, including a graphic data optimization module, an attribute data optimization module, a full-text retrieval module, and a cache optimization module;

[0019] The graphic data optimization module is used to pre-process the graphic data through topological thinning technology, data compression technology and spatial indexing technology to achieve high-quality compression and high-speed access to the graphic data;

[0020] The attribute data optimization module is used to optimize the administrative division attribute data by adopting a flat storage structure design, create an attribute index, and achieve one-step retrieval hit;

[0021] The full-text search module is used to create a full-text search table through the natural language-based search technology of FTS, and use the simple library to perform Chinese word segmentation to support full-text search based on administrative division names;

[0022] The cache optimization module is used to implement cache optimization for attribute retrieval and improve concurrency performance through a secondary cache strategy combining memory with a Redis key-value storage system.

[0023] Beneficial effects of the present invention:

[0024] The present invention provides a method and system for rapid retrieval and display of multi-level administrative division elements, which (1) realizes optimized processing and management of graphic data; (2) realizes optimized processing and management of attribute data; (3) realizes full-text retrieval based on administrative division names; and (4) realizes cache optimization of attribute retrieval to improve concurrency performance.

[0025] In summary, by using the rapid retrieval and display method and system of the present invention, it is possible to effectively reduce business costs and hardware resource requirements, and to meet the privatization, self-construction and data personalization needs of business application parties; and the operating performance is efficient, providing a solution that can meet the needs of business application parties based on their own data, controllable frequency, controllable security, and good efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the description of the embodiments of the present application will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0027] Figure 1 The present invention is a flowchart of a method for rapid retrieval and display of multi-level administrative division elements.

[0028] Figure 2 The present invention is a schematic diagram of the structure of a rapid retrieval and display system for multi-level administrative division elements.

[0029] Figure 3 The present invention is a flowchart of a graphic data optimization module of a rapid retrieval and display system for multi-level administrative division elements.

[0030] Figure 4 The present invention is a flowchart of an attribute data optimization module of a rapid retrieval and display system for multi-level administrative division elements.

[0031] Figure 5 The present invention is a flowchart of a full-text search module of a rapid search and display system for multi-level administrative division elements.

[0032] Figure 6 This is a flow chart of a cache optimization module of a rapid retrieval and display system for multi-level administrative division elements of the present invention.

[0033] Markings in the figure: 100 is a fast retrieval and display system for multi-level administrative division elements, 101 is a graphic data optimization module, 102 is an attribute data optimization module, 103 is a full-text retrieval module, and 104 is a cache optimization module. DETAILED DESCRIPTION

[0034] In order to make the technical problems, technical solutions and beneficial effects solved by the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and specific implementation methods. It should be understood that the specific implementation methods described herein are only used to explain the present invention and are not used to limit the present invention.

[0035] like Figure 1 As shown, a method for quickly retrieving and displaying multi-level administrative division elements provided by an embodiment of the present invention includes the following steps:

[0036] S1. Preprocess the graphic data through topological thinning technology, data compression technology and spatial indexing technology to achieve high-quality compression and high-speed access to graphic data.

[0037] Specifically, the topological thinning technology is different from the Douglas Peukel algorithm commonly used in the geographic information system (GIS). The topological thinning technology takes into account the topological relationship between graphics. The topological thinning technology specifically includes: unifying the topological relationship of the surface graphics of administrative divisions to form arcs and nodes, and achieving the topological consistency of the restored administrative division surface through thinning of arcs and reconstruction of topology.

[0038] Specifically, the data compression technology includes: calculating the outer enclosing rectangle of all points in the point coordinate array of each graphic, taking the lower left corner point of the rectangle as the origin coordinate {0, 0} to establish a Gaussian rectangular coordinate system, forming a local coordinate system for the graphic, and subtracting the coordinates of each point in the point coordinate array of the graphic from the origin coordinate to obtain a relative deviation value {dx, dy}, and converting this relative deviation value into an INT2 integer data type for storage.

[0039] Since each graphic data is composed of a point coordinate array, each point coordinate has two double type absolute coordinate values ​​of x and y, and the storage space is 16 bytes, it can be converted into INT2 integer data type through the above-mentioned data compression technology, which only occupies 4 bytes of storage space to meet the requirements; through the above-mentioned data compression technology, 75% of the storage space can be saved, and low storage can improve the query retrieval speed.

[0040] Taking a graphic with 1000 point coordinates as an example, the storage space is 16 bytes * 1000 = 16000 bytes. After this operation, only one origin absolute coordinate value {originX, originY} and 999 relative coordinate values ​​need to be stored, and the space occupied is 16 bytes + 999 * 4 bytes = 4,012 bytes, saving a total of 11,988 bytes.

[0041] Specifically, the spatial indexing technology includes: storing the graphic data preprocessed by topological rarefaction technology and data compression technology in a data file, and establishing an RTree index for the graphic based on the spatial indexing principle to achieve fast retrieval hits based on spatial location.

[0042] S2. Adopt a flat storage structure design, optimize the administrative division attribute data, create an attribute index, and achieve one-step retrieval hit; this can directly return the target data without the need for secondary data processing.

[0043] Specifically, the administrative division data structure is first standardized, and the attribute data and pre-processed graphic data are processed into tree data with a standard structure according to the administrative division level; then a flat storage structure design is adopted to store the administrative division code, name, center point, boundary range data together with the current level graphic data in the target format, all parent attributes and graphic information, and lower-level graphics and attribute information in a structured manner; when performing spatial location or attribute information retrieval, the target can be directly hit and the required administrative division element data can be obtained in one step; finally, the above-mentioned structured attribute data is stored in a data file, and an attribute index is created based on the characteristics of the attribute index, which further improves the retrieval efficiency of the attribute data.

[0044] The data information of a single administrative division includes a code, a name, a center point, a boundary range, and a graphic. The format of the graphic is processed according to the data compression technology in step S1, and has the characteristic of saving storage. An example of realizing fast retrieval of attribute data by using the attribute index described in step S2 is as follows:

[0045] {3613, Ganzhou City, [117.345262, 31.434206], [113.111111, 28.111111, 117.111111, 35.111111], 16-byte origin + N*4-byte graphic point array}, In the scenario of searching for a region, when a coordinate is given for searching, to hit a region, it is often necessary to obtain the superior and subordinate information of this region at the same time; for example: Ganzhou City’s superior Jiangxi Province and Ganzhou City’s subordinate counties, through one query and overall return, the number of communications is small, and the application is more convenient. Therefore, for a single administrative division, based on its own data information, the information of its superior and subordinate is merged. The superior and subordinate use the same data structure as above, the superior has 1 information, and the subordinate has N information (depending on the number of subordinate regions). The above 1 (local region) + 1 (superior region) + N (subordinate region) is stored as a structural unit to achieve the above purpose. For the convenience of retrieval, this structure creates a BTree attribute index for the code and name attributes of the local area to speed up data retrieval.

[0046] S3. Create a full-text search table through FTS's natural language-based search technology, use the simple library for Chinese word segmentation to support full-text search based on administrative division names, and use it to implement fast fuzzy matching search based on administrative division names; then obtain the target data based on the association between the full-text search data and the entity data.

[0047] Specifically, based on the association between full-text retrieval data and entity data, through precise or fuzzy retrieval of administrative division codes or names, the target data obtained includes: 1 or more fully matched or fuzzily matched administrative division data, each administrative division data contains 1 data information at the current level, 1 data information at the upper level and N data information at the lower level.

[0048] S4. Through the secondary cache strategy that combines memory with the Redis key-value storage system, cache optimization is implemented for attribute retrieval to improve concurrency performance, improve retrieval performance and reduce the number of access times for data files.

[0049] Specifically, first, when retrieving the attribute conditions of administrative division code, name, and ID, a cache manager is defined to manage the distributed cache; when retrieving through attributes, the cache is read, stored, and expired. When the data does not exist in the cache, the data file is accessed and the cache is updated; then the cached key-value pair storage structure is used, with the administrative division attribute conditions as the key and the query results as the value for storage; finally, the secondary cache is implemented by combining memory with the Redis key-value storage system, and the hot data is cached in the memory according to the hot ranking of the most recently accessed data, and the rest of the data is cached in the Redis key-value storage system; the cache is preheated in conjunction with the hot data, which effectively avoids cache penetration, breakdown, avalanche and other problems.

[0050] Specifically, the execution process of the secondary cache is implemented by combining memory with the Redis key-value storage system: a Key-Value primary cache is established in the memory, and a Key-Value secondary cache is established based on the Redis key-value storage system. In addition, an LRU (Least Recently Used) least recently used linked list mechanism is established in the memory. Each request forms a cumulative request count in the LRU, and the LRU is automatically sorted according to the access frequency to form a hot spot ranking mechanism; according to the set update time interval, the timed hot spot ranking is updated from the secondary cache to the primary cache. When the secondary cache does not exist, the data is read from the database.

[0051] Combination Figures 2 to 6 As shown, a fast retrieval and display system 100 for multi-level administrative division elements provided by an embodiment of the present invention includes a graphic data optimization module 101, an attribute data optimization module 102, a full-text retrieval module 103, and a cache optimization module 104.

[0052] The graphic data optimization module 101 is used to pre-process the graphic data through topological thinning technology, data compression technology and spatial indexing technology, so as to achieve high-quality compression and high-speed access of the graphic data; specifically, the topological thinning technology of the graphic data optimization module 101 is different from the Douglas Puke algorithm commonly used in the geographic information system GIS, and the topological thinning technology takes into account the topological relationship between graphics; the topological thinning technology includes: by uniformly constructing the topological relationship of the surface graphics of the administrative divisions, forming arcs and nodes, and by thinning the arcs and reconstructing the topology, the topological consistency of the restored administrative division surface is achieved. The data compression technology includes: for each graphic, based on the relative coordinate concept, taking the upper left corner point of its circumscribed rectangle as the reference point, and uniformly subtracting the other points from this point and converting them into the INT2 integer data type; by converting to the INT2 integer data type, only 2 bytes are occupied, and by the data compression technology, 75% of the storage space can be saved, and low storage can improve the query retrieval speed. The spatial indexing technology includes: storing the graphic data preprocessed by topological sparseness technology and data compression technology in a data file, and establishing an RTree index for the graphic based on the spatial indexing principle to achieve fast retrieval hits based on spatial location.

[0053] The attribute data optimization module 102 is used to optimize the administrative division attribute data by adopting a flat storage structure design, create an attribute index, and achieve a one-step search hit.

[0054] The attribute data optimization module 102 specifically includes: standardizing the administrative division data structure, processing the attribute data and pre-processed graphic data into tree data of standard structure according to the administrative division level; adopting a flat storage structure design, and storing the administrative division code, name, center point, boundary range data together with the current level graphic data of the target format, all parent attributes and graphic information, and lower-level graphics and attribute information in a structured manner; when performing spatial location or attribute information retrieval, the target can be directly hit and the required administrative division element data can be obtained in one step; the above-mentioned structured attribute data is stored in a data file, and an attribute index is created based on the characteristics of the attribute index, thereby further improving the retrieval efficiency of the attribute data.

[0055] The full-text search module 103 is used to create a full-text search table through FTS natural language-based search technology, use the simple library to perform Chinese word segmentation to support full-text search based on administrative division names; and then obtain target data based on the association between full-text search data and entity data.

[0056] The cache optimization module 104 is used to implement cache optimization for attribute retrieval through a secondary cache strategy combining memory with a Redis key-value storage system, thereby improving concurrency performance, improving retrieval performance and reducing the number of accesses to data files.

[0057] The cache optimization module 104 specifically includes: defining a cache manager to manage distributed cache when retrieving attribute conditions of administrative division code, name, and ID; performing cache reading, storage, and expiration management when retrieving through attributes, and accessing data files and updating cache when they do not exist in the cache; using the cache's key-value pair storage structure, with administrative division attribute conditions as keys and query results as values ​​for storage; using memory and Redis key-value storage system in combination to implement secondary cache, caching hot data in memory based on the hot ranking of recently accessed data, and caching the remaining data in the Redis key-value storage system; preheating the cache in conjunction with hot data, effectively avoiding cache penetration, breakdown, avalanche, and other problems.

[0058] A system 100 for rapid retrieval and display of multi-level administrative division elements provided in an embodiment of the present invention can implement each process of a method for rapid retrieval and display of multi-level administrative division elements provided in an embodiment of the present invention, and can achieve the same beneficial effects.

[0059] In summary, according to the technical solution of a method and system for rapid retrieval and display of multi-level administrative division elements of the present invention, the following advantages can be summarized: (1). The data preprocessing process has a high degree of automation and is executed based on algorithms without manual intervention. (2). Multi-dimensional optimization of graphic data and attribute data is achieved, and the retrieval speed is fast. Through testing, it is found that: more than 778,000 administrative division element data covering the national, provincial, municipal, county, township and village levels are retrieved, the total number of graphic nodes is about 920 million, the total data volume is about 18.5G, and the retrieval time is <100ms. (3). The data update cost is low, and it can be re-produced and put online based on new data as needed. (4). The requirements for hardware resources, namely computing resources and storage resources, are low. A 15G hard disk, a 4-core CPU, and a 6G memory can meet 1000 concurrent efficient accesses. (5). Support multiple sets of administrative division element data to run simultaneously to provide services.

[0060] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "illustrative embodiments", "preferred embodiments", "specific implementation", or "preferred implementation" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0061] It can be understood that the above specific description of the present invention is only used to illustrate the present invention and is not limited to the technical solutions described in the embodiments of the present invention. Those skilled in the art should understand that the present invention can still be modified or replaced by equivalents to achieve the same technical effects; as long as the use requirements are met, they are within the protection scope of the present invention.

Claims

1. A method for rapid retrieval and display of multi-level administrative division elements, characterized in that: The following steps are involved: S1. Preprocess the graphic data through topological sparseness technology, data compression technology and spatial indexing technology; Topological thinning technology, specifically including: uniformly constructing topological relationships of administrative division surface graphics to form arcs and nodes, and realizing topological consistency of restored administrative division surfaces by thinning arcs and reconstructing topology; data compression technology, specifically including: calculating the outer bounding rectangle of all points in the point coordinate array of each graphic, taking the lower left corner of the rectangle as the origin coordinate {0, 0} to establish a Gaussian rectangular coordinate system, forming a local coordinate system for the graphic, subtracting the coordinates of each point in the point coordinate array of the graphic from the origin coordinate to obtain a relative deviation value {dx, dy}, and converting this relative deviation value into INT2 integer data type for storage; spatial indexing technology, specifically including: storing graphic data preprocessed by topological thinning technology and data compression technology in a data file, and establishing an RTree index for the graphic based on the spatial index principle, for realizing fast retrieval hits based on spatial location; S2. Adopt a flat storage structure design to optimize the administrative division attribute data and create an attribute index. First, standardize the administrative division data structure, and process the attribute data and pre-processed graphic data into tree data with a standard structure according to the administrative division level. Then, adopt a flat storage structure design to store the administrative division code, name, center point, boundary range data together with the current level graphic data in the target format, all parent attributes and graphic information, and subordinate graphics and attribute information in a structured manner. Finally, store the structured attribute data in a data file, and create an attribute index based on the characteristics of the attribute index. S3. Create a full-text search table through FTS natural language-based search technology, and use the simple library for Chinese word segmentation to support full-text search based on administrative division names; S4. Implement cache optimization for attribute retrieval through a secondary cache strategy that combines memory with the Redis key-value storage system. First, define a cache manager to manage the distributed cache when retrieving attribute conditions such as administrative division code, name, and ID. Perform cache reading, storage, and expiration management when retrieving through attributes. Then, use the cached key-value pair storage structure to store administrative division attribute conditions as keys and query results as values. Finally, use memory in combination with the Redis key-value storage system to implement secondary cache. According to the hot ranking of recently accessed data, cache hot data in memory, and cache the rest of the data in the Redis key-value storage system.

2. A method for rapid retrieval and display of multi-level administrative division elements according to claim 1, characterized in that: The step S3 also includes: acquiring target data according to the association between the full-text search data and the entity data.

3. A fast retrieval and display system for multi-level administrative division elements, characterized in that: Including graphic data optimization module, attribute data optimization module, full-text search module, cache optimization module; The graphic data optimization module is used to pre-process the graphic data through topological thinning technology, data compression technology and spatial indexing technology; Topological thinning technology, specifically including: uniformly constructing topological relationships of administrative division surface graphics to form arcs and nodes, and realizing topological consistency of restored administrative division surfaces by thinning arcs and reconstructing topology; data compression technology, specifically including: calculating the outer bounding rectangle of all points in the point coordinate array of each graphic, taking the lower left corner of the rectangle as the origin coordinate {0, 0} to establish a Gaussian rectangular coordinate system, forming a local coordinate system for the graphic, subtracting the coordinates of each point in the point coordinate array of the graphic from the origin coordinate to obtain a relative deviation value {dx, dy}, and converting this relative deviation value into INT2 integer data type for storage; spatial indexing technology, specifically including: storing graphic data preprocessed by topological thinning technology and data compression technology in a data file, and establishing an RTree index for the graphic based on the spatial index principle, for realizing fast retrieval hits based on spatial location; The attribute data optimization module is used to optimize the administrative division attribute data and create an attribute index by adopting a flat storage structure design. First, the administrative division data structure is standardized, and the attribute data and the pre-processed graphic data are processed into tree data of a standard structure according to the administrative division level. Then, the administrative division code, name, center point, boundary range data and the current level graphic data of the target format, all parent attributes and graphic information, and subordinate graphics and attribute information are structured and stored together by adopting a flat storage structure design. Finally, the structured attribute data is stored in a data file, and an attribute index is created according to the characteristics of the attribute index. The full-text search module is used to create a full-text search table through the natural language-based search technology of FTS, and use the simple library to perform Chinese word segmentation to support full-text search based on administrative division names; The cache optimization module is used to implement cache optimization for attribute retrieval through a secondary cache strategy that combines memory with a Redis key-value storage system. First, when retrieving attribute conditions of administrative division code, name, and ID, a cache manager is defined to manage the distributed cache. When retrieving through attributes, cache reading, storage, and expiration management are performed. Then, a cached key-value pair storage structure is used to store administrative division attribute conditions as keys and query results as values. Finally, a secondary cache is implemented by combining memory with a Redis key-value storage system. According to the hot ranking of recently accessed data, hot data is cached in memory, and the rest of the data is cached in the Redis key-value storage system.

Images