A full-text search method and system about points of interest

Abstract

The application relates to a full-text retrieval method and system about interest points, and belongs to the technical field of data retrieval. The full-text retrieval method about interest points provided by the application is based on a full-text retrieval engine, uses interest point data to construct a spatial data index library, and can improve the accuracy of full-text retrieval. Furthermore, by mapping longitude and latitude coordinates of a spatial element into one-dimensional Moulton code, conversion from two-dimensional spatial data into one-dimensional data is realized, and the retrieval efficiency can be significantly improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data retrieval, in particular to a full-text retrieval method and system about interest points. BACKGROUND

[0002] The way of acquiring spatial data is changing rapidly, and the data volume is growing explosively. How to efficiently and conveniently organize and accurately retrieve data is the core problem faced by spatial data management. Traditional spatial data is mostly based on database keyword matching mode, and the retrieval method is single and inefficient, which is difficult to meet the demand of mass spatial data fast retrieval.

[0003] With the continuous development of full-text search and natural language processing technology, the full-text search engine (Lucene) and the widely used open source search engines such as LIUS, Egothor and Compass based on Lucene framework are formed. But Lucene only supports text and a small number of languages, and the efficiency and performance need to be improved. LIUS, Egothor and Compass have optimized the support of file types and languages, but there are deficiencies in cross-platform software development. SUMMARY

[0004] In order to solve the above problems existing in the prior art, the present application provides a full-text retrieval method and system about interest points.

[0005] In order to achieve the above purpose, the present application provides the following scheme:

[0006] A full-text retrieval method about interest points, comprising:

[0007] acquiring interest point data;

[0008] screening the interest point data according to use mode to obtain interest point element attribute information; the interest point element attribute information includes word segmentation attribute and query attribute;

[0009] convert the interest point data into a GeoJson object set, and provide attribute values for a document set in combination with the interest point element attribute information; the attribute values include word segmentation attribute values and query attribute values; the document set includes a word segmentation document set, an attribute document set and a date document set;

[0010] supplement a GeoHash field in the document set to construct a basic spatial index;

[0011] retrieve spatial elements within a preset range based on the basic spatial index by using a prefix matching method; the spatial elements are geographic information corresponding to the interest points;

[0012] map the longitude and latitude coordinates of the spatial elements into one-dimensional Morden code;

[0013] data mapping on the one-dimensional Morden code to obtain a GeoHash code;

[0014] generating a spatial data index library based on the GeoHash code and the basic spatial index;

[0015] obtaining a query condition input by a user;

[0016] retrieving the spatial data index library according to the query condition to obtain a retrieval result.

[0017] Preferably, the interest point data is filtered according to use modes to obtain interest point element attribute information, and specifically includes:

[0018] filtering an attribute used for keyword retrieval in the interest point data into a word segmentation attribute; the word segmentation attribute includes a name and an address;

[0019] filtering a field to be involved in query in the interest point data into a query attribute; the query attribute includes a coordinate range, a category and an administrative region number.

[0020] Preferably, the longitude and latitude coordinates of the spatial element are mapped into a one-dimensional Morden code, and specifically includes:

[0021] converting the longitude and latitude coordinates of the spatial element into a binary sequence;

[0022] generating the one-dimensional Morden code based on the binary sequence; even bits of the one-dimensional Morden code are longitude, and odd bits of the one-dimensional Morden code are latitude.

[0023] Preferably, a longitude and latitude approximation algorithm is used to convert the longitude and latitude coordinates of the spatial element into the binary sequence.

[0024] Preferably, data mapping is performed on the one-dimensional Morden code to obtain a GeoHash code, and specifically includes:

[0025] using a mapping relationship between a decimal and a Base32 code to convert and map the one-dimensional Morden code from high bits to low bits to obtain the GeoHash code.

[0026] Preferably, the one-dimensional Morden code is converted and mapped every 5 bits from high to low to obtain the GeoHash code.

[0027] According to specific embodiments of the present application, the following technical effects are provided:

[0028] The application provides a full-text search method for a point of interest, which is based on a full-text search engine and uses point of interest data to construct a spatial data index library, so that the accuracy of full-text search is improved. By mapping the longitude and latitude coordinates of a spatial element into a one-dimensional Mullen code, the conversion of two-dimensional spatial data into one-dimensional data is realized, and the search efficiency is significantly improved.

[0029] Corresponding to the full-text search method for a point of interest provided above, the application provides a full-text search system for a point of interest, which comprises:

[0030] A data source layer for generating point of interest data;

[0031] A user interface layer for inputting a query condition;

[0032] A search layer connected with the data source layer and the user interface layer respectively, for generating a spatial data index library according to the point of interest data, and searching the spatial data index library based on the query condition to obtain a search result.

[0033] Preferably, the search layer comprises:

[0034] A search module for searching the spatial data index library based on the query condition to obtain a search result.

[0035] The full-text search system for a point of interest provided by the application achieves the same technical effects as the full-text search method for a point of interest provided above, and thus will not be described here again. BRIEF DESCRIPTION OF DRAWINGS

[0036] In order to more clearly illustrate the technical solutions in the embodiments of the application or the prior art, the following will briefly introduce the drawings needed in the embodiments. Obviously, the drawings in the following description only only some embodiments of the application, and for those skilled in the art, other drawings can also be obtained without creative labor on the basis of these drawings.

[0037] Figure 1 The flowchart of the full-text search method for a point of interest provided by the application;

[0038] Figure 2 The structural schematic diagram of the full-text search system for a point of interest provided by the application;

[0039] Figure 3 The flowchart of the Xapian function construction provided by the embodiment of the application;

[0040] Figure 4 The flowchart of the construction of the spatial data index library provided by the embodiment of the application;

[0041] Figure 5 POI multi-mode retrieval flowchart provided for the embodiment of the present application;

[0042] Figure 6 POI retrieval efficiency analysis result chart provided for the embodiment of the present application. DETAILED DESCRIPTION

[0043] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work fall within the protection scope of the present application.

[0044] The purpose of the present application is to provide a full-text retrieval method and system about points of interest, which can improve retrieval efficiency while improving retrieval accuracy.

[0045] In order to make the above objectives, characteristics and advantages of the present application more apparent, the present application will be further described in detail below with reference to the drawings and specific embodiments.

[0046] As shown in Figure 1 The full-text retrieval method about points of interest provided by the present application comprises:

[0047] Step 100: Obtain point of interest data.

[0048] Step 101: Screen the point of interest data according to usage to obtain point of interest element attribute information. For example, screen the attribute for keyword retrieval in the point of interest data as a word segmentation attribute. Screen the field to be involved in the query in the point of interest data as a query attribute. The word segmentation attribute includes: name and address. The query attribute includes: coordinate range, category and administrative area number.

[0049] Step 102: Convert the point of interest data into a GeoJson object set, and provide attribute values for the document set in combination with the point of interest element attribute information. The attribute values include: word segmentation attribute values and query attribute values. The document set includes: a word segmentation document set, an attribute document set and a date document set, as shown in Figure 4 .

[0050] Step 103: Supplement a GeoHash field in the document set to construct a basic spatial index.

[0051] Step 104: Retrieve spatial elements within a preset range based on the basic spatial index using a prefix matching method. The spatial elements are geographic information corresponding to the points of interest.

[0052] Step 105: mapping the longitude and latitude coordinates of the spatial element into a one-dimensional Morden code. For example, converting the longitude and latitude coordinates of the spatial element into a binary sequence. Generating a one-dimensional Morden code based on the binary sequence. The even bits of the one-dimensional Morden code are the longitude, and the odd bits of the one-dimensional Morden code are the latitude. When performing binary sequence conversion, the longitude and latitude approximation algorithm is used to convert the longitude and latitude coordinates of the spatial element into a binary sequence.

[0053] Step 106: data mapping is performed on the one-dimensional Morden code to obtain the GeoHash code. Specifically, a GeoHash field is introduced, and the longitude and latitude information is mapped into sortable and comparable GeoHash codes through longitude and latitude approximation coding, Morden group coding, and Base32 coding. For example, using the mapping relationship between decimal and Base32 coding, the one-dimensional Morden code is converted and mapped every 5 bits from high to low to obtain the GeoHash code.

[0054] Step 107: generating a spatial data index library based on the GeoHash code and the basic spatial index.

[0055] Step 108: obtaining the query condition input by the user. The query condition of the present application is the retrieval condition.

[0056] Step 109: retrieving the spatial data index library according to the query condition to obtain the retrieval result.

[0057] Considering the full-text retrieval demand of spatial data, corresponding to the above-mentioned full-text retrieval method for points of interest, the full-text retrieval system for points of interest provided by the present application is designed as three layers of data source layer, retrieval layer and user interface layer. As shown in Figure 2 the data source layer is used to generate point of interest data. The user interface layer is used to input the query condition. The retrieval layer is connected with the data source layer and the user interface layer, respectively, and is used to generate a spatial data index library based on the point of interest data, and retrieve the spatial data index library based on the query condition to obtain the retrieval result. The retrieval layer includes a retrieval module. The retrieval module is used to retrieve the spatial data index library based on the query condition to obtain the retrieval result.

[0058] Further, in the implementation process, the data source layer contains point, line, surface multi-type spatial data and other multimedia spatial auxiliary data, which is organized in a spatial data resource warehouse mode. The retrieval layer contains two core business processes of index establishment and retrieval. The retrieval layer is the core content in the spatial data organization and retrieval system, and the index establishment means that the resources in the resource warehouse are parsed, the retrieval content is preprocessed, the index is formed, and the spatial data information is filled into the index library; the retrieval process in the retrieval layer generates a retrieval object through pre-analysis of the retrieval condition, searches the index library with the retrieval object to obtain the retrieval result, and returns the sorted result to the user layer through correlation model analysis.

[0059] In order to further improve the system efficiency, a multi-condition query, a synonym query, a spelling correction and a multi-mode retrieval function and a display interface can be set in the user interface layer.

[0060] The following will take Xapian as an example of the full-text retrieval method to illustrate the specific implementation process of the full-text retrieval method and system for the interest point provided above, which is not limited to the actual application.

[0061] Xapian follows the GPL agreement and is a set of open-source full-text retrieval framework with excellent performance. It abstracts records, indexes, retrieval, segmenters and other elements into specific classes, and has the characteristics of high cohesion and low coupling between modules, which can provide senior index construction and multi-mode information retrieval for developers. In particular, the index construction process is independent of the segmenter, which is easy to develop.

[0062] The general process of constructing a full-text retrieval engine by using Xapian is as follows: first, parse the data source, construct the index library to form the retrieval target; then, use the user input condition to construct the query object to retrieve the index library and perform relevance sorting, and return the query result.

[0063] The Xapian engine business process is shown in Figure 3 Database, Document, TermGenrator, QueryParser, Query, Enquire and other business classes. Database is an index library, which supports Flint, Chert, Brass, Remote and other types of databases, and can be flexibly selected according to actual use requirements; Database is composed of a large number of Document objects, and any content that needs to be retrieved must be converted or mapped into a Document object; TermGenrator is a segmenter relationship generator that stores the segmenting details in Document in Database to provide a basis for data retrieval and relevance sorting; QueryParser maps the user input query condition into a specific Query (query object), and Enquire (query session) runs the Query object and returns the retrieval result.

[0064] Generally in information retrieval system, as the index establishment and retrieval public foundation module, Chinese word segmentation module should ensure its high consistency, ensure the index and retrieval effect. Through comparative analysis, the embodiment selects MMSEG to realize Chinese word segmentation, which has the characteristics of simplicity, high efficiency, practicality and the like. MMSEG is realized based on maximum matching algorithm, the accuracy rate reaches 99.69%, and 93.21% of ambiguity can be eliminated by the ambiguity elimination rule in the algorithm. Compared with IK, MMSEG word segmentation has higher open source heat, the algorithm is relatively mature, and has more references than Paoding word segmentation. Compared with Imdict word segmentation algorithm, MMSEG is more easy to realize and complete the expansion of the word library.

[0065] POI(point of interest), i.e. interest point, hotspot, is a spatial element with geographical significance in a local range, which can be divided into government departments, commercial institutions, tourist attractions and the like according to categories, belongs to the core component of spatial data, and is different from other types of spatial data retrieval. POI retrieval has stronger flexibility and polymorphism. Therefore, the embodiment organizes and retrieves POI based on Xapian, can fully verify the functions and performance indicators of the Xapian engine, and can lay a solid foundation for full-text retrieval of other types of spatial data.

[0066] Based on this, the embodiment provides a specific implementation process of the full-text retrieval method and system for interest points:

[0067] Step 1, POI data analysis and organization

[0068] GDAL(Geospatial Data Abstraction Library) is a spatial data conversion open source library complying with X / MIT license agreement, and almost all geographic information system software bottom layers use GDAL to analyze and convert spatial data. The conversion process is mature and has excellent performance. Therefore, GDAL is used as a data analysis and conversion bottom layer implementation tool. GeoJson is realized based on Javascript, supports multiple geographic data exchange formats, and has clear storage structure, which is convenient for efficient analysis and storage of spatial elements. Therefore, GeoJson is determined as the physical storage form of the prototype system.

[0069] POI attributes are different due to different categories, and are divided into attributes, query attribute screening and data conversion according to the use mode (frequency, heat), such as Figure 4The attributes screened for keyword search become the segmentation attributes, the target GeoHash code (name, pinyin) is segmented by the Chinese segmentation module, and the corresponding Document::Term, such as name, address, etc. is obtained; the fields screened for participating in the query are used as the query attributes, which are the basis for complex query, result screening and sorting, and the corresponding Document::Value, such as coordinate range, category, administrative district number, etc. is obtained. The POI data is converted into a GeoJson object set by GDAL, and combined with the attribute screening category rules to provide attribute values for Document::Data (date document set), Document::Value (attribute document set), and Document::Term (segmentation document set).

[0070] Step 2, POI attribute organization optimization

[0071] Based on the existing POI element attribute information, the GeoHash field is supplemented for constructing the basic spatial index, which can be used for preliminary retrieval of nearby spatial elements by using the prefix matching method; at the same time, the latitude and longitude coordinates are mapped into one-dimensional Morden code, which is more convenient than two-dimensional spatial calculation.

[0072] Step 2-1, GeoHash encoding

[0073] GeoHash geocoding is proposed by Gustavo Niemeyer, which uses GeoHash encoding to represent the latitude and longitude of spatial elements. Each encoding represents a region. All points within the region can be represented by the same GeoHash encoding. The longer the GeoHash encoding, the more accurate the range representation. It belongs to the category of linear encoding of quadtree, which is easy to establish spatial index. By introducing the GeoHash field, the latitude and longitude information is mapped into sortable and comparable GeoHash encoding through latitude and longitude approximation encoding, Morden code, and Base32 encoding. Taking [116.389550, 39.928167] as an example, the encoding process is explained.

[0074] Step 2-11, latitude and longitude approximation encoding

[0075] The longitude and latitude intervals are [-180, 180] and [-90, 90], respectively. Approximation encoding can be performed based on the interval characteristics.

[0076] First, divide the longitude interval into [-180, 0) and [0, 180], i.e. left and right intervals. It can be determined that 116.389550 belongs to the right interval [0, 180], which is marked as 1;

[0077] Second step, divide [0, 180] into two parts [0, 90) and [90, 180], it can be determined that 116.389550 belongs to the right interval [90, 180), marked as 1;

[0078] Third step, recursively process the above process 116.389550 always belongs to some interval [a, b] or [a, b). After multiple iterations, the interval range gradually shrinks, and gradually approaches 116.389550. The latitude calculation mode is the same, thus generating the bit sequence 1101001011 (longitude), 1011100011 (latitude), as shown in Table 1.

[0079] Table 1: Latitude and longitude approximation encoding example table

[0080]

[0081] Step 2-12, Morton group code and Base32 encoding

[0082] Using the above latitude and longitude binary sequence, even bits are longitude and odd bits are latitude, the Merton group code is generated: 11100111010010001111. Using the decimal and Base32 mapping relationship, combined with the Merton group code from high to low every 5 bits are converted and mapped one by one, the final string wx4g is obtained, and the mapping relationship is shown in Table 2.

[0083] Table 2: Decimal and Base32 mapping relationship table

[0084] Decimal 0 1 3 4 5 6 7 8 9 10 11 12 13 14 15 16 base32 0 1 2 3 4 5 6 7 8 9 b c d e f g Decimal 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 base32 h j k m n p q r s t u v w x y z

[0085] Table 3: GeoHash encoding length and accuracy range table

[0086] GeoHash length Lat digits Lng digits Lat interval Lng interval Km interval 1 2 3 ±23 ±23 ±2500 2 5 5 ±2.8 ±5.6 ±630 3 7 8 ±0.70 ±0.7 ±78 4 10 10 ±0.087 ±0.18 ±20 5 12 13 ±0.022 ±0.022 ±2.4 6 15 15 ±0.0027 ±0.0055 ±0.61 7 17 18 ±0.00068 ±0.00068 ±0.076 8 20 20 ±0.000086 ±0.000172 ±0.01911 9 22 23 ±0.000021 ±0.000021 ±0.00478 10 25 25 ±0.00000268 ±0.00000536 ±0.0005971 11 27 28 ±0.00000067 ±0.00000067 ±0.0001492 12 30 30 ±0.00000008 ±0.00000017 ±0.0000186

[0087] Where, the range of Latitude: -90 to 90; the range of Longitude: -180 to 180; the circumference of the reference sphere: 40075016.68 meters.

[0088] As can be seen from Table 3, GeoHash encoding can easily cope with different levels of regional queries.

[0089] Step 2-2, coordinate encoding

[0090] Establish the mapping relationship between latitude and longitude coordinates and 32-bit integers (360° and 232), use all 32-bit signed integers. Therefore, the latitude and longitude coordinates are mapped in the range –231≤x<231(longitude) and –230≤y<230(latitude).

[0091] Morton code converts two numerical longitude (x) and latitude (y) into one-dimensional encoding, uniquely identifies POI elements. Encoding details as follows:

[0092] Get longitude and latitude binary sequence

[0093] X = X31X30...X1X0and Y = Y30...Y1Y0;

[0094] Morton encoding

[0095] Z = X31Y30X30...Y1X1Y0X0;

[0096] Conversion example

[0097] For example: lacquer tree (110.178882, 33.190662)

[0098] Integer mapping conversion:

[0099] X = 110.178882 * (232 / 360) = 1314485263.6106752;

[0100] Y = 33.190662 * (232 / 360) = 395980021.72941653333333333333333;

[0101] Convert to binary:

[0102] X = 01001110010110010111010000001111;

[0103] Y = 0010111100110100010110011110101;

[0104] Morton encoding:

[0105] Z = 001001001111110100100111100100100011101101100001010101001110111 = 1332665031744072311 (decimal) = 127E93C91DB0AA77 (hexadecimal).

[0106] Morton code coordinates compared with traditional two-dimensional coordinate storage mode, while ensuring the same coordinate precision attribute field reduction, help to improve the efficiency of data organization and spatial analysis.

[0107] Step 3, POI data retrieval

[0108] POI full-text retrieval prototype system supports multiple types of retrieval mode, retrieval process as Figure 5As shown. First, the search keywords in the query condition are subjected to Chinese word segmentation preprocessing to obtain a plurality of Terms (word set), and the Term Matcher finds the documents containing the corresponding Terms in the database and returns a preliminary result list; second, if the query condition also includes field query conditions (Files), for example: "latitude and longitude range: (112, 33) to (112.2, 33.20)", the Range Matcher needs to traverse the preliminary result list and filter the documents according to the spatial range to obtain a further result list; third, the MatchDecider can be used to perform high-performance filtering according to the DocumentValues (Files) to form the final search results.

[0109] The POI full-text search method and system provided by the application are used to organize and manage 9.29 million POI data in the whole country and perform query experiments to verify the superiority of the application.

[0110] The query test samples (single word, multiple words) and the time-consuming situation are shown in Table 4.

[0111] Table 4 POI full-text search test sample table

[0112]

[0113] Based on the ubuntu operating system, the time-consuming situation of simple fuzzy query is compared and tested in the states of 2G, 4G and 6G memory. The test situation shows that the search time-consuming does not linearly increase with the decrease of memory and has little influence; with the gradual increase of data volume, the time-consuming growth also gradually tends to be stable. As shown in Table 5, in general, the time-consuming of most queries is within 1s, and the efficiency is high. Figure 6

[0114] Based on the analysis of the structure and characteristics of the full-text search engine, the application is designed in combination with the characteristics of spatial data to realize the organization and management of massive POI data. Through testing and analysis, the application has high efficiency and feasibility. Moreover, the application uses the existing spatial data full-text search prototype system and can be easily extended to other application platforms in specific fields through cross-compilation and applied to the fields of military, transportation, aerospace and the like.

[0115] The embodiments in the specification are described in a progressive manner, and each embodiment focuses on the difference from other embodiments, and the same and similar parts of each embodiment can be referred to each other. For the system disclosed by the embodiments, since it corresponds to the method disclosed by the embodiments, the description is relatively simple, and the related parts can be referred to the method part.

[0116] ​The principles and implementation manners of the present application are described by using specific examples in the present application, and the above examples are only used to help understand the method of the present application and its core idea; meanwhile, for the general technical personnel in the art, the specific implementation manners and application ranges will be changed according to the idea of the present application. In conclusion, the content of the present specification should not be understood as the limitation of the present application.

Claims

1. A method of full-text search on points of interest, characterized by, The method comprises the following steps: acquiring point of interest data; filtering the point of interest data according to usage to obtain point of interest element attribute information; the point of interest element attribute information comprises a segmentation attribute and a query attribute; converting the point of interest data into a GeoJson object set, and combining the point of interest element attribute information to provide attribute values for a document set; the attribute values comprise a segmentation attribute value and a query attribute value; the document set comprises a segmentation document set, an attribute document set and a date document set; supplementing a GeoHash field in the document set to construct a basic spatial index; based on the basic spatial index, using a prefix matching method to search for spatial elements within a preset range; the spatial elements are geographic information corresponding to the point of interest; mapping the longitude and latitude coordinates of the spatial elements into one-dimensional Morgen code; performing data mapping on the one-dimensional Morgen code to obtain GeoHash code; wherein, by using the mapping relationship between decimal and Base32 code, the one-dimensional Morgen code is converted and mapped from high bits to low bits to obtain the GeoHash code; generating a spatial data index library based on the GeoHash code and the basic spatial index; acquiring a query condition input by a user; searching the spatial data index library according to the query condition to obtain a search result.

2. The method of claim 1, wherein, The filtering of the point of interest data according to usage to obtain point of interest element attribute information specifically comprises: filtering attributes for keyword search in the point of interest data into segmentation attributes; the segmentation attributes comprise a name and an address; filtering fields to be involved in query in the point of interest data into query attributes; the query attributes comprise a coordinate range, a category and an administrative region number.

3. The method of claim 1, wherein the full-text search is performed with respect to a point of interest. The mapping of the longitude and latitude coordinates of the spatial elements into one-dimensional Morgen code specifically comprises: converting the longitude and latitude coordinates of the spatial elements into a binary sequence; generating the one-dimensional Morgen code based on the binary sequence; even bits of the one-dimensional Morgen code are longitude, and odd bits of the one-dimensional Morgen code are latitude.

4. The method of claim 3, wherein the full-text search is performed with respect to a point of interest. The longitude and latitude coordinates of the spatial elements are converted into a binary sequence by using a longitude and latitude approximation algorithm.

5. The method of claim 1, wherein the full-text search on the point of interest is performed by using a search engine. The one-dimensional Morgen code is converted and mapped every 5 bits from high to low to obtain the GeoHash code.

6. A full-text search system for points of interest, characterized by The full-text search system for points of interest is implemented by using the full-text search method for points of interest according to any one of claims 1-5; the system comprises: a data source layer for generating point of interest data; a user interface layer for inputting a query condition; a search layer connected with the data source layer and the user interface layer respectively, for generating a spatial data index library according to point of interest data, and searching the spatial data index library based on a query condition to obtain a search result.

7. The full-text search system for points of interest according to claim 6, wherein, The search layer comprises: a search module for searching the spatial data index library based on a query condition to obtain a search result.

Images