Spatial data retrieval method, device, electronic device and storage medium
By performing dimensionality reduction encoding and screening on spatial data in a distributed database, the problem of high-concurrency real-time retrieval under large-scale data sets is solved, and efficient spatial data retrieval is achieved.
Patent Information
- Application Number
- CN202210492015.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-05-07
- Publication Date
- 2025-09-05
- Estimated Expiration
- 2042-05-07
AI Technical Summary
Existing spatial data retrieval solutions cannot achieve high-concurrency real-time retrieval under large-scale data sets, and due to the edge mutation characteristics of dimensionality reduction coding, a lot of invalid data is queried, which cannot meet the high-concurrency query scenario.
The spatial coding rules of the distributed database are used to perform dimensionality reduction coding on the query space range, and the codes that overlap with the query space range are screened out to determine the target physical files, and spatial data that meets the query filtering conditions are searched in these files.
It achieves high-concurrency real-time retrieval under large-scale data sets, reduces the amount of invalid data, and improves retrieval efficiency and real-time performance.
Smart Images

Figure CN114880350B_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to the field of computer technology, and in particular to a spatial data retrieval method, device, electronic device, and storage medium. Background Art
[0002] Spatial information has become a crucial data source for digital city infrastructure development. People need convenient and efficient spatial information services in all aspects of life, including economics, transportation, and culture. Currently, due to the frequent updates, massive scale, explosive growth, and complex geospatial models of spatial data, researchers are focusing on the storage and organization of spatial data and the spatial operation algorithms to meet the ever-increasing demand for spatial information processing.
[0003] With the development of spatial information cloud computing technology, a spatial data retrieval scheme for non-relational databases (NoSQL) based on dimensionality reduction coding has been proposed. In this scheme, multidimensional spatial data is converted into a one-dimensional string through dimensionality reduction coding, thereby converting multidimensional spatial filtering queries into one-dimensional range search queries. However, since commonly used dimensionality reduction coding schemes have the characteristics of edge mutation, a lot of invalid data outside the query space range is queried, which requires a large amount of additional filtering work. This is more suitable for offline analysis requirements and cannot meet the needs of high-concurrency query scenarios.
[0004] Therefore, there is an urgent need for a retrieval solution that can support large-scale data sets and has high concurrent real-time retrieval capabilities. Summary of the Invention
[0005] In order to solve the above technical problems or at least partially solve the above technical problems, the present disclosure provides a spatial data retrieval method, device, electronic device and storage medium.
[0006] In a first aspect, an embodiment of the present disclosure provides a spatial data retrieval method, the method comprising:
[0007] receiving a search request for spatial data to be queried, the search request including a query filter condition corresponding to the spatial data to be queried, the query filter condition including a spatial range to be queried;
[0008] Performing dimensionality reduction coding on the spatial range to be queried according to a target coding rule to obtain multiple one-dimensional codes, wherein the target coding rule is a spatial coding rule of a distributed database, each one-dimensional code is used to indicate a physical file in the distributed database, and each physical file includes at least one spatial data;
[0009] Filtering out at least one code to be queried from the multiple one-dimensional codes, wherein a spatial range corresponding to each code to be queried overlaps with the spatial range to be queried;
[0010] Determine at least one target physical file from the distributed database, each target physical file being a physical file in the distributed database corresponding to one of the codes to be queried;
[0011] Determine the spatial data to be queried that meets the query filter condition from the at least one target physical file.
[0012] In a second aspect, an embodiment of the present disclosure further provides a spatial data retrieval device, the device comprising: a receiving module, an encoding module, a screening module, and a determining module;
[0013] The receiving module is configured to receive a search request for the spatial data to be queried, wherein the search request includes a query filter condition corresponding to the spatial data to be queried, and the query filter condition includes a spatial range to be queried;
[0014] The encoding module is used to perform dimensionality reduction encoding on the query space range according to a target encoding rule to obtain multiple one-dimensional codes, wherein the target encoding rule is a spatial encoding rule of a distributed database, each one-dimensional code is used to indicate a physical file in the distributed database, and each physical file includes at least one spatial data;
[0015] The screening module is configured to screen out at least one code to be queried from the multiple one-dimensional codes obtained by the encoding module, wherein the spatial range corresponding to each code to be queried overlaps with the spatial range to be queried;
[0016] The determining module is configured to determine at least one target physical file from the distributed database, each target physical file being a physical file in the distributed database corresponding to a code to be queried obtained by the screening module;
[0017] The determining module is further configured to determine, from the at least one target physical file, the spatial data to be queried that meets the query filtering condition.
[0018] In a third aspect, an embodiment of the present disclosure further provides an electronic device, comprising: a memory and a processor; the memory is used to store the processor executable instructions; the processor is used to read the executable instructions from the memory and execute the executable instructions to implement any of the spatial data retrieval methods provided in the embodiments of the present disclosure.
[0019] In a fourth aspect, an embodiment of the present disclosure further provides a computer-readable storage medium, wherein the storage medium stores a computer program, and the computer program is used to execute any of the spatial data retrieval methods provided by the embodiments of the present disclosure.
[0020] In a fifth aspect, an embodiment of the present disclosure further provides a computer program product, which is used to execute any of the spatial data retrieval methods provided by the embodiments of the present disclosure.
[0021] Compared with the prior art, the technical solution provided by the embodiment of the present disclosure has at least the following advantages: In the embodiment of the present disclosure, a retrieval request for spatial data to be queried is received (the retrieval request includes a query filter condition corresponding to the spatial data to be queried, and the query filter condition includes a spatial range to be queried); according to a target coding rule (a spatial coding rule of a distributed database), a dimensionality reduction coding is performed on the spatial range to be queried to obtain a plurality of one-dimensional codes (each one-dimensional code is used to indicate a physical file in the distributed database, and each physical file includes at least one spatial data); at least one code to be queried is screened out from the plurality of one-dimensional codes (the spatial range corresponding to each code to be queried overlaps with the spatial range to be queried); at least one target is determined from the distributed database. physical file (each target physical file is a physical file in the distributed database corresponding to a code to be queried); from the at least one target physical file, determining the spatial data to be queried that meets the query filtering condition (the query filtering condition includes the spatial range to be queried), in this solution, multiple one-dimensional codes obtained by dimensionality reduction coding of the spatial range to be queried are screened to obtain at least one code to be queried whose corresponding spatial range overlaps with the spatial range to be queried. In this way, under the premise that the solution based on dimensionality reduction coding combined with distributed database supports large-scale data, the problem of querying a large amount of invalid data outside the spatial range to be queried due to the edge mutation characteristics of dimensionality reduction coding is solved, thereby realizing high-concurrency real-time retrieval of distributed databases. BRIEF DESCRIPTION OF THE DRAWINGS
[0022] The above and other features, advantages, and aspects of the various embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and that the originals and elements are not necessarily drawn to scale.
[0023] Figure 1 A schematic diagram of a spatial data retrieval process provided by an embodiment of the present disclosure;
[0024] Figure 2 A schematic diagram of another spatial data retrieval process provided by an embodiment of the present disclosure;
[0025] Figure 3 A flowchart of another spatial data retrieval method provided by an embodiment of the present disclosure;
[0026] Figure 4A schematic structural diagram of another spatial data retrieval device provided in an embodiment of the present disclosure;
[0027] Figure 5 A schematic diagram of an index structure within a spatial grid of a distributed system based on dimensionality reduction coding provided by an embodiment of the present disclosure;
[0028] Figure 6 A schematic structural diagram of a spatial data retrieval device provided in an embodiment of the present disclosure;
[0029] Figure 7 A schematic structural diagram of an electronic device provided in an embodiment of the present disclosure. DETAILED DESCRIPTION
[0030] The following describes embodiments of the present disclosure in more detail with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as limited to the embodiments described herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are for illustrative purposes only and are not intended to limit the scope of protection of the present disclosure.
[0031] It should be understood that the various steps described in the method embodiments of the present disclosure may be performed in different orders and / or in parallel. In addition, the method embodiments may include additional steps and / or omit the steps shown. The scope of the present disclosure is not limited in this respect.
[0032] As used herein, the term "including" and its variations are open-ended, i.e., "including but not limited to." The term "based on" means "based, at least in part, on." The term "one embodiment" means "at least one embodiment," the term "another embodiment" means "at least one additional embodiment," and the term "some embodiments" means "at least some embodiments." Other terms are defined in the following description.
[0033] It should be noted that the concepts of "first" and "second" mentioned in this disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.
[0034] It should be noted that the modifications of "one" and "multiple" mentioned in the present disclosure are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, they should be understood as "one or more".
[0035] First, some nouns or terms involved in the claims and description of the present disclosure are explained below.
[0036] Spatial index tree (Rtree): The retrieval of spatial data in the database is usually implemented based on Rtree.
[0037] Dimensionality reduction coding: Based on the space-filling curve, multi-dimensional data is represented by one-dimensional data to achieve the purpose of dimensionality reduction.
[0038] Hbase: A distributed storage database that provides distributed storage and dynamic routing of data based on a one-dimensional primary key.
[0039] In real-time spatial search and filtering scenarios involving two-dimensional or even three-dimensional space, such as high-precision maps, involving large amounts of point cloud data, Rtree-based retrieval solutions based on traditional relational databases (PostgreSQL, MySQL) offer rich spatial query capabilities and are suitable for indexing small volumes of data. However, once the volume exceeds 100 million, index bloat occurs. Limited by the system resources available in a single-server system, Rtree indexes struggle to meet the storage, expansion, and specific query requirements of massive path data. Furthermore, Rtree indexes undergo index rebuilding (rebuilding specific paths) when data is updated, which can also degrade database performance. Retrieval solutions based on dimensionality reduction encoding in NoSQL databases (such as HBase) convert multidimensional spatial data into one-dimensional strings through encoding schemes, converting multidimensional spatial filtering queries into one-dimensional range search queries. However, due to the edge mutation characteristics of common encoding schemes, these schemes can result in the discovery of a significant amount of invalid data outside the query space, requiring extensive filtering work. While these solutions are more suitable for offline analysis, they are not suitable for high-concurrency query scenarios.
[0040] In general, existing solutions in the industry are more suitable for real-time spatial retrieval at a small scale or offline analysis of spatial data at a large scale. Therefore, the spatial data retrieval method provided in the embodiments of the present disclosure aims to solve the problem that existing indexing solutions only support real-time spatial retrieval at a small scale or offline analysis of spatial data at a large scale, but cannot support real-time retrieval scenarios with large-scale spatiotemporal data volumes.
[0041] Figure 1 This is a flowchart of a spatial data retrieval method provided by an embodiment of the present disclosure, which can be applied to real-time retrieval scenarios involving large volumes of spatial data. This spatial data retrieval method can be performed by a spatial data retrieval device, which can be implemented using software and / or hardware and integrated into any electronic device or server with computing capabilities.
[0042] like Figure 1 As shown, the spatial data retrieval method provided by the embodiment of the present disclosure may include:
[0043] S100: Receive a search request for spatial data to be queried.
[0044] The search request includes query filter conditions corresponding to the spatial data to be queried. The query filter conditions include the spatial range to be queried. The query filter conditions may also include other filter conditions such as attribute filter conditions, which may be determined based on actual conditions and are not limited here.
[0045] S101. According to target coding rules, perform dimensionality reduction coding on the query space range to obtain multiple one-dimensional codes.
[0046] The target encoding rule is a spatial encoding rule of a distributed database, each one-dimensional code is used to indicate a physical file in the distributed database, and each physical file includes at least one spatial data.
[0047] Optionally, the distributed database may be any existing NoSQL database, or a customized database with indexing capability built based on an underlying distributed file system. The specific database may be determined based on actual conditions and is not limited here.
[0048] Among them, the distributed database includes multiple physical files, each physical file corresponds to a one-dimensional code, and different physical files correspond to different one-dimensional codes; each physical file includes at least one spatial data, and the number of spatial data included in different physical files can be the same or different, which can be determined according to actual conditions and is not limited here.
[0049] It can be understood that in the embodiments of the present disclosure, a two-dimensional index or a three-dimensional index based on space is transformed into a one-dimensional index, or a three-dimensional index or a four-dimensional index based on time and space is transformed into a one-dimensional index through a dimensionality reduction coding method. Each one-dimensional code obtained by coding corresponds one-to-one to each physical file in the distributed database. In this way, the coding rules of the distributed database can be used to perform dimensionality reduction coding on the spatial range to be queried. In this way, multiple one-dimensional codes corresponding to the spatial range to be queried can be obtained, and each one-dimensional code corresponds to a physical file in the distributed database.
[0050] The target encoding rule may be ZOrder encoding based on a Z curve, S2 encoding based on a Hilbert curve, or other encodings, which may be determined based on actual conditions and are not limited here.
[0051] Exemplarily, the target encoding rule is GeoHash encoding in ZOrder encoding based on the Z curve.
[0052] S102: Filter out at least one code to be queried from the multiple one-dimensional codes.
[0053] The spatial range corresponding to each code to be queried overlaps with the spatial range to be queried.
[0054] It can be understood that since commonly used dimensionality reduction coding schemes all have the characteristics of edge mutation, it is necessary to screen multiple one-dimensional codes in combination with the spatial range to be queried, and filter out invalid (irrelevant) one-dimensional codes introduced due to edge mutations, that is, filter out one-dimensional codes whose corresponding spatial range does not overlap with the spatial range to be queried.
[0055] It can be understood that because the one-dimensional code and the actual spatial range are one-to-one corresponding, the one-dimensional code obtained by dimensionality reduction coding can be restored to a specific spatial bounding box (Bounding Box, BBOX). Based on the spatial intersection judgment between the query spatial range and the BBOX restored from each one-dimensional code, invalid one-dimensional codes can be eliminated.
[0056] Exemplarily, each one-dimensional code among the multiple one-dimensional codes obtained by dimensionality reduction coding can be restored to a corresponding spatial range, and then it is determined whether the spatial range restored from each one-dimensional code overlaps with the spatial range to be queried (i.e., a spatial intersection determination is performed). If the spatial range restored from a one-dimensional code overlaps with the spatial range to be queried (i.e., there is a common spatial range, and the spatial intersection is determined), then the one-dimensional code is determined to be a valid one-dimensional code corresponding to the spatial range to be queried; if the spatial range restored from a one-dimensional code does not overlap with the spatial range to be queried (there is no common spatial range, and the spaces are determined to be non-intersecting), then the one-dimensional code is determined to be an invalid one-dimensional code corresponding to the spatial range to be queried, and the one-dimensional code is eliminated.
[0057] S103: Determine at least one target physical file from the distributed database.
[0058] Each target physical file is a physical file in the distributed database corresponding to a code to be queried.
[0059] It can be understood that each code to be queried is a one-dimensional code corresponding to the spatial range to be queried, and the physical file in the distributed database indicated by each code to be queried is a target physical file corresponding to the spatial range to be queried.
[0060] S104: Determine, from the at least one target physical file, spatial data to be queried that meets the query filter condition.
[0061] Optionally, the query filtering condition may further include at least one of an attribute filtering condition, a combined query condition, and a topology query condition, and may further include other query conditions, which may be specifically determined according to actual conditions and are not limited here.
[0062] It can be understood that the spatial data to be queried is filtered out from the spatial data in at least one target physical file through the query filtering condition. The specific filtering query process can be determined according to the query filtering condition. Please refer to the relevant technology and will not be described here.
[0063] In the embodiment of the present disclosure, multiple one-dimensional codes obtained by dimensionality reduction coding of the spatial range to be queried are screened to obtain at least one code to be queried whose corresponding spatial range overlaps with the spatial range to be queried. In this way, under the premise of supporting large-scale data based on the solution of dimensionality reduction coding combined with distributed database, the problem of querying a lot of invalid data outside the spatial range to be queried due to the edge mutation characteristics of dimensionality reduction coding is solved, thereby realizing high-concurrency real-time retrieval of distributed databases.
[0064] Figure 2 This is a flow chart of another spatial data retrieval method provided by the embodiment of the present disclosure, which is further optimized and expanded based on the above technical solution and can be combined with the above optional implementation methods. Figure 2 As shown, the spatial data retrieval method provided by the embodiment of the present disclosure may include:
[0065] S201: Divide the data space in the global coordinate system into a plurality of spatial grids of preset sizes.
[0066] S202: Perform dimensionality reduction coding on each spatial grid according to the target coding rule to obtain a one-dimensional code corresponding to each spatial grid.
[0067] The preset size can be determined according to actual conditions and is not limited here.
[0068] It can be understood that at a specified scale, the data space is divided into spatial grids of a preset size in the global coordinate system, and the data space to which the spatial grid belongs is dimensionality reduced and encoded to obtain a one-dimensional code corresponding to each spatial grid. The one-dimensional code corresponding to each spatial grid is a globally unique grid code, which is hereinafter referred to as GridKey.
[0069] S203 : Based on the target coding rule, perform dimensionality reduction coding on each spatial data to be stored to obtain a one-dimensional data code corresponding to each spatial data to be stored.
[0070] The prefix of each one-dimensional data code is the one-dimensional code corresponding to a spatial grid.
[0071] S204: Store each spatial data to be stored in a physical file corresponding to the spatial grid indicated by the prefix of the corresponding one-dimensional data code to obtain the distributed database.
[0072] The identifier of each physical file is a one-dimensional code of the corresponding spatial grid.
[0073] It can be understood that after the spatial data to be stored is dimensionality reduced and encoded according to the same encoding rules as the spatial grid encoding (i.e., the target encoding rules), a one-dimensional data encoding corresponding to each spatial data to be stored is obtained, which is hereinafter referred to as DataKey (DataKey is a globally unique encoding of spatial data). Based on the prefix matching relationship between DataKey and GridKey (GridKey is the prefix of DataKey), each spatial data to be stored is assigned to the physical file corresponding to the specified spatial grid to obtain a distributed database.
[0074] S205: Receive a search request for spatial data to be queried.
[0075] S206 , performing dimensionality reduction coding on the query space range according to the target coding rule to obtain multiple one-dimensional codes.
[0076] S207: Filter out at least one code to be queried from the multiple one-dimensional codes.
[0077] S208: Determine at least one target physical file from the distributed database.
[0078] S209: Determine, from the at least one target physical file, spatial data to be queried that meets the query filter condition.
[0079] The description of S205 to S209 may refer to the relevant description of S100 to S104, and is not limited here.
[0080] In the embodiment of the present disclosure, by dividing the data space into multiple grid spaces, the multiple grid spaces and the multiple spatial data to be stored are respectively subjected to dimensionality reduction encoding based on the target encoding rule, and then the spatial data to be stored is mapped to the corresponding spatial grid according to the prefix matching relationship between the spatial grid encoding and the spatial data encoding to be stored, thereby establishing a distributed database with indexing capabilities based on the underlying distributed system. In this way, it is convenient to quickly and in real time retrieve the spatial data to be queried from the distributed database.
[0081] On the basis of the above technical solution, optionally, combined with Figure 2 ,like Figure 3 As shown, the above S204 may specifically include:
[0082] S204a: Divide each space data to be stored into multiple data sets.
[0083] The prefix of the one-dimensional data encoding corresponding to the spatial data in each data set is the same.
[0084] It can be understood that the prefix of one-dimensional data encoding is one-dimensional encoding, and the one-dimensional encoding indicates a spatial grid. Therefore, the spatial data in each data set corresponds to a spatial grid.
[0085] S204b: Count the metadata description of each data set.
[0086] The metadata description of each data set includes: the spatial range of each spatial data in the corresponding data set, and the non-spatial attribute information of each spatial data.
[0087] It can be understood that counting the metadata description of each data set is to count some global statistical information under the spatial grid corresponding to each data set.
[0088] The metadata description may include: the actual spatial range of the spatial data under the corresponding spatial grid, and the non-spatial attribute information of the spatial data under the corresponding spatial grid that can participate in filtering.
[0089] The non-spatial attribute information may include at least one of non-spatial attributes such as a time attribute and a type attribute, which may be determined based on actual conditions and is not limited here.
[0090] For example, the time attribute may be the time range of the spatial data under the statistical spatial grid, and the type attribute may be an enumeration of spatial data type values under the statistical spatial grid.
[0091] S204c: Generate corresponding physical files based on the spatial data in each data set and the corresponding metadata description to obtain the distributed database.
[0092] The identifier of each physical file is a one-dimensional code of a corresponding spatial grid, and the metadata description in each physical file is an index of the spatial data in the corresponding physical file.
[0093] It can be understood that the spatial data in each data set and the corresponding metadata description are mapped to the corresponding spatial grid, and then the data corresponding to each spatial grid (spatial data and metadata description) are grid-persisted to generate the corresponding physical file and store it on the distributed file system. The GridKey corresponding to the spatial grid is used as the file identifier of the physical file (that is, the index corresponding to the physical file).
[0094] Each physical file consists of two parts: one is the metadata description (MetaSection), which stores the global statistical information of all spatial data in the spatial grid; the other is the data segment (DataSection), which stores all spatial data in the spatial grid. During the retrieval process, the spatial data to be queried can be retrieved and located from the DataSection based on the MetaSection (that is, the metadata description is an index of the spatial data stored in the corresponding physical file). In this way, the spatial data in the DataSection can be searched secondary based on the MetaSection, realizing the index within the physical file, which can more quickly retrieve the spatial data to be queried.
[0095] In the embodiment of the present disclosure, the spatial data to be stored is divided into multiple data sets according to the corresponding spatial grids, and then the metadata description of each data set is counted. Then, a corresponding physical file is generated based on the spatial data and metadata description in each data set, wherein the metadata description is an index of the spatial data stored in the corresponding physical file, that is, an index within the file. In this way, on the premise of searching the spatial grid through one-dimensional coding, the secondary index within the spatial grid is combined to realize the search within the physical file, and then the spatial data in the physical file is quickly filtered through the metadata description, which can effectively reduce the query of useless data. It is more efficient than directly filtering all the spatial data in the physical file, and can improve the real-time retrieval of spatial data.
[0096] Based on the above technical solution, optionally, each physical file includes at least one slice file, and each slice file stores at least one spatial data in the corresponding physical file.
[0097] It can be understood that dividing a physical file into at least one slice file, that is, performing slice management on spatial data within the physical file, can improve the management efficiency and retrieval efficiency of the spatial data in the physical file.
[0098] The number of slice files included in each physical file may be the same or different, which may be determined according to actual conditions and is not limited here.
[0099] The size of each slice file in each physical file may be the same or different, and may be determined according to actual conditions, which is not limited here.
[0100] For example, the spatial data stored in the entire physical file may be divided into multiple slice files according to a certain size, and each slice file is recorded as a Stripe.
[0101] Based on the above technical solution, optionally, the metadata description in each physical file includes: the spatial range of each slice file corresponding to the physical file, and non-spatial attribute information of the spatial data in each slice file corresponding to the physical file.
[0102] It can be understood that the metadata description includes the spatial range of each slice file corresponding to the physical file. When retrieving spatial data in the physical file, the target slice file that matches the spatial data to be queried can be determined based on whether the spatial range of the spatial data to be queried matches the spatial range of each slice file (i.e., whether there is overlap based on spatial intersection). Then, the spatial data to be queried can be retrieved from the target slice file based on the attribute filtering conditions. This can effectively reduce the query of useless data, is more efficient than directly filtering all spatial data in the physical file, and can improve the real-time performance of spatial data retrieval.
[0103] It can be understood that the spatial data in each physical file in the distributed database can be managed in shards (that is, the physical file is divided into multiple shard files); the spatial data in each physical file in the distributed database can also not be managed in shards; or the spatial data in some physical files in the distributed database can be managed in shards, while the spatial data in some physical files cannot be managed in shards. The specific details can be determined based on actual conditions and are not limited here.
[0104] It can be understood that when the spatial data in a physical file in a distributed database is managed in slices (i.e., the physical file is divided into multiple slice files), the metadata description can be managed for each slice file or for the entire physical file. The specific method can be determined according to actual conditions and is not limited here.
[0105] Based on the above technical solution, optionally, the at least one spatial data in each slice file is stored in a columnar storage mode; and the at least one slice file included in each physical file is stored in a row storage mode.
[0106] It can be understood that the spatial data within the slice file is stored in a columnar storage mode. From the perspective of the entire physical file, each slice file is equivalent to a logical row. Based on the spatial range index in the metadata description (the spatial range corresponding to each slice file), the logical row (i.e., slice file) corresponding to the spatial range to be queried can be quickly located.
[0107] In the disclosed embodiment, each physical file as a whole is a combination of row-based and column-based storage. This allows for high compression and facilitates the positioning of local data blocks.
[0108] On the basis of the above technical solution, optionally, combined with Figure 3 ,like Figure 4 As shown, the above S204 may further include:
[0109] S204d: Based on the local coordinates of the corresponding spatial grid, locally encode the spatial data in each data set to obtain the local data encoding of the spatial data in the corresponding data set.
[0110] Each local data code is obtained by removing the prefix from the one-dimensional data code of the corresponding spatial data.
[0111] It can be understood that the prefix is the one-dimensional code corresponding to the spatial grid, and the local data code is the one-dimensional data code of the corresponding spatial data minus the one-dimensional code of the spatial grid to which the spatial data belongs.
[0112] It should be noted that in order to facilitate the management of spatial data in the physical file corresponding to the spatial grid, and to better illustrate the relationship between one-dimensional data encoding, one-dimensional data encoding, and local data encoding, the concepts of parent grid and child grid are introduced. Among them, each spatial grid is a parent grid, and the spatial range corresponding to each spatial data under the spatial grid is the child grid of the corresponding spatial grid. The encoding of the parent grid is a one-dimensional encoding, and the encoding of the child grid is a one-dimensional data encoding. The encoding of the parent grid (one-dimensional encoding) is the prefix of the child grid encoding (one-dimensional data encoding). The local data encoding is the subgrid encoding (one-dimensional data encoding) minus the prefix of the parent grid encoding (one-dimensional encoding).
[0113] The term "subgrid" is a logical concept, and the code corresponding to a subgrid is essentially equivalent to the subgrid's globally unique code. Theoretically, the spatial range corresponding to each piece of spatial data can be equivalent to a subgrid of the spatial grid to which the spatial data belongs. However, in practice, since the subgrid corresponding to the spatial range of spatial data is relatively small, the spatial grid can be first divided into multiple subgrids (i.e., the physical file is divided into multiple slice files), each of which contains multiple smaller subgrids (i.e., the spatial data under each slice file, i.e., the subgrid corresponding to the spatial data under the subgrid. For ease of distinction, the subgrid corresponding to the spatial data is referred to as a leaf grid below).
[0114] It can be understood that each slice file corresponds to a sub-grid in the spatial grid, and the same encoding rules are used for dimensionality reduction encoding of spatial grids and sub-grids. However, for grids of different sizes, the larger the grid, the shorter the encoded string, and the smaller the grid, the longer the encoded string; at the same time, if a small grid belongs to a large grid, then the code of the large grid must be a prefix of the code of the small grid. This is a feature of ZOrder type encoding.
[0115] Since the method of dimensionality reduction encoding for spatial data is the same as that for sub-grids, the spatial range of spatial data belongs to the spatial range of the corresponding sub-grid. The one-dimensional data encoding actually includes the globally unique encoding of the sub-grid (slice file) to which the spatial range of the corresponding spatial data belongs. The encoding of the corresponding sub-grid is the prefix of the one-dimensional data encoding of the spatial data.
[0116] It can be understood that the code of the spatial grid is the prefix of the code of each sub-grid, and the code of the sub-grid is the prefix of the code of each leaf grid (at the same time, the code of the spatial grid is also the prefix of the code of each leaf grid). Exemplarily, in the embodiment of the present disclosure, the prefix is defined as the prefix of the code of the sub-grid, and the prefix of the code of the leaf grid. The local data code is the code obtained by removing the one-dimensional code of the spatial grid from the one-dimensional data code of each spatial data in the spatial grid. The local data code includes both a unique code segment indicating the sub-grid to which the spatial data belongs and a unique code segment indicating the spatial data.
[0117] For example, assuming the target encoding rule is ZOrder encoding, assuming the global data space is divided into 300*300 meter grids and dimensionality reduction encoding is performed according to the target encoding rule, the GridKey of one of the grids is 0030110321000223030. Assuming that there are four 150*150 meter subgrids in this grid (in practice, the sizes of each subgrid can be the same or different, depending on the actual situation and not limited here), the corresponding codes of each subgrid are: 00301103210002230300, 00301103210002230301, 00301103210002230302, and 00301103210002230303. It can be seen that the prefix of the subgrid is the parent grid, which is a prefix matching relationship. Assuming that the sub-grid 00301103210002230300 includes 2 spatial data, the one-dimensional data code corresponding to each spatial data is 003011032100022303001 and 003011032100022303002. It can be obtained that the sub-grid code is the prefix of the spatial data code, and the parent grid code is also the prefix of the spatial data code.
[0118] Exemplarily, following the above example, the local data codes corresponding to each subgrid in the spatial grid are 0, 1, 2, and 3, respectively, wherein the local data codes corresponding to the subgrids uniquely indicate a subgrid; the local data codes of each spatial data in the subgrid 00301103210002230300 are 01 and 02, wherein the local data code "01" of the spatial data includes both the unique code segment "0" of the subgrid to which the spatial data belongs, and the code segment "1" that uniquely indicates the spatial data in the subgrid; the local data code "02" of the spatial data includes both the unique code segment "0" of the subgrid to which the spatial data belongs, and the code segment "2" that uniquely indicates the spatial data in the subgrid.
[0119] In the embodiment of the present disclosure, the local data encoding of each spatial data obtained by local encoding can not only represent the sub-grid in the spatial grid to which the spatial data belongs, but also effectively reduce the length of the encoded data actually required to be stored, thereby saving storage space.
[0120] S204e: Sort the spatial data in each slice file according to the corresponding local data code.
[0121] It can be understood that the spatial data belonging to the spatial grid is re-encoded based on the local coordinates relative to the spatial grid to generate a local data code in the local coordinate system. The spatial data in each slice file is sorted according to the order of the local data codes. This makes it easier to quickly find the spatial data to be queried in the slice file, which can improve retrieval efficiency.
[0122] Based on the above technical solution, optionally, the query filtering condition further includes an attribute filtering condition; the above S209 may specifically include:
[0123] S209a: Based on the spatial range of each slice file in the metadata description of each target physical file, determine at least one target slice file that meets the spatial range to be queried from the at least one target physical file.
[0124] S209b: Based on the non-spatial attribute information of the spatial data in each slice file in the metadata description of each target physical file, determine the spatial data to be queried that meets the attribute filtering condition from the at least one target slice file.
[0125] The attribute filtering condition is the filtering condition corresponding to the non-spatial attribute information.
[0126] It can be understood that in the embodiment of the present disclosure, the dimensionality reduction coding of the spatial range to be queried and the invalid coding of the spatial range to be queried are first performed, and then the spatial grid corresponding to the spatial range to be queried is determined based on the correspondence between the valid coding of the spatial range to be queried (the coding to be queried) and the one-dimensional coding of the spatial grid in the global coordinate system, that is, at least one target physical file corresponding to the spatial data to be queried is queried (first-level retrieval is completed), and then the spatial range index in the metadata description of the at least one target physical file is accessed, and based on the spatial range to be queried, at least one target slice file is determined from the at least one target physical file, and finally the non-spatial attribute index in the metadata description of the at least one target physical file is accessed, and the data to be queried is determined from the at least one target slice file. In this way, the combination of the spatial range to be queried and the attribute filtering condition realizes the rapid indexing of the spatial data to be queried.
[0127] For example, Figure 5 As shown, the metadata description (FileFooter) is the metadata description of the entire physical file. On the one hand, FileFooter stores the offset (offset), length (length) and other location information of each stripe in the physical file, as well as the non-spatial attribute information (attribute) of the spatial data in each stripe. On the other hand, FileFooter stores statistical information of the physical file as a whole and at the stripe level, such as the actual spatial range of each stripe and the actual spatial range of the entire physical file, thereby constructing a small spatial index tree for fast data query and positioning. Among them, the root node (Root) is the spatial index tree of the actual spatial range of each stripe stored in FileFooter. Based on the Root, the stripe corresponding to the search range can be determined, and then the data block corresponding to the stripe corresponding to the search range can be found based on the location information of each stripe in FileFooter. Then, based on the non-spatial attribute information of the spatial data in the stripe in FileFooter, the final spatial data to be queried that meets the attribute filtering conditions can be determined.
[0128] In the embodiment of the present disclosure, based on the dimensionality reduction coding scheme, the data space is divided into standard spatial grids, each spatial grid corresponds to a physical file on the distributed storage. The spatial data is stored in the corresponding physical file according to the spatial ownership relationship. At the same time, a secondary index is constructed in each grid to perform filtering and retrieval within the grid. Finally, a set of Figure 5The indexing scheme for the dimensionality reduction grid shown in the figure. This scheme, based on the dimensionality reduction encoding scheme, enables sharded storage of spatial data on a distributed file system. The sharded storage is located in real time based on the encoding function, eliminating the need to maintain a large amount of index information and removing the reliance on neutralization. Furthermore, by constructing local secondary indexes within the physical files corresponding to the spatial grid, they can be used for secondary retrieval and filtering within the physical files, ensuring overall efficiency.
[0129] In the embodiments of this application, a new indexing scheme for real-time retrieval and filtering of massive spatial data, based on dimensionality reduction coding and multi-level indexing, is designed on a distributed file system, combining the characteristics of spatial data and spatial queries. This scheme can support spatial retrieval and filtering capabilities in the hundreds of milliseconds, even at a data scale of petabytes.
[0130] It should be noted that this solution uses open source software for spatial data analysis and only uses binary programs, without involving code modification.
[0131] Figure 6 This is a schematic structural diagram of a spatial data retrieval device provided in an embodiment of the present disclosure. The device can be implemented using software and / or hardware and can be integrated into any electronic device with computing capabilities.
[0132] like Figure 6 As shown, the spatial data retrieval device 600 provided by the embodiment of the present disclosure may include a receiving module 601, an encoding module 602, a screening module 603 and a determining module 604, wherein:
[0133] A receiving module 601 is configured to receive a search request for spatial data to be queried, the search request including a query filter condition corresponding to the spatial data to be queried, the query filter condition including a spatial range to be queried;
[0134] The encoding module 602 is configured to perform dimensionality reduction encoding on the query space range according to a target encoding rule to obtain multiple one-dimensional codes. The target encoding rule is a spatial encoding rule of a distributed database. Each one-dimensional code is used to indicate a physical file in the distributed database, and each physical file includes at least one spatial data.
[0135] The screening module 603 is configured to screen out at least one code to be queried from the multiple one-dimensional codes obtained by the encoding module, wherein the spatial range corresponding to each code to be queried overlaps with the spatial range to be queried;
[0136] The determining module 604 is configured to determine at least one target physical file from the distributed database, where each target physical file is a physical file in the distributed database corresponding to a code to be queried obtained by the screening module;
[0137] The determining module 604 is further configured to determine, from the at least one target physical file, the spatial data to be queried that meets the query filtering condition.
[0138] Optionally, the spatial data retrieval device 600 further includes: a partitioning module and a storage module; wherein,
[0139] The partitioning module is used to divide the data space in the global coordinate system into multiple spatial grids of preset sizes;
[0140] The encoding module 602 is further configured to perform dimensionality reduction encoding on each spatial grid obtained by the division module according to the target encoding rule to obtain a one-dimensional code corresponding to each spatial grid;
[0141] The encoding module 602 is further configured to perform dimensionality reduction encoding on each spatial data to be stored based on the target encoding rule to obtain a one-dimensional data code corresponding to each spatial data to be stored, wherein the prefix of each one-dimensional data code is a one-dimensional code corresponding to a spatial grid;
[0142] The storage module is used to store each spatial data to be stored in a physical file corresponding to the spatial grid indicated by the prefix of the corresponding one-dimensional data code to obtain the distributed database, and the identifier of each physical file is the one-dimensional code of the corresponding spatial grid.
[0143] Optionally, the storage module is specifically used to:
[0144] Dividing each spatial data to be stored into multiple data sets, wherein the prefix of the one-dimensional data code corresponding to the spatial data in each data set is the same;
[0145] Counting metadata descriptions of each data set, where the metadata descriptions of each data set include: spatial extent of each spatial data in the corresponding data set, and non-spatial attribute information of each spatial data;
[0146] Based on the spatial data in each data set and the corresponding metadata description, a corresponding physical file is generated to obtain the distributed database;
[0147] The metadata description in each physical file is an index of the spatial data in the corresponding physical file.
[0148] Optionally, each physical file includes at least one slice file, and each slice file stores at least one spatial data in the corresponding physical file;
[0149] The metadata description in each physical file includes: the spatial range of each slice file corresponding to the physical file, and non-spatial attribute information of the spatial data in each slice file corresponding to the physical file.
[0150] Optionally, the at least one spatial data in each slice file is stored in a columnar storage mode;
[0151] The at least one slice file included in each physical file is stored in a row-based storage mode.
[0152] Optionally, the storage module is further configured to:
[0153] Based on the local coordinates of the corresponding spatial grid, the spatial data in each data set is locally encoded to obtain the local data code of the spatial data in the corresponding data set, where each local data code is obtained by removing the prefix of the one-dimensional data code of the corresponding spatial data;
[0154] The spatial data in each slice file is sorted according to the corresponding local data code.
[0155] Optionally, the query filter condition also includes an attribute filter condition;
[0156] The determining module 604 is specifically configured to:
[0157] Determining, from the at least one target physical file, based on the spatial range of each slice file in the metadata description of each target physical file, at least one target slice file that satisfies the spatial range to be queried;
[0158] Based on the non-spatial attribute information of the spatial data in each slice file in the metadata description of each target physical file, the spatial data to be queried that meets the attribute filtering condition is determined from the at least one target slice file.
[0159] The spatial data retrieval device provided in the embodiments of the present disclosure can execute any spatial data retrieval method provided in the embodiments of the present disclosure, and has the corresponding functional modules and beneficial effects of the execution method. For any content not fully described in the embodiments of the present disclosure, please refer to the description of any method embodiment of the present disclosure.
[0160] Figure 7 This is a structural diagram of an electronic device provided in an embodiment of the present disclosure, which is used to exemplify an electronic device that implements any spatial data retrieval method in an embodiment of the present disclosure and should not be understood as a specific limitation on the embodiments of the present disclosure.
[0161] like Figure 7As shown, the electronic device 700 may include a processor (e.g., a CPU, a graphics processor, etc.) 701, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 702 or a program loaded from a storage device 708 into a random access memory (RAM) 703. Various programs and data required for the operation of the electronic device 700 are also stored in the RAM 703. The processor 701, the ROM 702, and the RAM 703 are connected to each other via a bus 704. An input / output (I / O) interface 705 is also connected to the bus 704.
[0162] Typically, the following devices may be connected to the I / O interface 705: an input device 706 including, for example, a touch screen, a touchpad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 707 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 708 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 709. The communication device 709 may allow the electronic device 700 to communicate with other devices wirelessly or by wire to exchange data. Although the electronic device 700 is shown as having various devices, it should be understood that it is not required to implement or have all of the devices shown. More or fewer devices may be implemented or have alternatively.
[0163] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a non-transitory computer-readable medium, and the computer program includes program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network via the communication device 709, or installed from the storage device 708, or installed from the ROM 702. When the computer program is executed by the processor 701, the functions defined in any spatial data retrieval method provided by the embodiment of the present disclosure can be executed.
[0164] It should be noted that the computer-readable medium mentioned above in the present disclosure may be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. Computer-readable storage media may be, for example, but not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or components, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, device, or component. In the present disclosure, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, which carries computer-readable program code. Such a propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transport a program for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium may be transmitted using any suitable medium, including but not limited to wires, optical cables, RF (radio frequency), etc., or any suitable combination thereof.
[0165] In some embodiments, the client and server can communicate using any currently known or future developed network protocol, such as HTTP (HyperText Transfer Protocol), and can be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), an internet (e.g., the Internet), and a peer-to-peer network (e.g., an ad hoc peer-to-peer network), as well as any currently known or future developed network.
[0166] The computer-readable medium may be included in the electronic device, or may exist independently without being incorporated into the electronic device.
[0167] The computer-readable medium carries one or more programs. When the one or more programs are executed by the electronic device, the electronic device: receives a retrieval request for spatial data to be queried, the retrieval request includes a query filter condition corresponding to the spatial data to be queried, and the query filter condition includes a spatial range to be queried; performs dimensionality reduction coding on the spatial range to be queried according to a target coding rule to obtain multiple one-dimensional codes, the target coding rule is a spatial coding rule of a distributed database, each one-dimensional code is used to indicate a physical file in the distributed database, and each physical file includes at least one spatial data; selects at least one code to be queried from the multiple one-dimensional codes, and the spatial range corresponding to each code to be queried overlaps with the spatial range to be queried; determines at least one target physical file from the distributed database, each target physical file being a physical file in the distributed database corresponding to one of the codes to be queried; and determines the spatial data to be queried that meets the query filter condition from the at least one target physical file.
[0168] In embodiments of the present disclosure, computer program code for performing the operations of the present disclosure may be written in one or more programming languages or a combination thereof, including but not limited to object-oriented programming languages such as Java, Smalltalk, C++, and conventional procedural programming languages such as "C" or similar programming languages. The program code may be executed entirely on the computer, partially on the computer, as a separate software package, partially on the computer and partially on a remote computer, or entirely on the remote computer or server. In cases involving a remote computer, the remote computer may be connected to the computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).
[0169] The flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram can represent a module, program segment, or a part of code, and the module, program segment, or a part of code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order than that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of the boxes in the block diagram and / or flowchart, can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.
[0170] The units involved in the embodiments described in this disclosure may be implemented in software or hardware, wherein the name of a unit does not necessarily limit the unit itself.
[0171] The functions described above herein may be performed, at least in part, by one or more hardware logic components. For example, and without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chip (SOCs), complex programmable logic devices (CPLDs), and the like.
[0172] In the context of the present disclosure, a computer-readable medium can be a tangible medium that can contain or store a program for use by an instruction execution system, device or equipment or used in combination with an instruction execution system, device or equipment. A computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. A computer-readable medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or equipment, or any suitable combination of the foregoing. A more specific example of a computer-readable storage medium can include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
[0173] The above description is merely a preferred embodiment of the present disclosure and an illustration of the technical principles employed. Those skilled in the art should understand that the scope of disclosure involved in the present disclosure is not limited to the technical solutions formed by the specific combination of the above-mentioned technical features, but also includes other technical solutions formed by any combination of the above-mentioned technical features or their equivalents without departing from the above-mentioned disclosed concepts. For example, a technical solution formed by replacing the above-mentioned features with (but not limited to) technical features with similar functions disclosed in this disclosure.
[0174] In addition, although each operation is described in a specific order, this should not be understood as requiring these operations to be performed in the specific order shown or in a sequential order. Under certain circumstances, multitasking and parallel processing may be advantageous. Similarly, although some specific implementation details have been included in the above discussion, these should not be interpreted as limiting the scope of the present disclosure. Some features described in the context of a separate embodiment can also be implemented in a single embodiment in combination. On the contrary, the various features described in the context of a single embodiment can also be implemented in multiple embodiments individually or in any suitable sub-combination mode.
[0175] Although the subject matter has been described in language specific to structural features and / or methodological logical acts, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are merely example forms of implementing the claims.
Claims
1. A spatial data retrieval method, comprising: receiving a search request for spatial data to be queried, the search request including a query filter condition corresponding to the spatial data to be queried, the query filter condition including a spatial range to be queried; Performing dimensionality reduction coding on the spatial range to be queried according to a target coding rule to obtain multiple one-dimensional codes, wherein the target coding rule is a spatial coding rule of a distributed database, each one-dimensional code is used to indicate a physical file in the distributed database, and each physical file includes at least one spatial data; Restoring each of the multiple one-dimensional codes to a corresponding spatial range, and when the spatial range restored from the one-dimensional code overlaps with the spatial range to be queried, determining the one-dimensional code as a code to be queried; Determine at least one target physical file from the distributed database, each target physical file being a physical file in the distributed database corresponding to one of the codes to be queried; The spatial data to be queried that meets the query filter condition is determined from the at least one target physical file.
2. The method according to claim 1, wherein Before receiving the search request for the spatial data to be queried, the method further includes: Divide the data space in the global coordinate system into multiple spatial grids of preset sizes; Performing dimensionality reduction coding on each spatial grid according to the target coding rule to obtain a one-dimensional code corresponding to each spatial grid; Based on the target coding rule, each spatial data to be stored is subjected to dimensionality reduction coding to obtain a one-dimensional data code corresponding to each spatial data to be stored, wherein the prefix of each one-dimensional data code is a one-dimensional code corresponding to a spatial grid; Each of the spatial data to be stored is stored in a physical file corresponding to the spatial grid indicated by the prefix of the corresponding one-dimensional data code to obtain the distributed database, wherein the identifier of each physical file is the one-dimensional code of the corresponding spatial grid.
3. The method according to claim 2, wherein: The step of storing each of the spatial data to be stored in a physical file corresponding to the spatial grid indicated by the prefix of the corresponding one-dimensional data code to obtain the distributed database includes: Dividing each of the spatial data to be stored into multiple data sets, wherein the prefixes of the one-dimensional data codes corresponding to the spatial data in each data set are the same; Counting metadata descriptions of each data set, wherein the metadata descriptions of each data set include: spatial range of each spatial data in the corresponding data set, and non-spatial attribute information of each spatial data; Based on the spatial data in each data set and the corresponding metadata description, a corresponding physical file is generated to obtain the distributed database; The metadata description in each physical file is an index of the spatial data in the corresponding physical file.
4. The method according to claim 3, wherein: Each physical file includes at least one slice file, and each slice file stores at least one spatial data in the corresponding physical file; The metadata description in each physical file includes: the spatial extent of each slice file corresponding to the physical file, and non-spatial attribute information of the spatial data in each slice file corresponding to the physical file.
5. The method according to claim 4, wherein The at least one spatial data in each slice file is stored in a columnar storage mode; The at least one slice file included in each physical file is stored in a row-based storage mode.
6. The method according to claim 4 or 5, wherein: The method further comprises: Based on the local coordinates of the corresponding spatial grid, the spatial data in each data set is locally encoded to obtain local data codes of the spatial data in the corresponding data set, where each local data code is obtained by removing a prefix from the one-dimensional data code of the corresponding spatial data; The spatial data in each slice file is sorted according to the corresponding local data code.
7. The method according to claim 4 or 5, wherein: The query filtering conditions also include attribute filtering conditions; The determining, from the at least one target physical file, the spatial data to be queried that satisfies the query filtering condition includes: Determining, from the at least one target physical file, at least one target slice file that satisfies the spatial range to be queried based on the spatial range of each slice file in the metadata description of each target physical file; Based on the non-spatial attribute information of the spatial data in each slice file in the metadata description of each target physical file, the spatial data to be queried that meets the attribute filtering condition is determined from the at least one target slice file.
8. A spatial data retrieval device comprising: Receiving module, encoding module, screening module and determining module; The receiving module is configured to receive a search request for the spatial data to be queried, wherein the search request includes a query filter condition corresponding to the spatial data to be queried, and the query filter condition includes a spatial range to be queried; The encoding module is used to perform dimensionality reduction encoding on the query space range according to a target encoding rule to obtain multiple one-dimensional codes, wherein the target encoding rule is a spatial encoding rule of a distributed database, each one-dimensional code is used to indicate a physical file in the distributed database, and each physical file includes at least one spatial data; The screening module is configured to restore each of the multiple one-dimensional codes to a corresponding spatial range, and when the spatial range restored from the one-dimensional code overlaps with the spatial range to be queried, determine the one-dimensional code as a code to be queried; The determining module is configured to determine at least one target physical file from the distributed database, each target physical file being a physical file in the distributed database corresponding to a code to be queried obtained by the screening module; The determining module is further configured to determine, from the at least one target physical file, the spatial data to be queried that meets the query filtering condition.
9. An electronic device comprising a memory and a processor, wherein the memory is used to store the processor-executable instructions; the processor is used to read the executable instructions from the memory and execute the executable instructions to implement the spatial data retrieval method according to any one of claims 1 to 7.
10. A computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the spatial data retrieval method according to any one of claims 1 to 7.
Citation Information
Patent Citations
Data query method and device, index generation method and device and electronic equipment
CN113076334A