Method and equipment for gridding storage, index construction and query of space-time big data
By establishing the correspondence between spatiotemporal data and spatial pyramid grid and building a multi-modal index model, the problem of low efficiency of multi-source spatiotemporal big data analysis is solved, efficient data retrieval and unified management are achieved, and user experience is improved.
Patent Information
- Application Number
- CN202410871233.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-01
- Publication Date
- 2024-11-26
AI Technical Summary
It is difficult for the existing technology to use multi-source spatiotemporal big data for efficient analysis at the same time. It is necessary to search various types of data separately and analyze and study based on spatiotemporal relationships.
By establishing the correspondence between spatiotemporal data and spatial pyramid grids, performing grid storage and downsampling processing, building a global spatiotemporal data pyramid, and using GeoHash algorithm and quadtree to organize data, combining Flume, Kafka and HDFS, designing Real-time index construction and query methods to implement multi-modal composite indexing models and fast query.
It realizes the unified organization and management of multi-source spatiotemporal big data, improves the availability and discoverability of data, improves data retrieval and analysis efficiency, and improves query response speed and user experience.
Smart Images

Figure CN119025692A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of spatiotemporal big data processing, and in particular to a grid storage, index construction and query method and device for spatiotemporal big data. Background Art
[0002] Spatiotemporal big data is the integration of big data and geographic spatiotemporal data. It is based on the Earth, a unified spatiotemporal benchmark, and is directly or indirectly related to location in time and space. It includes three-dimensional information such as time, space, and thematic attributes. In the field of natural resource spatiotemporal big data, the key issue is how to provide storage and multi-dimensional and efficient data retrieval methods for multi-source spatiotemporal big data including vector data and remote sensing image data.
[0003] In the prior art, a spatial pyramid is constructed for each source of spatiotemporal data for data storage. Spatial pyramid technology is an image processing method based on a multi-resolution hierarchical model. It realizes the ability to process and analyze images at different resolutions by dividing and layering images. By constructing multiple resolution levels, spatial pyramid technology can more effectively represent the characteristics and structure of images, thereby providing richer information for image processing and analysis.
[0004] However, based on the existing storage methods, if users want to use spatiotemporal big data from different sources for analysis at the same time, they need to first search the data from different sources separately, and then conduct analysis and research based on the spatiotemporal relationship between the multi-source data. Summary of the invention
[0005] In order to solve the above problems in the prior art, the present invention proposes a grid storage, index construction and query method and device for spatiotemporal big data, which improves the efficiency of retrieval and analysis.
[0006] In a first aspect of the present invention, a grid storage method for spatiotemporal big data is proposed, the storage method comprising:
[0007] According to the resolution and spatial range of multi-source spatiotemporal data, the correspondence between spatiotemporal data and the grid of spatial pyramid is established;
[0008] According to the corresponding relationship, the multi-source spatiotemporal data are copied to corresponding positions in the grid of the spatial pyramid to generate a grid at the highest resolution level;
[0009] Based on the grid at the highest resolution level, the spatiotemporal data is downsampled step by step to obtain grid data at a lower resolution level, thereby constructing a global spatiotemporal data pyramid.
[0010] Preferably, establishing a correspondence between the spatiotemporal data and the grids of the spatial pyramid according to the resolution and spatial range of the multi-source spatiotemporal data includes:
[0011] Encoding each grid in the spatial pyramid according to level, row number and column number;
[0012] The spatial resolution corresponding to a grid is calculated according to the following formula:
[0013]
[0014] Among them, gridRes represents the resolution of the grid, lon max and lon min They represent the maximum and minimum values of the longitude range in the current coordinate system, level represents the level of the grid, and gridSize represents the size of the grid;
[0015] Calculate the row and column numbers of the grid based on the spatial extent of the spatiotemporal data:
[0016]
[0017]
[0018] span = gridRes × gridSize
[0019] Among them, row and col represent the row and column numbers of the grid respectively, lon and lat represent the longitude and latitude of the center of the grid respectively, lon min ,lat max They represent the minimum longitude and maximum latitude of the current coordinate system respectively, and span represents the size of the longitude range and latitude range represented by each grid.
[0020] Preferably, the geographic coordinate system of the grid in the spatial pyramid is WGS-84 (World Geodetic System-1984 Coordinate System, a coordinate system established for use by the GPS global positioning system) or CGCS2000 (China Geodetic Coordinate System 2000, 2000 National Geodetic Coordinate System), and the longitude and latitude grid is used as the division standard. The origin of the grid data in the spatial pyramid is set to (180°W, 90°N), and the GeoHash (an address encoding method that can encode two-dimensional spatial longitude and latitude data into a string) algorithm is used to encode the grid, and the quadtree is used to organize the data.
[0021] In a second aspect of the present invention, a method for constructing an index of spatiotemporal big data is proposed. The index construction method is based on a spatiotemporal data pyramid constructed by the grid storage method of spatiotemporal big data described above. The index construction method comprises:
[0022] Import data from the spatiotemporal data pyramid into a relational database, use Flume (a highly available, highly reliable, distributed system for massive log collection, aggregation and transmission provided by Cloudera) to collect self-increasing table data in the database in real time, and transmit it through the Kafka (an open source stream processing platform developed by the Apache Software Foundation) message queue;
[0023] Receive data from Kafka and parse it;
[0024] Determine the index type according to the parsed index metadata information, and create corresponding index data according to the index type;
[0025] The created index data is stored in HDFS (Hadoop Distributed File System) through the StoreFile mechanism (a persistent storage file in HBase, used to store written data);
[0026] in,
[0027] The index type includes: spatiotemporal index or non-spatiotemporal index;
[0028] The spatiotemporal index includes: a time period index, a space priority index or a time priority index;
[0029] The non-temporal and spatial index includes: a single-attribute index or a multi-attribute index.
[0030] Preferably, the creating corresponding index data according to the index type includes:
[0031] If the index type is the time period index, then create corresponding daily partition (per day) jump table time and space index data;
[0032] If the index type is the space-priority index, then create corresponding space-priority partition skip list spatiotemporal index data;
[0033] If the index type is the time-first index, then create corresponding time-first partition skip table time-space index data;
[0034] If the index type is the single-attribute index, then create corresponding single-attribute index data;
[0035] If the index type is the multi-attribute index, corresponding multi-attribute joint index data is created.
[0036] In a third aspect of the present invention, a query method for spatiotemporal big data is proposed. The query method is based on index data constructed by the index construction method for spatiotemporal big data described above. The query method comprises:
[0037] Determine the query type according to the query request; the query type includes: spatiotemporal query and / or non-spatial query;
[0038] If the query type includes the spatiotemporal query, partition screening is performed, and then a local search is performed in the screened partition to obtain a first data set;
[0039] If the query type includes the non-temporal and spatial query, performing a single attribute retrieval or a multi-attribute retrieval to obtain a second data set;
[0040] combining the first data set and the second data set into a candidate set;
[0041] The candidate sets are integrated to generate query results.
[0042] Preferably, the spatiotemporal query includes: time period query, space priority query and / or time priority query;
[0043] The performing of partition screening and then performing local search in the screened partition to obtain the first data set includes:
[0044] If the query type includes the time period query, partition screening is performed according to the daily partition jump table time and space index data;
[0045] If the query type includes the space-first query, partition screening is performed according to the space-first partition skip table spatiotemporal index data;
[0046] If the query type includes the time-first query, partition screening is performed according to the time-first partition skip table temporal and spatial index data;
[0047] A local skip table index is performed in the screened partitions to obtain the first data set.
[0048] Preferably, the non-spatiotemporal query includes: a single attribute query or the multi-attribute query;
[0049] The performing of single attribute retrieval or multi-attribute retrieval to obtain a second data set includes:
[0050] If the query type is the single-attribute query, the second data set is retrieved according to the single-attribute index data; otherwise, the second data set is retrieved through the multi-attribute joint index data.
[0051] According to a fourth aspect of the present invention, a processing device is provided, comprising a memory and a processor, wherein the memory stores a computer program that can be loaded by the processor and execute the above method.
[0052] According to a fifth aspect of the present invention, a computer-readable storage device is provided, storing a computer program that can be loaded by a processor and execute the above method.
[0053] The present invention has the following beneficial effects:
[0054] The grid storage method for spatiotemporal big data proposed in the present invention breaks down the barriers between spatiotemporal data from different sources and structures in the grid structure of the same spatiotemporal pyramid, effectively integrates and manages various spatiotemporal data, realizes the unified organization and management of multi-source spatiotemporal big data, improves the availability and discoverability of data, and provides a guarantee for improving the efficiency of data retrieval and analysis.
[0055] The GeoHash algorithm is introduced to achieve efficient encoding and query of geographic spatial information and improve the retrieval efficiency of spatiotemporal data.
[0056] The spatiotemporal big data index construction method of the present invention designs a real-time data index construction process, including the preparation work before index construction, the index construction process and the storage of indexed data. A multi-mode combined index model is proposed, including space-first partition skip table spatiotemporal index, time-first partition skip table spatiotemporal index and day-partition skip table spatiotemporal index, to achieve efficient retrieval of data.
[0057] The spatiotemporal big data query method of the present invention realizes rapid response to query requests and accurate matching of index data, thereby improving user experience. BRIEF DESCRIPTION OF THE DRAWINGS
[0058] Figure 1 It is a schematic diagram comparing the structures of a traditional space pyramid and the space pyramid of the present invention;
[0059] Figure 2 It is a schematic diagram of the main steps of an embodiment of the grid storage method for spatiotemporal big data in the present invention;
[0060] Figure 3 It is a schematic diagram of the main steps of an embodiment of the method for constructing an index of spatiotemporal big data in the present invention;
[0061] Figure 4 It is a schematic diagram of the main steps of an embodiment of the query method for spatiotemporal big data in the present invention. DETAILED DESCRIPTION
[0062] The preferred embodiments of the present invention are described below with reference to the accompanying drawings. It should be understood by those skilled in the art that these embodiments are only used to explain the technical principles of the present invention and are not intended to limit the protection scope of the present invention.
[0063] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0064] It should be noted that, in the description of the present invention, the terms "first" and "second" are only for the convenience of description, and do not indicate or imply the relative importance of the devices, elements or parameters, and therefore cannot be understood as limiting the present invention. In addition, the term "and / or" in the present invention is only a description of the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article, unless otherwise specified, generally indicates that the associated objects before and after are in an "or" relationship.
[0065] Figure 1 Schematic diagram of the comparison between the traditional space pyramid and the space pyramid structure of the present invention. Figure 1 As shown, the grid diagrams on the left and right are respectively schematic diagrams of pyramids constructed according to traditional methods using two types of spatiotemporal data from different sources, and the grid diagram in the middle is a schematic diagram of the structure after the two types of data are integrated into the same spatial pyramid in the present invention.
[0066] In the present invention, under the same spatiotemporal data grid, the classification and association of multi-source spatiotemporal data are realized based on the spatiotemporal characteristics of the data, which can break the barriers between spatiotemporal data from different sources and structures, for example, including but not limited to remote sensing image data, drone acquisition data, and vector format data. The organizational structure of the global spatiotemporal data set can be used to effectively integrate and manage various spatiotemporal data, thereby realizing the unified organization and management of multi-source spatiotemporal big data. Drawing on the ideas of traditional pyramid technology, a spatiotemporal data grid model based on the spatial pyramid is established. The model disperses data onto data grids of different scales through unified rules, and associates spatiotemporal information on these grids. By utilizing the spatiotemporal attributes of the data, the data structure is optimized, so that the spatiotemporal characteristics of the data at different scales can be better displayed and analyzed.
[0067] In the embodiment of the present invention, the geographic coordinate system of the grid in the spatial pyramid is WGS-84 or CGCS2000, the latitude and longitude grid is used as the division standard, the origin of the grid data in the spatial pyramid is set to (180°W, 90°N), the GeoHash algorithm is used to encode the grid, and the quadtree is used to organize the data.
[0068] GeoHash is an algorithm that encodes geographic coordinate systems through regular grids. The algorithm can express two-dimensional geographic spatial information in the form of a one-dimensional string, and has the characteristics of uniqueness and spatial recursion. The GeoHash algorithm can convert geographic coordinates into more accurate, smaller area codes, and finally convert binary sequences into string form through the Base-32 encoding rule, so as to achieve accurate description and fast retrieval of geographic locations.
[0069] In the field of big data applications, data may encounter uneven distribution problems in distributed architectures, which in turn affects the efficiency of single-index retrieval mechanisms. Data skew not only increases the pressure on server nodes, but also increases the demand for cross-node queries, thereby consuming more storage bandwidth and computing resources. In view of the storage and management challenges and low retrieval efficiency caused by data skew, developing and optimizing combined index models has become a key research direction in this field. The query performance of combined indexes is often unsatisfactory, which is mainly reflected in the lack of full consideration of more detailed partition-level indexes and the high time cost of partition index updates. In order to enhance retrieval efficiency, it is necessary to further refine the granularity of indexes, reduce the time overhead required for index updates, and seek more effective index combination strategies.
[0070] Figure 2 Schematic diagram of the main steps of the grid storage method of spatiotemporal big data in the present invention. Figure 2 As shown, the storage method of this embodiment includes steps A10-A30:
[0071] Step A10, establishing a correspondence between the spatiotemporal data and the grid of the spatial pyramid according to the resolution and spatial range of the multi-source spatiotemporal data. This step may specifically include steps A11-A13:
[0072] Step A11, encode each grid in the spatial pyramid according to the level, row number and column number.
[0073] Step A12, calculate the spatial resolution corresponding to a grid according to the method shown in formula (1):
[0074]
[0075] Among them, gridRes represents the resolution of the grid, lon max and lonmin They represent the maximum and minimum values of the longitude range in the current coordinate system, level represents the level of the grid, and gridSize represents the size of the grid.
[0076] Step A13, calculate the row and column numbers of the corresponding grid according to the spatial range of the spatiotemporal data, as shown in the following formulas (2)-(4):
[0077]
[0078] span=gridRes×gridSize (4)
[0079] Among them, row and col represent the row and column numbers of the grid respectively, lon and lat represent the longitude and latitude of the center of the grid respectively, lon min ,lat max They represent the minimum longitude and maximum latitude of the current coordinate system respectively, and span represents the size of the longitude range and latitude range represented by each grid.
[0080] Step A20, based on the corresponding relationship, copy the multi-source spatiotemporal data to the corresponding position in the grid of the spatial pyramid to generate a grid at the highest resolution level.
[0081] Step A30, based on the grid at the highest resolution level, downsample the spatiotemporal data step by step to obtain grid data at a lower resolution level, thereby constructing a global spatiotemporal data pyramid.
[0082] Figure 3 Schematic diagram of the main steps of the index construction method of spatiotemporal big data in the present invention. The index construction method of this embodiment is based on the spatiotemporal data pyramid constructed by the grid storage method of spatiotemporal big data described above, such as Figure 3 As shown, the index construction method of this embodiment includes steps B10-B40:
[0083] Step B10, importing data from the spatiotemporal data pyramid into a relational database, using Flume to collect self-increasing table data in the database in real time, and transmitting it through the Kafka message queue.
[0084] Step B20, receiving data from Kafka and parsing it.
[0085] Step B30, determining the index type according to the parsed index metadata information, and creating corresponding index data according to the index type.
[0086] Among them, the index type includes: spatiotemporal index subtype or non-spatiotemporal index subtype; the spatiotemporal index subtype includes: time period index, space priority index or time priority index; the non-spatiotemporal index subtype includes: single attribute index or multi-attribute index.
[0087] Specifically, if the index type is a time period index, then create corresponding daily partition jump table spatiotemporal index data; if the index type is a space priority index, then create corresponding space priority partition jump table spatiotemporal index data; if the index type is a time priority index, then create corresponding time priority partition jump table spatiotemporal index data; if the index type is a single attribute index, then create corresponding single attribute index data; if the index type is a multi-attribute index, then create corresponding multi-attribute joint index data.
[0088] Skip Lists are an efficient data structure based on linked lists. By integrating multi-layer index structures into linked lists, fast query operations are achieved. This data structure combines the ideas of linked lists and binary search, which significantly improves query efficiency. The design of skip lists makes them highly concurrent and highly available, making them suitable for scenarios that require frequent queries. By using skip lists, faster query speeds can be achieved while maintaining a relatively small space complexity.
[0089] In the multi-level index query mechanism of the skip list, since the construction of the index layer has a certain degree of randomness, the number of query operations is usually proportional to the number of layers of the skip list, making the time complexity of querying a certain data element reach O(logN), where N is the total number of elements in the skip list. For operations such as insertion, update, and deletion, the elements in the linked list also need to be processed layer by layer, so the time complexity of these operations is also O(logN).
[0090] The number of node levels in the skip list is determined using a randomization strategy, that is, a level value between 1 and the maximum number of levels is randomly generated through the RandomLevel() function. This method eliminates the need for balance adjustments after the node is inserted, and effectively reduces the memory overhead that may be caused by level-crossing operations.
[0091] Compared with other tree index structures, skip lists may not have significant advantages in query efficiency, but their structure is simple and does not require complex balancing operations (such as those required by red-black trees or B-trees), thus simplifying the implementation difficulty. Considering the efficiency and structural complexity, skip lists are adopted by many open source software due to their simple structure and effective use of memory resources.
[0092] Step B40, storing the created index data in HDFS through the StoreFile mechanism.
[0093] Hadoop is an open source distributed computing framework designed to efficiently process massive data sets. Inspired by Google's MapReduce algorithm and Google File System (GFS), the framework aims to provide a reliable and efficient data storage and processing solution. Hadoop's core components include the HDFS (Hadoop Distributed File System) distributed file system and the MapReduce distributed computing model, which supports parallel processing of data on hundreds or even thousands of servers to achieve high reliability and scalability, thereby meeting the needs of large-scale data processing.
[0094] Figure 4 Schematic diagram of the main steps of the query method of spatiotemporal big data in the present invention. The query method of this embodiment is based on the index data constructed by the index construction method of spatiotemporal big data described above, such as Figure 4 As shown, the query method of this embodiment includes steps C10-C50:
[0095] Step C10: determining the query type according to the query request.
[0096] The query type includes: spatiotemporal query subtype and / or non-spatiotemporal query subtype.
[0097] Step C20: If the query type includes spatiotemporal query, partition screening is performed, and then a local search is performed within the screened partitions to obtain the first data set.
[0098] The subtypes of spatiotemporal query include: time period query, space priority query and / or time priority query. This step may specifically include steps C21-C24:
[0099] Step C21: If the query type includes time period query, partition screening is performed according to the daily partition jump table time and space index data.
[0100] Step C22: If the query type includes space-first query, partition screening is performed according to the space-first partition skip table spatiotemporal index data.
[0101] Step C23: If the query type includes time-priority query, partition screening is performed according to the time-priority partition skip table temporal and spatial index data.
[0102] Step C24, performing local skip table indexing in the screened partitions to obtain a first data set.
[0103] Step C30: If the query type includes non-spatial-temporal query, a single attribute search or a multi-attribute search is performed to obtain a second data set.
[0104] The non-spatial-temporal query subtypes include: single-attribute query or the multi-attribute query;
[0105] This step may specifically include:
[0106] If the query type is the single-attribute query, the second data set is retrieved according to the single-attribute index data; otherwise, the second data set is retrieved through the multi-attribute joint index data.
[0107] Step C40: combining the first data set and the second data set into a candidate set.
[0108] Step C50, integrating the candidate sets to generate query results.
[0109] Although the various steps in the above embodiment are described in the above-mentioned order, those skilled in the art can understand that in order to achieve the effect of this embodiment, different steps do not have to be executed in such an order. They can be executed simultaneously (in parallel) or in a reverse order. These simple changes are within the scope of protection of the present invention.
[0110] Furthermore, the present invention also provides an embodiment of a processing device. The processing device of this embodiment includes a memory and a processor. The memory stores a computer program that can be loaded by the processor and execute the method as described above.
[0111] Furthermore, the present invention also provides an embodiment of a computer-readable storage device, in which the storage device of this embodiment stores a computer program that can be loaded by a processor and execute the method described above.
[0112] The computer-readable storage device may include: a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and other media that can store program codes.
[0113] Those skilled in the art should be able to appreciate that the method steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of electronic hardware and software, the composition and steps of each example have been generally described in the above description according to function. Whether these functions are performed in electronic hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.
[0114] So far, the technical solutions of the present invention have been described in conjunction with the preferred embodiments shown in the accompanying drawings. However, it is easy for those skilled in the art to understand that the protection scope of the present invention is obviously not limited to these specific embodiments. Without departing from the principle of the present invention, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions after these changes or substitutions will fall within the protection scope of the present invention.
Claims
1. A grid storage method for spatiotemporal big data, characterized in that: The storage method comprises: According to the resolution and spatial range of multi-source spatiotemporal data, the correspondence between spatiotemporal data and the grid of spatial pyramid is established; According to the corresponding relationship, the multi-source spatiotemporal data are copied to corresponding positions in the grid of the spatial pyramid to generate a grid at the highest resolution level; Based on the grid at the highest resolution level, the spatiotemporal data is downsampled step by step to obtain grid data at a lower resolution level, thereby constructing a global spatiotemporal data pyramid.
2. The grid storage method for spatiotemporal big data according to claim 1 is characterized in that: The establishing of a correspondence between the spatiotemporal data and the grid of the spatial pyramid according to the resolution and spatial range of the multi-source spatiotemporal data includes: Encoding each grid in the spatial pyramid according to level, row number and column number; The spatial resolution corresponding to a grid is calculated according to the following formula: Among them, gridRes represents the resolution of the grid, lon max and lon min They represent the maximum and minimum values of the longitude range in the current coordinate system, level represents the level of the grid, and gridSize represents the size of the grid; Calculate the row and column numbers of the corresponding grid according to the spatial range of the spatiotemporal data: span = gridRes × gridSize Among them, row and col represent the row and column numbers of the grid respectively, lon and lat represent the longitude and latitude of the center of the grid respectively, lon min ,lat max They represent the minimum longitude and maximum latitude of the current coordinate system respectively, and span represents the size of the longitude range and latitude range represented by each grid.
3. The grid storage method for spatiotemporal big data according to claim 2 is characterized in that: The geographic coordinate system of the grid in the spatial pyramid is WGS-84 or CGCS2000, with the latitude and longitude grid as the division standard. The origin of the grid data in the spatial pyramid is set to (180°W, 90°N), the GeoHash algorithm is used to encode the grid, and the quadtree is used to organize the data.
4. A method for constructing an index of spatiotemporal big data, characterized in that: The index construction method is based on the spatiotemporal data pyramid constructed by the grid storage method for spatiotemporal big data according to any one of claims 1 to 3, and the index construction method comprises: Import data from the spatiotemporal data pyramid into a relational database, use Flume to collect self-increasing table data in the database in real time, and transmit it through the Kafka message queue; Receive data from Kafka and parse it; Determine the index type according to the parsed index metadata information, and create corresponding index data according to the index type; Store the created index data in HDFS through the StoreFile mechanism; in, The index type includes: spatiotemporal index or non-spatiotemporal index; The spatiotemporal index includes: a time period index, a space priority index or a time priority index; The non-temporal and spatial index includes: a single-attribute index or a multi-attribute index.
5. The index construction method of spatiotemporal big data according to claim 4 is characterized in that: The creating corresponding index data according to the index type includes: If the index type is the time period index, then create corresponding daily and partitioned skip table time and space index data; If the index type is the space-priority index, then create corresponding space-priority partition skip list spatiotemporal index data; If the index type is the time-first index, then create corresponding time-first partition skip table time-space index data; If the index type is the single-attribute index, then create corresponding single-attribute index data; If the index type is the multi-attribute index, corresponding multi-attribute joint index data is created.
6. A query method for spatiotemporal big data, characterized in that: The query method is based on the index data constructed by the index construction method for spatiotemporal big data according to claim 5, and the query method includes: Determine the query type according to the query request; the query type includes: spatiotemporal query and / or non-spatial query; If the query type includes the spatiotemporal query, partition screening is performed, and then a local search is performed in the screened partition to obtain a first data set; If the query type includes the non-temporal and spatial query, performing a single attribute retrieval or a multi-attribute retrieval to obtain a second data set; combining the first data set and the second data set into a candidate set; The candidate sets are integrated to generate query results.
7. The query method for spatiotemporal big data according to claim 6, characterized in that: The spatiotemporal query includes: time period query, space priority query and / or time priority query; The performing of partition screening and then performing local search in the screened partition to obtain the first data set includes: If the query type includes the time period query, partition screening is performed according to the daily partition jump table time and space index data; If the query type includes the space-first query, partition screening is performed according to the space-first partition skip table spatiotemporal index data; If the query type includes the time-first query, partition screening is performed according to the time-first partition skip table temporal and spatial index data; A local skip table index is performed in the screened partitions to obtain the first data set.
8. The query method for spatiotemporal big data according to claim 6, characterized in that: The non-spatial-temporal query includes: a single-attribute query or the multi-attribute query; The performing of single attribute retrieval or multi-attribute retrieval to obtain a second data set includes: If the query type is the single-attribute query, the second data set is retrieved according to the single-attribute index data; otherwise, the second data set is retrieved through the multi-attribute joint index data.
9. A processing device, characterized in that: The method comprises a memory and a processor, wherein the memory stores a computer program that can be loaded by the processor and execute the method according to any one of claims 1 to 8.
10. A computer-readable storage device, characterized in that: A computer program is stored which can be loaded by a processor and execute the method according to any one of claims 1 to 8.
Citation Information
Cited By
Intelligent interpretation system and method
CN120599464A
