Data query method, data storage method and corresponding device, system and medium

Abstract

The application provides a data query method, a data storage method, and corresponding devices, systems, and media. The data query method includes: converting a received query statement into a query syntax tree, the query statement being a structured query language (SQL) statement used to query target data stored in a storage resource, the target data including location information; generating a query plan based on the query syntax tree; and performing a scan query on the target data stored in the storage resource based on the query plan. According to the embodiments of the application, standard SQL queries can be supported, and efficient geofencing queries can be implemented.

Description

Technical Field

[0001] This application relates to the field of cloud computing technology, and in particular to a data query method, a data storage method, and corresponding equipment, systems, and media. Background Technology

[0002] Geofencing is an application of Location-Based Service (LBS) technology. Its principle is to use a virtual fence to enclose a virtual geographic boundary. When a mobile terminal is active within or near this virtual geographic boundary, it can receive automatic notifications and alerts. Geofencing query is a query method based on geofencing technology. For a data table storing geographic data such as latitude and longitude coordinates, given a latitude and longitude range of a polygon (polygon or circle), a geofencing query can find points whose coordinates fall within that range. Among related technologies, some data storage architectures can support geofencing queries on massive amounts of geographic data, but they do not support standard SQL (Structured Query Language) queries for geofencing, lacking ease of use. Summary of the Invention

[0003] This application provides a data query method, a data storage method, and corresponding devices, systems, and media to solve the problems existing in the prior art.

[0004] In a first aspect, embodiments of this application provide a data query method, including:

[0005] The received query statement is converted into a query syntax tree; the query statement is a structured query language (SQL) statement used to query the target data stored in the storage resources, and the target data includes location information.

[0006] Generate a query plan based on the query syntax tree;

[0007] The target data stored in the storage resources is scanned and queried based on the query plan.

[0008] Secondly, embodiments of this application provide a data storage method, including:

[0009] The received table creation statement and data insertion statement are converted into table creation syntax tree and data insertion syntax tree, respectively; both table creation statement and data insertion statement are Structured Query Language (SQL) statements.

[0010] Generate table creation execution plan and data insertion execution plan based on table creation syntax tree and data insertion syntax tree respectively;

[0011] Based on the table creation execution plan, a data table is constructed in the starting area of ​​the storage resources to store the target data;

[0012] The target data is inserted into the constructed data table based on the data insertion execution plan.

[0013] Thirdly, embodiments of this application provide a data query system that can be used to execute any of the methods provided in embodiments of this application. The system includes: computing resources and storage resources.

[0014] Communication connections between computing and storage resources;

[0015] Computational resources are used to convert the received SQL statements into corresponding syntax trees, and to generate corresponding execution plans based on the syntax trees;

[0016] Storage resources are used to execute this execution plan.

[0017] Fourthly, embodiments of this application provide an electronic device, including: a memory, a processor, and a computer program stored in the memory. When the processor executes the computer program, it can realize the functions of computing resources and / or storage resources in the data query system provided in the third aspect of this application.

[0018] Fifthly, embodiments of this application provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements any of the methods provided in embodiments of this application.

[0019] Compared with the prior art, this application has the following advantages:

[0020] According to the technical solution of this application embodiment, by setting an SQL layer in the data query system, the data query system and its corresponding data storage and data query can support standard SQL queries, effectively improving the usability of the query system; the objects stored and queried are data containing location information, which can support the processing function of such data; it can be applied to cloud-native multi-model databases (a cloud-native database that can be applied to any size and multiple models) to improve the data processing capability of cloud-native multi-model databases and realize efficient geofence query.

[0021] The above description is only an overview of the technical solution of this application. In order to better understand the technical means of this application, it can be implemented according to the contents of the specification. In order to make the above and other objects, features and advantages of this application more obvious and understandable, specific embodiments of this application are given below. Attached Figure Description

[0022] In the accompanying drawings, unless otherwise specified, the same reference numerals throughout the various drawings denote the same or similar parts or elements. These drawings are not necessarily drawn to scale. It should be understood that these drawings depict only some embodiments according to this application and should not be construed as limiting the scope of this application.

[0023] Figure 1 This is a schematic diagram of the structural framework of a data query system provided in an embodiment of this application;

[0024] Figure 2 A flowchart illustrating a data storage method provided in an embodiment of this application;

[0025] Figure 3 This is a schematic diagram of the table creation statements and data tables in the embodiments of this application;

[0026] Figure 4 This is a flowchart illustrating a table creation method in an embodiment of this application;

[0027] Figure 5 This is a flowchart illustrating a data insertion method according to an embodiment of this application;

[0028] Figure 6 A flowchart illustrating a data query method provided in an embodiment of this application;

[0029] Figure 7 This is a schematic diagram illustrating the principle of determining the encoding interval in the embodiments of this application;

[0030] Figure 8 This is a schematic diagram illustrating the principle of how the query compiler and query optimizer compile and optimize the query syntax tree in an embodiment of this application.

[0031] Figure 9 This is a schematic diagram illustrating the principle of two domain servers executing each sub-query plan in parallel, as described in this embodiment of the application.

[0032] Figure 10 This is a schematic diagram illustrating the execution process of a scanning subprocess of a region query plan in an embodiment of this application;

[0033] Figure 11 This is a flowchart illustrating an example of a query method in an embodiment of this application; and

[0034] Figure 12 This is a schematic diagram of the structural framework of an electronic device provided in an embodiment of this application. Detailed Implementation

[0035] In the following description, only certain exemplary embodiments are briefly described. As those skilled in the art will recognize, the described embodiments can be modified in various ways without departing from the concept or scope of this application. Therefore, the drawings and description are considered to be exemplary in nature and not restrictive.

[0036] The following is a brief introduction to some of the technical terms used in the embodiments of this application:

[0037] S2 geofence index: This indexing technique uses the S2 algorithm (an open-source encoding algorithm) to store the encoded values ​​of query objects (points), calculate the range of encoded values ​​for query conditions (areas), and accelerate queries through prefix matching.

[0038] Spatiotemporal data: data that has both a time dimension (i.e., time information) and a spatial dimension (i.e., spatial information or location information). Geographic data is data with a spatial dimension.

[0039] In related technologies, there exists an unstructured data storage scheme that supports geofencing queries on massive amounts of target data. The specific principle is as follows: before querying the stored point coordinate data, a 2dsphere (an index used to support spherical geometry calculations) or 2d (an index used to query data stored as points on a two-dimensional plane, which can use a 2d index) filled curve-coded index is created on the point coordinate field. During the query, the smallest bounding rectangle of the polygon or circular range is first generated, then the corresponding coded range is generated, and records belonging to the coded range are filtered out using the index, followed by a fine-tuning of the polygon range. However, because this scheme uses an unstructured database, it is incompatible with SQL, therefore it does not support standard SQL queries for geofencing and cannot achieve parallel queries, resulting in low query efficiency.

[0040] In related technologies, there is another data storage scheme that uses SimpleFeature (a geographic geometry object with attributes) as the storage structure to store geospatial points, supporting geofencing queries on massive target data. The specific principle is as follows: by default, a z2-level (a two-dimensional Z-order curve indexing method for indexing latitude and longitude of point data) curve-coded index is created for the point coordinates. During a query, the smallest bounding rectangle of the polygonal or circular range is first generated, then the corresponding coded range is generated, and records belonging to the coded range are filtered out using the index, followed by a fine-grained determination of the polygonal range. However, because this scheme uses SimpleFeature as the storage structure, it is incompatible with SQL, therefore it does not support standard SQL queries for geofencing, and parallel queries require manual configuration and cannot adaptively perform parallel queries, resulting in low query efficiency.

[0041] To facilitate understanding of the technical solutions of the embodiments of this application, the relevant technologies of the embodiments of this application are described below. The following relevant technologies are optional solutions and can be combined with the technical solutions of the embodiments of this application in any way, and all of them fall within the protection scope of the embodiments of this application.

[0042] This application provides a data query system that may include computing resources and storage resources. The computing resources and storage resources can communicate with each other. The computing resources can be used to convert received SQL statements into corresponding syntax trees and generate corresponding execution plans based on the syntax trees. The storage resources can be used to execute the above execution plans.

[0043] Based on computing resources, the following can be formed: Figure 1 The SQL layer and query processing layer shown can be configured based on storage resources as follows: Figure 1 As shown, the storage layer and computing resources can implement the processing of the SQL layer and query processing layer, and the storage resources can implement, for example... Figure 2 The storage layer shown demonstrates how computing and storage resources work together to enable data interaction and processing between the SQL layer, query processing layer, and storage layer, thereby achieving data storage and retrieval.

[0044] The computing resources can also communicate with the client. The client can receive the SQL statement (i.e., SQL string) input by the user and forward the SQL statement to the SQL layer. The SQL layer can convert the received SQL statement into the corresponding syntax tree. The query processing layer can generate the corresponding execution plan based on the syntax tree and execute the execution plan, or send the execution plan to the storage layer, which will then execute the received execution plan.

[0045] Computing resources may include at least one computing device. For example, it may include one computing device, upon which an SQL layer and a query processing layer can be formed. Alternatively, it may include two computing devices, with the SQL layer formed on the resources provided by one device and the query processing layer on the resources provided by the other. The computing device can be a server or other devices with computing capabilities. Storage resources may include at least one storage device. Multiple storage devices can form a distributed database cluster as a storage layer, such as a key-value (KV) storage layer, to achieve distributed data storage and parallel processing of data storage and query operations. The storage device can be a server or other devices with storage capabilities.

[0046] Reference Figure 1For example, the SQL layer can include a syntax parser and a syntax compiler, applicable to data storage and data query scenarios. For instance, the syntax parser can parse and identify identifiers in the SQL statement input by the user, and the syntax compiler can compile the identified identifiers into a syntax tree. The identifiers can be keywords such as select and where in the SQL statement. The query processor can include a compiler, a query optimizer, and an executor. The compiler and executor can be applied to data storage and data query scenarios, while the query optimizer can be applied to data query scenarios. For instance, the compiler can compile the syntax tree compiled by the syntax compiler into corresponding execution plans, such as table creation execution plans, data insertion execution plans, and query plans. The executor can send the execution plan compiled by the compiler to the storage layer, and the query optimizer can optimize the query plan compiled by the compiler to obtain an optimized query plan.

[0047] Reference Figure 1 For example, the storage device can be a domain server (Region Server). Each domain server can include multiple regions for storing data. Each region can store consecutive records from a data table sorted by the primary key. The primary key is implemented in the storage layer by the row key, which can be considered the primary key in the storage layer and is called the row primary key. In one example, the data of a data table can span multiple domain servers, meaning it is stored simultaneously on multiple domain servers, with each domain server storing a portion of the data table's data. In another example, the data of a data table can be stored on a single domain server, and a single domain server can store data from multiple data tables simultaneously. Data stored within a region typically belongs to a single data table.

[0048] Reference Figure 1 For example, each block can contain multiple LDFiles. Each LDFile can store a column family of data recorded within its region. Each column family can include multiple key-value pairs, which are the smallest storage units representing data from a specified row, column, and timestamp in a data table, such as spatiotemporal or geographic data. The stored data can be in the form of point coordinates. The key can consist of a primary key, column identifier, and timestamp, and the value can be the point coordinates of a cell formed by the intersection of rows and columns in the data table. The size of an LDFile is usually limited. A column family can span multiple LDFiles, meaning it can be stored in multiple LDFiles simultaneously. Data within an LDFile can belong to a column family of a specific data table.

[0049] The data query system provided in this application embodiment, based on the SQL layer settings, enables the data query system to support standard SQL queries, effectively improving the ease of use of the data query system. The aforementioned data query system can store and query spatiotemporal or geographic data. It can be applied to cloud-native multi-model databases to increase the processing capabilities of these databases for spatiotemporal and geographic data, compensating for the deficiencies in spatiotemporal and geographic data processing. This allows the cloud-native multi-model database to efficiently support queries and processing of various spatiotemporal models. The aforementioned data query system integrates a distributed KV storage structure, a wide table engine, and an SQL engine. Based on this, a geofencing query engine can be built to achieve efficient geofencing query functionality. When applied to cloud-native multi-model databases, the SQL semantics can be directly extended to support spatial operators of the OpenGIS (Open Geodata Interoperation Specification) standard, making geofencing support standard SQL queries and facilitating database user understanding and use. Under the same distributed database cluster configuration, the average query time for geofencing queries using the data query system provided in this application embodiment is better than that of the query schemes in the related technologies described above.

[0050] The data query system provided in this application embodiment can be used to execute any of the methods provided in this application embodiment, and the methods provided in this application embodiment will be described in subsequent embodiments.

[0051] Based on the same technological concept, such as Figure 2 As shown, this application embodiment provides a data storage method 200, which may include the following steps S201-S203: S201, converting the received table creation statement and data insertion statement into a table creation syntax tree and a data insertion syntax tree, respectively; S202, generating a table creation execution plan and a data insertion execution plan based on the table creation syntax tree and the data insertion syntax tree, respectively; S203, constructing a data table for storing target data in the starting area of ​​the storage resource based on the table creation execution plan; S204, inserting target data into the constructed data table based on the data insertion execution plan. The target data may include location information, and in some examples, may also include other information, such as time information.

[0052] The above data storage method 200 can be applied to the above data query system. In one example, steps S201 and S202 can be applied to computing resources, and steps S203 and S204 can be applied to storage resources.

[0053] In step S201 above, the table creation statement can be an SQL statement used to create a data table, and the data insertion statement can be an SQL statement used to insert target data into the data table. The table creation statement and the data insertion statement can be entered by the user into the client and transmitted to the computing resources by the client. The table creation statement and the data insertion statement can be transmitted to the computing resources by the client simultaneously or sequentially by the client.

[0054] The above step S201 can be implemented by the SQL layer. For example, the identifiers of the table creation statement and the data insertion statement can be parsed and identified by the syntax parser in the SQL layer. Then, the identifiers of the table creation statement can be compiled into a table creation syntax tree and the identifiers of the data insertion statement can be compiled into a data insertion syntax tree by the syntax compiler in the SQL layer. The processing of the table creation statement and the processing of the data insertion statement in the SQL layer can be performed simultaneously or sequentially.

[0055] Step S202 described above can be implemented by the query processing layer. For example, the compiler in the query processing layer can compile the table creation syntax tree into a table creation execution plan with a specified encoding algorithm and a row-unique id field as the row primary key, and compile the data insertion syntax tree into a data insertion execution plan. The processing of the table creation syntax tree and the processing of the data insertion execution plan by the query processing layer can be performed simultaneously or sequentially.

[0056] Steps S303 and S304 described above can be handled by the storage layer. The executor in the query processing layer can send a table creation execution plan and a first RPC (Remote Procedure Call) instruction to the storage layer. The storage layer can respond to the first RPC instruction by executing the table creation execution plan to build a data table for storing target data in the starting area of ​​the storage layer. The executor in the query processing layer can also send a data insertion execution plan and a second RPC instruction to the storage layer. The storage layer can respond to the second RPC instruction by executing the data insertion execution plan to realize data insertion.

[0057] The data storage method 200 provided in this application embodiment can receive SQL statements and process them to generate a table creation execution plan and a data insertion execution plan. Based on the table creation execution plan, a data table can be built in the storage resources. Then, based on the data insertion execution plan, data can be inserted into the built data table to realize data storage. This storage method can support subsequent standard SQL queries. The inserted data can be target data, which can support low-cost processing of massive target data and better meet the storage needs of scenarios such as vehicle networking and the Internet of Things.

[0058] In one implementation, the table creation statement may include primary key information. This primary key information may include a row-unique id (identifier, referring to the row identifier) ​​field and a specified primary key encoding type. The primary key encoding type can be a primary key corresponding to an encoding algorithm that maps two-dimensional coordinates to one-dimensional coordinates while maintaining spatial proximity, such as a primary key based on the S2 algorithm or a primary key based on the z-order (Morton coding) algorithm. Different primary key encoding types lead to different storage and query efficiencies; the corresponding encoding algorithm can be specified according to the required query efficiency. The table creation execution plan generated based on the table creation statement may be a plan containing the specified primary key encoding type and the aforementioned id field. The specified primary key encoding type and the aforementioned id field are written into the table creation metadata along with the table creation information. The data insertion statement may include a unique id value to be inserted and the target data to be inserted. The target data may be in the form of point coordinates. In some examples, the data insertion statement may also include other attribute information (e.g., timestamps). Based on the information contained in the table creation and data insertion statements in this implementation, a data table for storing key-value pairs can be constructed to achieve distributed and highly feasible storage.

[0059] Figure 3 An example data table is shown for constructing, such as Figure 3 The table creation statement shown can provide encoding algorithms for row-unique id fields and row primary keys, such as the S2 algorithm. After processing by the SQL layer, query processing layer, and key-value storage layer, the table creation statement can construct... Figure 3 The data table shown corresponds to a distributed storage structure with key-value pairs as the smallest unit.

[0060] like Figure 3 In the data table shown, the row primary key value is represented as "S2(x,y), record_id", where S2(x,y) represents the encoding algorithm part of the row primary key, record_id represents the unique id field of the row, and columns 1 to 4 are the column names of the four columns within a default column family. These column names can be uniformly represented as cq, and the column family name can be represented as cf. The column names within and in the column family together form a column identifier, which can be represented as cf:cq. The row containing the primary key and the column containing the column identifier cf:cq in the data table can form a cell used to store data or data-related information. For example... Figure 3 The primary key value in the middle row and the column name (x) of the first column (i.e., column 1) can form a cell used to store the value of the first column. Figure 3 The primary key value in the middle row and the column name (y) of the second column (i.e., column2) can form a cell used to store the values ​​of the second column. Figure 3The primary key value in the middle row and the column name of the third column (i.e., column 3) (i.e., collect_time) can form a cell used to store the value of the third column. Figure 3 The primary key value of the middle row and the column name of the fourth column (i.e., column4) (i.e., car_id) can form a cell to store the value of the fourth column. car_id is an attribute field that can store specific attributes of the data. The complete information of the key-value pair stored in each cell includes the row primary key value, column name, timestamp, and column value. Figure 3 In the data table shown, VARBINARY (variable-length binary data) is the data type of S2(x,y), VARCHAR (variable-length string) is the data type of record_id, x and y are the two parameters of the filtering condition, and DOUBLE (double-precision floating-point) is the data type of the column values ​​of column x and column y.

[0061] In one implementation, the data insertion execution plan may include the id value to be inserted and the target data to be inserted. Correspondingly, in step S204 above, inserting the target data into the constructed data table based on the data insertion execution plan may include: encoding the target data based on the primary key encoding type of the data table, concatenating the encoding result and the id value into the primary key, and inserting it into the data table. For example, if the metadata of the constructed data table shows that the primary key used by the constructed data table is an S2 primary key (i.e., a primary key formed based on the S2 algorithm), then the target data can be encoded based on the S2 algorithm, and the S2 encoding result and the id value to be inserted can be concatenated into the primary key and inserted into the data table.

[0062] In one embodiment, the data storage method 200 may further include: during the process of inserting target data, determining whether the amount of data already inserted in the starting region has reached a preset data volume limit; if the amount of data already inserted in the starting region has reached the preset data volume limit, increasing the number of regions in the current server and continuing to insert target data in the added regions; if the amount of data already inserted in the starting region has not reached the preset data volume limit, continuing to insert target data in the starting region.

[0063] Based on a preset data volume limit, the data storage method 200 provided in this application embodiment can identify the data volume saturation in the starting area. When it is identified that the starting data volume is saturated, the number of areas of the current server can be adaptively increased. The data table in the starting area is also expanded to the new area as the number of areas increases. The target data continues to be inserted into the data table in the new area, so that the target data can be evenly distributed in different areas, avoiding excessive data volume and high power consumption in one area.

[0064] The determination of whether the amount of data inserted into the starting area has reached the preset upper limit of data volume can be performed in real time during the data insertion process into the starting area, or it can be performed periodically during the data insertion process into the starting area.

[0065] The number of regions to be added can be one or more. When adding multiple regions, they can be added one by one. Each time a region is added, it can be determined whether the amount of data already inserted into the region has reached the preset data volume limit during the process of inserting target data into the region. If the amount of data already inserted into the region has reached the preset data volume limit, the next region is added to the current server, and the target data is continued to be inserted into the next region. If the amount of data already inserted into the region has not reached the preset data volume limit, the target data can continue to be inserted into the currently added region.

[0066] Figure 4 An exemplary method for creating a table is shown. Figure 5 An exemplary data insertion method is shown. (Refer to...) Figure 4 The table creation method may include: the SQL layer receiving the table creation statement; the SQL layer parsing and compiling the table creation statement to obtain a table creation syntax tree; the query processing layer compiling the table creation syntax tree and determining whether the primary key used by the data table is an S2 primary key. If so, an S2 primary key is added, and a table creation execution plan is generated; otherwise, the table creation execution plan is directly generated, and the KV storage layer writes the data table's metadata based on the table creation execution plan. (See reference...) Figure 5 The data insertion statement can include: the SQL layer receiving the data insertion statement; the SQL layer parsing and compiling the data insertion statement to obtain a data insertion syntax tree; the query processing layer compiling the data insertion syntax tree and determining whether the primary key used by the data table is an S2 primary key. If so, it generates an S2 encoded value based on the S2 algorithm, thereby forming a data insertion execution plan. If not, it directly generates a data insertion plan; the KV storage layer inserts data into the data table based on the data insertion plan and automatically splits out hotspot areas.

[0067] Based on the same technological concept, such as Figure 6 As shown, this application embodiment also provides a data query method 600, which may include: S601, converting the received query statement into a query syntax tree; S602, generating a query plan based on the query syntax tree; S603, scanning and querying the target data stored in the storage resources based on the query plan.

[0068] In step S601 above, the query statement can be an SQL statement used to query target data stored in the storage resource. This statement can be entered by the user into the client and transmitted by the client to the computing resource. The query statement can include the required column information and filtering conditions.

[0069] The above step S601 can be implemented by the SQL layer. For example, the identifiers of the query statement can be parsed and recognized by the syntax parser in the SQL layer, and then the identifiers of the query statement can be compiled into a query syntax tree by the syntax compiler in the SQL layer.

[0070] Steps S602 and S603 described above can be implemented by the query processing layer and the storage layer. For example, the query syntax tree can be compiled into a query plan by the compiler in the query processing layer, and the query plan and third RPC instructions can be sent to the storage resource by the executor in the query processing layer. In response to the third RPC instructions, the storage resource can execute the query plan to scan and query the target data stored in the storage resource. In one example, the executor of the query processing layer can create a scanning master process based on the query plan, and the storage layer executes the scanning master process to scan and query the target data stored in the data table.

[0071] The data query method 600 provided in this application embodiment can receive SQL statements and process them to generate a query execution plan, and then implement standard SQL queries based on the execution of the query execution plan; the data queried can be target data, thereby efficiently supporting the query and processing of target data and realizing the function of efficient geofence query.

[0072] In one implementation, step S602 above, generating a query plan based on the query syntax tree, may include: generating an initial query plan based on the query syntax tree, performing parallel optimization on the initial query plan to obtain a parallel query plan including multiple sub-query plans, which serves as the query plan. The query scope of the initial query plan can be the entire range of the data tables in the storage resources, and the query scope of each sub-query plan can be a partial range of the data table; the query scopes of different sub-query plans may not overlap.

[0073] Parallel optimization of the initial query plan can be achieved through the query optimizer in the query processing layer. The query optimizer can split an initial query plan with a large query range into multiple sub-query plans with smaller query ranges through parallel optimization, forming a parallel query plan that includes multiple sub-query plans. Each sub-query plan in the parallel query plan can be executed in parallel, which can effectively shorten the query time and improve the overall query efficiency compared to serial execution.

[0074] In one implementation, parallel optimization of the initial query plan may include: identifying multiple coded intervals in storage resources to characterize geofence ranges; splitting the initial query plan into multiple interval query plans based on the multiple coded intervals; each interval query plan having a query range equal to that of a coded interval; and each interval query plan being used to scan and query target data stored within a coded interval. The number of interval query plans may be equal to the number of coded intervals. These sub-query plans can be interval query plans, and multiple sub-query plans constitute multiple interval query plans. These multiple interval query plans can be executed in parallel, allowing for parallel scanning and querying of target data stored in each interval. Since the range of a coded interval is relatively small, meaning the range that the interval query plan corresponding to a coded interval needs to query is relatively small, the parallel execution of multiple interval query plans can significantly shorten the query time and improve query efficiency.

[0075] In another implementation, parallel optimization of the initial query plan may include: identifying multiple coded intervals in the storage resources used to characterize the geofence range; merging the multiple coded intervals by region to obtain multiple query intervals corresponding to the multiple regions, where the query range of each query interval can be the spatial range corresponding to a region; splitting the initial query plan into multiple regional query plans based on the multiple query intervals, where the query range of each regional query plan is the range of a query interval, and each regional query plan can be used to scan and query the target data stored in a query interval. The number of regional query plans can be equal to the number of query intervals generated by merging, and thus also equal to the number of regions in the storage layer. The aforementioned sub-query plans can be regional query plans, and multiple sub-query plans constitute multiple regional query plans. Multiple regional query plans can be executed in parallel, thereby enabling parallel scanning and querying of the target data stored in each region, shortening the query time, and improving query efficiency.

[0076] When merging multiple coded intervals by region, the region to which each coded interval belongs can be determined first, and coded intervals belonging to the same region can be merged into a single query interval. Since each interval query plan requires one process resource from the executor during execution, parallel execution of the interval query plans corresponding to each coded interval can consume excessive process resources when the number of determined coded intervals is large. By merging coded intervals and splitting the result into subquery plans, the number of subquery plans requiring parallel execution can be reduced, thus avoiding excessive consumption of process resources.

[0077] For cross-regional coding intervals, the sub-intervals located in different regions can be merged into the corresponding query intervals of the respective regions. For example, if part of the coding interval A, sub-interval A1, belongs to region B1, and another part, sub-interval A2, belongs to region B2, then coding interval A is a coding interval that spans regions B1 and B2. Sub-interval A1 can be merged into the query interval corresponding to region B1, and sub-interval A2 can be merged into the query interval corresponding to region B2. That is, coding interval A is split into two parts and merged into two query intervals respectively. This can avoid the impact on the operation of query plans with larger query ranges due to too many query plans with smaller query ranges competing for resources in parallel, which can further improve query efficiency and better adapt to the scale of data.

[0078] Whether a given coding interval is a cross-region coding interval, and which regions it crosses, can be determined based on the interval value (i.e., the range of coding values ​​used to match row primary keys for queries) and the primary key range of the data tables in each region. The query optimizer can use Zookeeper (distributed application coordination service software) to find all domain servers in the database cluster, obtain the primary key range of the data tables in each region from each domain server, and determine which regions' primary key ranges the starting and ending values ​​of the coding interval fall within. This determines which regions the coding interval crosses. Coding intervals and / or sub-regions of the same region can be merged into a larger query interval. The starting value of this query interval (i.e., the starting coding value of the interval) can be the minimum of the starting values ​​of all coding intervals belonging to this region, and the ending value of this query interval (i.e., the ending coding value of the interval) can be the maximum of the ending values ​​of all coding intervals belonging to this region. This allows for a balanced allocation of the spatial range of a single scan query based on the interval query plan.

[0079] In another implementation, parallel optimization of the initial query plan may include: identifying multiple coded intervals in the storage resources used to characterize the geofence range; splitting the initial query plan into multiple interval query plans based on the multiple coded intervals, where each interval query plan's query range is the range of one coded interval, and each interval query plan can be used to scan and query the data within one coded interval; merging the multiple interval query plans by region to obtain multiple merged region query plans, where each region query plan's query range is the spatial range of one region, and each region query plan can be used to scan and query the target data stored in one region. The aforementioned sub-query plans can be region query plans, and multiple sub-query plans constitute multiple region query plans. These multiple region query plans can be executed in parallel, thereby enabling parallel scanning and querying of the target data stored in each region, shortening the query time, and improving query efficiency.

[0080] When merging multiple interval query plans by region, you can first determine the region to which the coding interval corresponding to each interval query plan belongs, and then merge the interval query plans corresponding to the coding intervals belonging to the same region into one region query plan.

[0081] The initial query plan can include information on the query range and filtering conditions. In the initial query plan, the query range can be the entire range of the data table. A full table scan can be performed based on the initial query plan. When the initial query plan is split into multiple interval query plans or multiple region query plans, the query range in the initial query plan can be modified from the entire range of the data table to multiple different local ranges of the data table, thereby forming multiple new query plans with different query ranges, namely interval query plans or region query plans. An interval query plan can be used to perform an interval scan targeting a coded region, and a region query plan can be used to perform a region scan targeting a region.

[0082] In one implementation, the storage resource may include multiple storage modules, and the target data may be geographic data. Correspondingly, in step S603 above, scanning and querying the target data stored in the storage resource based on a query plan may include: executing each sub-query plan corresponding to that storage module in parallel within each storage module; and based on the execution of each sub-query plan, performing parallel scanning and querying of the geographic data stored in each query range within each storage module. When the storage resource includes a storage device with multiple storage spaces, each storage space in that storage device can serve as a storage module; when the storage resource includes multiple storage devices, each storage device can serve as a storage module.

[0083] The parallel execution of each subquery plan can be jointly implemented by the query processing layer and the storage layer. For example, the executor in the query processing layer can send the corresponding subquery plan to each storage module in the storage resources, so that each storage module can execute the received subquery plan in parallel to improve query efficiency.

[0084] In one implementation, a subquery plan may include: a query range, a list of scanned column names, and a first filter condition. A query range may be a range of a coding interval or a region, the list of scanned column names may provide the names of the required columns, and the first filter condition may be used to filter the required rows. Correspondingly, based on the execution of each subquery plan, parallel scanning queries are performed on the geographic data stored in each query range in each storage module. This may include: when executing each subquery plan, scanning queries are performed on the target data stored in a query range based on the list of scanned column names and the first filter condition in the subquery plan.

[0085] The first filtering condition can be the ST_Contains function. The first parameter of the ST_Contains function is a constant parameter of type Polygon or Circle. The second and third parameters are the longitude and latitude coordinates of the point, respectively. Both are columns of numerical type in the data table. The columns required in the list of column names to be scanned can be at least one of the following: projection columns, columns required by the ST_Contains function for filtering, etc.

[0086] The encoding range can be determined based on the ST_Contains function. In one example, the first parameter of the ST_Contains function, namely the constant range of the face (polygon / circle) type, can be extracted, and the S2 algorithm can be used to obtain the encoding range as follows: Figure 7 The grid shown represents multiple approximate coverage areas, with each grid being a coding interval. By sorting all coding intervals according to their starting values, multiple non-overlapping coding intervals with increasing starting values ​​can be obtained, which approximately represent the geographic fence coverage area.

[0087] When the subquery plan is a range query plan, each range query plan can be executed in parallel. When the subquery plan is a region query plan, each region query plan can be executed in parallel.

[0088] When performing a query, the executor in the query processing layer can send a subquery plan, including the query range, a list of scanned column names, and a first filter condition, to the storage layer. The list of scanned column names includes projection columns, which is equivalent to pushing both the projection operation based on the projection columns and the filtering operation based on the first filter condition down from the query processing layer to the storage layer. When the storage layer scans the target data, it can perform projection operations based on the projection columns and filtering operations based on the filter conditions, and only return fields that meet the projection columns and can pass the first filter condition to the query processing layer, which can reduce unnecessary data transmission between the query processing layer and the storage layer.

[0089] In one implementation, based on the query range, the list of scanned column names, and the first filtering condition, the sub-query plan may further include a second filtering condition. The second filtering condition can be an interval skip condition, specifically including the range of multiple coded intervals in the storage resource. The second filtering condition can be used to skip the gaps between any two coded intervals within the same query range of the storage resource. Correspondingly, based on the execution of each sub-query plan, parallel scanning queries are performed on the geographic data stored in each query range in each storage module. This can include filtering out the gaps between any two coded intervals within the same query range based on the second filtering condition. That is, only the target data within the coded interval range is scanned, and the gaps between two coded intervals are not scanned, avoiding unnecessary fine-grained calculations and further improving query efficiency.

[0090] The second filtering condition can be set before the first filtering condition. Before performing filtering operations based on the first filtering condition, the gaps between any two coding intervals within the same region are first filtered out based on the second filtering condition, and these gaps are not scanned or queried. If there is only one coding interval in the region before merging coding intervals, it can be assumed that the second filtering condition is passed, and no filtering is performed based on the second filtering condition.

[0091] Figure 8 This relates to the principle by which query compilers and query optimizers compile and optimize query syntax trees. Figure 8 The left side shows a query syntax tree that provides the following information: the geofencing query object is the data table `table`, the field to be returned by the `select` statement is `car_id`, the required data, the filtering condition is the `ST_Contains` function, the first parameter of the `ST_Contains` function is a constant parameter of the face type (Literal), the face type can be polygon (Polygon) or circle (Circle), the second and third parameters are the longitude coordinates (x) and latitude coordinates (y) of the point, respectively, and the Identifier represents the identifier number, where x and y are both column identifiers, i.e., column names. Figure 9 The query syntax shown above, after being processed by the query compiler, can generate an initial query plan. The query scope in the initial query plan is a full table scan, the filter condition is the ST_Contains function, and the initial query plan can also include a projection condition.

[0092] Reference Figure 8 The query optimizer can perform the following optimizations on the initial query plan: push the projection and filtering condition functions down to the storage layer, i.e., to a position closer to the data source; perform S2 geofencing index optimization, which includes extracting the face parameters of the ST_Contains function, generating S2 encoded intervals, and rewriting the query range in the initial query plan. After index optimization, multiple interval query plans can be obtained. Figure 8 Only two interval query plans are shown as examples. These two interval query plans have different query ranges, which are the ranges of the corresponding S2 encoded intervals. Figure 8 S2Range Scan1 and S2 Range Scan2 are represented respectively; the subquery plans optimized by the index are grouped and merged by region to obtain multiple merged region query plans. Figure 8 Only two regional query plans are shown as examples. These two regional query plans have different query ranges, which are the ranges of the corresponding regions. Figure 8These are represented as region-combined S2 RangeScan1 and region-combined S2 Range Scan2, respectively. In addition, another filtering condition, namely the Interval Skip Filter, is added to the region query plan to filter out gaps between any two coded intervals in the same region.

[0093] Figure 9 This diagram illustrates the principle of two domain servers (i.e., two storage modules) executing subquery plans in parallel. (Refer to...) Figure 9 When the subquery plans obtained by the query optimizer are input into the executor, the executor's main query process can start a scan subprocess for each subquery plan. Figure 9 Only four scanning subprocesses, subprocesses 1 through 4, are shown as an example. These four subprocesses can execute in parallel to perform parallel queries on four regions. Subprocess 1 scans Region 1 in the first domain server (Region Server 1), subprocess 2 scans Region 2 in the first domain server (Region Server 1), subprocess 3 scans Region 1 in the second domain server (Region Server 2), and subprocess 4 scans Region 2 in the second domain server (Region Server 2). Within each region, the subprocesses can scan each key-value pair in the LDFile. The key-value pair information includes the row primary key value (e.g., ...). Figure 9 The S2 rowkey shown), column identifier (such as...) Figure 9 The cf:cq shown), timestamp (such as Figure 9 The ts shown) and column values ​​(such as Figure 9 (The value shown).

[0094] Reference Figure 9 In each region, to ensure scanning efficiency, the scan pointer can move sequentially forward from the starting row (i.e., the starting rowkey) to the ending row (i.e., the ending rowkey). Whenever the scan pointer moves to a row of the data table, it can determine whether the column names of each column in that row are in the scan column name list and whether the data in that row can pass the first filter condition and the second filter condition. Data in that row whose column names belong to the scan column name list and can pass the first filter condition and the second filter condition are returned to the query processing layer.

[0095] Figure 10 The execution process of a scanning subprocess of a region query plan is illustrated exemplarily, with reference to... Figure 10The execution process of a scanning process includes the following steps S1001-S1015:

[0096] S1001, move the scan pointer to the KV (key-value pair) of the first row whose primary key value is not less than the starting encoding value of the current region, initialize the current row identifier passFlag to UNDECIDED, and initialize the result temporary storage set and the return result set to empty sets.

[0097] UNDECIDED indicates that it is uncertain whether the data in the current row needs to be returned to the query processing layer; it is in a pending state. The result staging set is used to temporarily store data that may need to be returned to the query processing layer (i.e., it is not yet finalized whether it should be returned). The return result set is used to store data that is finally determined to be returned to the query processing layer. Data staging in the result staging set can be moved into the return result set when it is finally determined that it needs to be returned to the query processing layer.

[0098] S1002, move the scan pointer to the next KV and use it as the current KV. Determine whether the current KV is the first KV. If the current KV is not the first KV, proceed to step S1003; if the current KV is the first KV, proceed to step S1009.

[0099] Depending on the actual needs, the determination of whether the current key value is the first key value can be omitted, and the process can proceed directly to step S1203.

[0100] S1003, determine whether a new row has been scanned. If yes, proceed to step S1004; otherwise, proceed to step S1009.

[0101] One specific way to determine whether a new row has been scanned is to check whether the primary key of the current key-value pair is the same as the primary key of the previous key-value pair. If they are the same, it means that no new row has been scanned; if they are different, it means that a new row has been scanned.

[0102] S1004. Check if the row identifier passFlag is INCLUDE. If yes, proceed to step S1005. If no, proceed to step S1006.

[0103] INCLUDE indicates that the data in the current row needs to be returned to the query processing layer.

[0104] S1005: Store all values ​​in the result temporary set into the return result set;

[0105] S1006, Clear the result temporary storage set;

[0106] S1007, reset passFlag to UNDECIDED.

[0107] S1008: Determine whether the current region has reached the termination line. If yes, terminate the scan; otherwise, execute S1009.

[0108] The specific way to determine whether the current region has reached the end row of the current region can be: to determine whether the current region has reached the end marker of the current region, or whether the row primary key value of the current KV is greater than the maximum value of the scan range; if either determination is yes, it means that the current region has reached the end row of the current region, otherwise the current region has not reached the end row of the current region.

[0109] S1009, determine whether the column corresponding to the current KV is a projection column. If yes, execute S1010; otherwise, execute S1011.

[0110] One specific way to determine whether the column corresponding to the current key-value pair is a projection column is to check if the column identifier cf:cq of the current key-value pair is the column identifier of the projection column in the list of scanned column names.

[0111] S1010, store the current KV into the result temporary storage set, and then execute step S1012.

[0112] S1011, determine whether the column corresponding to the current KV is the column required by the ST_Contains function for filtering. If yes, proceed to step S1012. If no, it means that the current KV is not needed, and proceed to step S1002.

[0113] One specific way to determine whether the column corresponding to the current key-value pair is the column required for filtering by the ST_Contains function is to check whether the column identifier cf:cq of the current key-value pair is the column identifier of the column required for filtering by the ST_Contains function in the scan column name list.

[0114] exist Figure 3 In the data table shown, the columns containing x and y are the columns required for filtering by the ST_Contains function.

[0115] S1012, check if passFlag is INCLUDE or EXCLUDE. If yes, proceed to step S1002; otherwise, proceed to step S1013.

[0116] EXCLUDE indicates that the data in the current row does not need to be returned to the query processing layer.

[0117] S1013, determine whether the current KV row primary key (S2 rowkey) can pass the gap filtering condition. If yes, execute S1016; otherwise, execute S1014.

[0118] For example, it can first be determined whether the row primary key of the current KV is greater than or equal to the minimum primary key value (minRowkey) in the second filtering condition. If not, it is considered that the current KV cannot pass the second filtering condition, and step S1014 is executed. If yes, it is determined by binary search whether the prefix of the row primary key of the current KV is located at the boundary or inside the encoding interval in the second filtering condition. If the prefix of the row primary key of the current KV is located at the boundary or inside the encoding interval in the second filtering condition, it is considered that the current KV can pass the second filtering condition, and step S1016 is executed. If the prefix of the row primary key of the current KV is not located at the boundary or inside the encoding interval in the second filtering condition, it is considered that the current KV cannot pass the second filtering condition, and the minimum primary key value can be updated to the starting encoding value of the nearest encoding interval after the current row primary key, and then step S1014 is executed.

[0119] The initial value of the minimum primary key can be set to 0.

[0120] S1014, set passFlag to EXCLUDE, and then execute step S1002.

[0121] S1015, determine whether the column corresponding to the current KV is the column required for filtering by the ST_Contains function. If yes, proceed to step S1016. If no, the current KV is just a projection column, and proceed to step S1002.

[0122] S1016 adds the current KV to the variable parameter set of the ST_Contains function.

[0123] S1017. Determine whether the variable parameter set of the ST_Contains function has collected all the variable parameters. If yes, proceed to step S1018; otherwise, proceed to step S1002.

[0124] In such Figure 3 In the data table shown, x and y are the variable parameters of the ST_Contains function.

[0125] S1018, Perform geofence fine-grained judgment calculation to determine whether the current coordinates are within the geofence range; if yes, the data record of the current row has passed all filtering conditions, proceed to S1019; if no, the data record of the current row has not passed all filtering conditions, proceed to step S1020.

[0126] S1019, set passFlag to INCLUDE, clear the variable parameter set of the ST_Contains function, and then execute step S1202.

[0127] S1020, set passFlag to EXCLUDE, clear the variable parameter set of the ST_Contains function, and then execute step S1002.

[0128] Figure 10 For the sake of brevity, not all of the above steps are shown. Figure 10 During the scanning process described above, whenever the amount of data in the result set returned by the scanning process within a certain region reaches the preset data limit for that region, the storage layer returns the data from the result set to the query processing layer. The query processing layer can then append the returned data to a shared result set queue maintained by the main query process. This shared result set queue ensures process safety. After appending, the storage layer can clear the result set. The query processing layer can then retrieve the latest data from the shared result set sub-queue and return it to the client.

[0129] Figure 11 An example of the data query method described above is shown below, see reference. Figure 11 The data query method can include: the SQL layer receiving the query statement; the SQL layer parsing and compiling the query statement to obtain a query syntax tree; the query processing layer compiling the query syntax tree to obtain an initial query plan; the query processing layer performing query optimization processing on the initial query plan, such as projection conditions and filtering conditions, and determining whether the primary key used by the data table is an S2 primary key. If so, the S2 index is used in the query optimization process to obtain the S2 encoding range, and the S2 encoding range is further scanned and merged to generate an optimized parallel query plan. If not, the optimized parallel query plan is generated directly; the KV storage layer executes the parallel query plan to perform parallel scanning.

[0130] Based on the same technical concept, embodiments of this application also provide an electronic device, such as... Figure 12 As shown, the electronic device includes a memory 1201 and a processor 1202. The memory 1201 stores a computer program that can run on the processor 1202. When the processor 1202 executes the computer program, it implements the functions of computing resources and / or storage resources in the data query system provided in this application embodiment. The number of memories 1201 and processors 1202 can be one or more.

[0131] In one example, an electronic device can execute method steps corresponding to either computing resources or storage resources, thus enabling a single electronic device to perform the functionality of either computing or storage resources—that is, to implement the functionality of a single type of resource. In another example, an electronic device can execute method steps corresponding to both computing and storage resources, thus enabling a single electronic device to perform the functionality of both types of resources—that is, to implement the functionality of two types of resources.

[0132] The aforementioned data query system may include at least one electronic device. When the data query system includes one electronic device, that electronic device can perform the functions of two resources. When the data query system includes multiple electronic devices, each electronic device can perform the function of one resource, and multiple electronic devices can jointly perform the functions of two resources. Alternatively, each electronic device can perform the functions of two resources. For example, each electronic device can perform partial functions of two resources, and multiple electronic devices can jointly perform complete functions of two resources. Or, each electronic device can perform complete functions of two resources, with one electronic device performing a specific function while other devices serve as backup devices. When the data query system includes multiple electronic devices, the functions that each electronic device can perform can be configured according to the device's attributes and actual needs. This application embodiment does not limit this, nor is it limited to the few cases exemplified in this application.

[0133] The electronic device also includes:

[0134] The communication interface 1203 is used to communicate with external devices and exchange and transmit data.

[0135] If the memory 1201, processor 1202, and communication interface 1203 are implemented independently, they can be interconnected via a bus to communicate with each other. This bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. This bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 12 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0136] Optionally, in a specific implementation, if the memory 1201, processor 1202 and communication interface 1203 are integrated on a single chip, the memory 1201, processor 1202 and communication interface 1202 can communicate with each other through an internal interface.

[0137] This application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements any of the methods provided in this application.

[0138] This application also provides a chip, which includes a processor for calling and executing instructions stored in a memory, causing a communication device equipped with the chip to perform any of the methods provided in this application.

[0139] This application also provides a chip, including: an input interface, an output interface, a processor, and a memory. The input interface, output interface, processor, and memory are connected through an internal connection path. The processor is used to execute code in the memory. When the code is executed, the processor is used to execute the method provided in the application embodiment.

[0140] It should be understood that the processor in this application can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. General-purpose processors can be microprocessors or any conventional processor. It is worth noting that the processor can be a processor supporting Advanced Reduced Instruction Set Machines (ARM) architecture.

[0141] Further, optionally, the memory in this application may include read-only memory and random access memory. The memory may be volatile memory or non-volatile memory, or may include both. Non-volatile memory may include read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory may include random access memory (RAM), which serves as an external cache. Many forms of RAM are available by way of example, but not limitation. Examples include Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDR SDRAM), Enhanced Synchronous DRAM (ESDRAM), Synchronous Link DRAM (SLDRAM), and Direct Rambus RAM (DR RAM).

[0142] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product. A computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions according to this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transferred from one computer-readable storage medium to another.

[0143] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of those different embodiments or examples.

[0144] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this application, "a plurality of" means two or more, unless otherwise explicitly specified.

[0145] Any process or method described in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more executable instructions for implementing a particular logical function or process. Furthermore, the scope of the preferred embodiments of this application includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functionality involved.

[0146] The logic and / or steps described in the flowchart or otherwise herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus or device (such as a computer-based system, a processor-included system or other system that can fetch and execute instructions from, an instruction execution system, apparatus or device).

[0147] It should be understood that various parts of this application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution system. All or part of the steps of the methods in the above embodiments can be implemented by a program instructing related hardware, the program being stored in a computer-readable storage medium, which, when executed, includes one or a combination of the steps of the method embodiments.

[0148] Furthermore, the functional units in the various embodiments of this application can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into a module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. This storage medium can be a read-only memory, a disk, or an optical disk, etc.

[0149] The above description is merely an exemplary embodiment of this application, but the scope of protection of this application is not limited thereto. Any person skilled in the art can easily conceive of various variations or substitutions within the technical scope described in this application, and these should all be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A data query method, characterized in that, include: Transform the received query statement into a query syntax tree; The query statement is a Structured Query Language (SQL) statement used to query target data stored in the storage resources, and the target data includes location information; Generate a query plan based on the query syntax tree; Based on the query plan, the target data stored in the storage resources is scanned and queried; The process of generating a query plan based on the query syntax tree includes: An initial query plan is generated based on the query syntax tree; the query scope of the initial query plan is the entire range of data tables in the storage resources; The initial query plan is optimized in parallel to obtain a parallel query plan that includes multiple sub-query plans, which is used as the query plan; the query range of the sub-query plan is a portion of the data table. The parallel optimization of the initial query plan includes: Identify multiple coded intervals in the storage resources used to characterize the geofence extent; The multiple encoding intervals are merged according to the storage area to obtain multiple query intervals corresponding to the multiple areas respectively; The initial query plan is divided into multiple regional query plans based on the multiple query intervals; the query scope of each regional query plan is a query interval, and the sub-query plan is the regional query plan.

2. The data query method according to claim 1, characterized in that, The parallel optimization of the initial query plan includes: Identify multiple coded intervals in the storage resources used to characterize the geofence extent; The initial query plan is split into multiple interval query plans based on the multiple coding intervals; the query range of each interval query plan is one coding interval; The multiple interval query plans are merged by region to obtain multiple merged regional query plans; the query range of each regional query plan is a query interval, and the sub-query plan is the regional query plan.

3. The data query method according to claim 1, characterized in that, The storage resources include multiple storage modules, and the target data is geographic data; The step of scanning and querying the target data stored in the storage resources based on the query plan includes: Each subquery plan corresponding to that storage module is executed in parallel within each storage module. Based on the execution of each subquery plan, the geographic data stored in each query range within each storage module is scanned and queried in parallel.

4. The data query method according to claim 3, characterized in that, The subquery plan includes: query range, scan column name list and first filter condition, the first filter condition is used to filter the required rows, and the scan column name list is used to provide the column names of the required columns; The execution of each sub-query plan involves parallel scanning and querying of the geographic data stored in each query range within each storage module, including: When executing each subquery plan, a scan query is performed on the target data stored in a query range based on the list of scan column names in the subquery plan and the first filter condition.

5. The data query method according to claim 4, characterized in that, The subquery plan further includes a second filtering condition, which includes the range of multiple encoding intervals in the storage resource; The execution of each sub-query plan involves parallel scanning and querying of the geographic data stored in each query range within each storage module, including: Based on the second filtering condition, gaps between any two encoding intervals within the same query range are filtered out.

6. A data storage method, characterized in that, include: The received table creation statement and data insertion statement are converted into table creation syntax tree and data insertion syntax tree, respectively; Both the table creation statement and the data insertion statement are SQL statements. The table creation statement includes primary key information, which includes a row-unique identifier (id) field and a specified primary key encoding type. Based on the table creation syntax tree and the data insertion syntax tree, generate the table creation execution plan and the data insertion execution plan respectively; Based on the table creation execution plan, a data table for storing the target data is constructed in the initial storage area of ​​the storage resources; The execution plan inserts the target data into the constructed data table based on the data.

7. The data storage method according to claim 6, characterized in that, Also includes: During the process of inserting the target data, it is determined whether the amount of data already inserted into the starting storage area has reached the preset data volume limit; If the amount of data already inserted into the initial storage area reaches the preset data limit, the number of areas in the current server is increased, and the target data is continued to be inserted into the increased areas.

8. The data storage method according to claim 7, characterized in that, The data insertion execution plan includes the id value to be inserted and the target data to be inserted; The step of inserting the target data into the constructed data table based on the data insertion execution plan includes: The target data to be inserted is encoded based on the primary key encoding type of the data table; The encoded result and the id value are concatenated into a primary key and inserted into the data table.

9. A data query system, characterized in that, The system for performing the method as described in any one of claims 1-8 includes: computing resources and storage resources; The computing resources and the storage resources are connected in a communication manner; The computing resources are used to convert the received SQL statements into corresponding syntax trees and generate corresponding execution plans based on the syntax trees. The storage resources are used to execute the execution plan.

10. An electronic device, characterized in that, include: A memory, a processor, and a computer program stored in the memory, wherein the processor, when executing the computer program, performs the functions of the computing resources and / or storage resources in the data query system of claim 8.

11. A computer-readable storage medium storing a computer program that, when executed by a processor, implements the method of any one of claims 1-7.

Images