Data processing method, data query method, computing device, storage medium, and program product

Abstract

Provided in the embodiments of the present disclosure are a data processing method, a data query method, a computing device, a storage medium, and a program product. A target region is divided into a geographic grid, and for at least one geographic grid cell covered by a target spatial object in a database, a grid index is queried to determine a grid cell record corresponding to the at least one geographic grid cell, wherein the grid cell record comprises a key-value pair that takes a grid cell code as a key and takes a primary key set of at least one spatial object as a value; and on the basis of a primary key of the target spatial object, the grid cell record is updated, wherein the grid index is used for determining, for an object to be queried in a query command, a target primary key set consisting of a value in a grid cell record that corresponds to at least one geographic grid cell covered by the object to be queried, and executing a query operation taking into account the target primary key set. The technical solution provided in the embodiments of the present disclosure improves the query efficiency.

Description

[0001] Data Processing Method, Data Query Method, Computing Device, Storage Medium, and Program Product This disclosure claims priority to Chinese Patent Application No. 202410371568.8, filed with the China Patent Office on March 28, 2024, entitled "Data Processing Method, Data Query Method, Computing Device, Storage Medium, and Program Product," the entire contents of which are incorporated herein by reference. Technical Field: Embodiments of the present disclosure relate to the field of database technology, and more particularly to a data processing method, a data query method, a computing device, a storage medium, and a program product. Background: Spatial objects in a database refer to data structures containing geographic location information, such as objects that may include points, lines, polygons, or other complex geographic shapes. In practical applications, such as travel and location-based services, queries on spatial objects are often involved, such as querying the area where a location is located or the roads within a certain area. Therefore, improving the efficiency and cost-effectiveness of spatial object queries remains a technical problem that needs to be addressed. SUMMARY OF THE INVENTION Embodiments of the present disclosure provide a data processing method, a data query method, a computing device, a storage medium, and a program product to address the technical problem of low query efficiency in the prior art. In a first aspect, embodiments of the present disclosure provide a data processing method, comprising: determining at least one geographic grid covered by a target spatial object in a database; the geographic grid being obtained by dividing a target area; querying a grid index to determine a grid record corresponding to the at least one geographic grid; the grid record comprising a key-value pair with a grid code as a key and a primary key set of at least one spatial object as a value; updating the grid record based on the primary key of the target spatial object; the grid index being used to determine, for a query object in a query command, a target primary key set consisting of values ​​in grid records corresponding to the at least one geographic grid covered by the query object; and performing a query operation in conjunction with the target primary key set. In a second aspect, an embodiment of the present disclosure provides a data query method, comprising: obtaining a query command, determining an object to be queried; determining at least one geographic grid covered by the object to be queried; the geographic grid being obtained by dividing a target area; querying a grid record corresponding to the at least one geographic grid from a grid index to obtain a target primary key set consisting of values ​​in the grid record corresponding to the at least one geographic grid; the grid record including a key-value pair having a grid code of any geographic grid as a key and a primary key set of at least one spatial object as a value; updating and generating corresponding grid records using the grid index based on the at least one geographic grid covered by the target spatial object and its primary key in a database; and executing a query operation corresponding to the query command in combination with the target primary key set.In a third aspect, embodiments of the present disclosure provide a computing device comprising a processing component and a storage component; the storage component stores one or more computer instructions; the one or more computer instructions are configured to be called and executed by the processing component to implement the data processing method described in the first aspect or the data query method described in the second aspect. In a fourth aspect, embodiments of the present disclosure provide a computer-readable storage medium storing a computer program. When the computer program is executed by the processing component, the computer program implements the data processing method described in the first aspect or the data query method described in the second aspect. In a fifth aspect, embodiments of the present disclosure provide a computer program product comprising a computer program / instructions. When the computer program / instructions are executed by the processing component, the computer program / instructions implement the data processing method described in the first aspect or the data query method described in the second aspect. In an embodiment of the present disclosure, a target area is divided into geographic grids, and a grid index is constructed. The grid index includes grid records in a key-value storage format. Each grid record uses the grid code of the geographic grid as the key and the primary key set of at least one spatial object corresponding to the geographic grid as the value. For a target spatial object in a database, at least one geographic grid covered by the target object can be determined, and the grid index is queried to determine the grid record corresponding to the at least one geographic grid. This grid record can then be updated based on the primary key of the target spatial object. Thus, the grid index can determine the target primary key set corresponding to the at least one geographic grid covered by the target object in a query command, and execute the query operation based on the target primary key set. The grid index enables fast querying of spatial objects in the target area, and a single geographic grid in the grid index can record multiple spatial objects, effectively reducing storage space. These and other aspects of the present disclosure will be more readily understood in the following description of the embodiments. BRIEF DESCRIPTION OF THE DRAWINGS To more clearly illustrate the embodiments of the present disclosure or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present disclosure. Those skilled in the art can derive other drawings based on these drawings without inventive effort.Figure 1 shows a flowchart of an embodiment of a data processing method provided by the present disclosure; Figure 2 shows a schematic diagram of geographic grid division in an actual application of an embodiment of the present disclosure; Figure 3 shows a schematic diagram of the structure of a spatial index tree in an actual application of an embodiment of the present disclosure; Figure 4 shows a schematic diagram of the file format in an actual application of an embodiment of the present disclosure; Figure 5 shows a flowchart of an embodiment of a data query method provided by the present disclosure; Figure 6 shows a schematic diagram of scene interaction in an actual application of an embodiment of the present disclosure; Figure 7 shows a schematic diagram of the structure of an embodiment of a data processing device provided by the present disclosure; Figure 8 shows a schematic diagram of the structure of an embodiment of a data query device provided by the present disclosure; and Figure 9 shows a schematic diagram of the structure of an embodiment of a computing device provided by the present disclosure. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS To help those skilled in the art better understand the present disclosure, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the accompanying drawings. Some processes described in the specification and claims of this disclosure, as well as in the aforementioned figures, include multiple operations that appear in a specific order. However, it should be understood that these operations may be executed out of the order in which they appear herein or in parallel. Operation numbers, such as 101 and 102, are merely used to distinguish between different operations and do not represent any specific order of execution. Furthermore, these processes may include more or fewer operations, and these operations may be executed sequentially or in parallel. It should be noted that terms such as "first" and "second" are used herein to distinguish between different messages, devices, modules, and the like, and do not imply a sequential order or limit "first" and "second" to different types. The technical solutions of the embodiments of the present disclosure are applied to the query scenario of spatial objects. Spatial objects refer to data structures used to represent geographic spatial information. They encompass various two-dimensional and three-dimensional geometric entities, such as point objects, which represent specific geographic locations, such as the coordinates of a city or the location of an earthquake; line objects, which represent continuous linear spatial features, such as roads, rivers, and regional boundaries, and are formed by a series of sequentially connected points; and area objects, which represent closed areas in two-dimensional space, such as land parcels, lakes, and national territories, and are formed by a series of enclosed line segments. To improve query efficiency, the inventors developed the technical solutions of the embodiments of the present disclosure after a series of research. In these embodiments, a grid index is used to achieve rapid querying of spatial objects in a target area. In addition, a single geographic grid in the grid index can record multiple spatial objects, effectively reducing storage space.Furthermore, grid indexes can be implemented based on multi-level geographic grids of varying precision, effectively representing spatial objects and thus improving query performance. It should be noted that the embodiments of this disclosure may involve the use of user data. In practical applications, user-specific personal data can be used in the solutions described herein, subject to compliance with applicable laws and regulations of the country in which the application is located (e.g., with explicit user consent, effective user notification, etc.), and within the scope permitted by applicable laws and regulations. It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, storage, and display, etc.) involved in this disclosure are all authorized by the user or fully authorized by all parties. The collection, use, and processing of relevant data must comply with the relevant laws, regulations, and standards of the relevant countries and regions, and corresponding operation portals are provided for users to choose to authorize or deny. It should be noted that the technical solutions of the embodiments of this disclosure are applicable to virtual network environments. The users described are generally "virtual users." Real users can register user accounts on the server through registration to obtain user identities in the network environment. The following will provide a clear and complete description of the technical solutions in the embodiments of the present disclosure, in conjunction with the accompanying drawings. Obviously, the described embodiments represent only a portion of the embodiments of the present disclosure, and are not exhaustive. All other embodiments derived by those skilled in the art based on the embodiments of the present disclosure without inventive effort are within the scope of protection of the present disclosure. Figure 1 is a flowchart of an embodiment of a data processing method provided by the embodiments of the present disclosure. The technical solutions of this embodiment can be executed by a query server. The method may include the following steps:

[0002] 101: Determine at least one geographic grid covered by the target spatial object in the database. The geographic grid can be obtained by dividing the target area. Alternatively, the space can be divided into geographic grids of uniform size. Furthermore, to effectively represent spatial objects, particularly irregular surface objects and line objects, and to improve query performance, the geographic grid can optionally be obtained by dividing the target area into multiple levels. The target area can be layered and segmented at different resolutions to form a series of nested grids with clear inclusion relationships, i.e., a multi-level nested geographic grid. Alternatively, the geographic grid can be rectangular, or alternatively, triangular, hexagonal, or other shapes. The target area can refer to the Earth's surface or a specific space corresponding to the database. Optionally, the target area can be divided into multiple levels of nested geographic grids. For example, a maximum and minimum accuracy thresholds for the resolution can be set. Based on the accuracy range defined by the maximum and minimum accuracy thresholds, the number of levels can be determined, such as global, continental, national, provincial, municipal, and county levels. Each level represents a different resolution. Starting from the lowest level (the level corresponding to the minimum accuracy threshold), the target area can be evenly divided according to a fixed grid size. For example, the first level of geographic grids on the Earth's surface may be large, covering several thousand square kilometers. Each geographic grid in the next level can then be further divided into multiple smaller geographic grids. Typically, each child grid (a geographic grid in the previous level) is half or one-quarter the size of the parent grid (a geographic grid in the next level). This ensures that the geographic grids in adjacent levels are nested, thereby achieving the division of the target area into multiple levels of nested grids. It should be noted that this is merely an illustration of possible geographic grid division methods, and the present disclosure is not limited thereto. A search can be performed based on the geographic location information (e.g., longitude and latitude coordinates) of the target spatial object to determine at least one geographic grid covered by it. For example, assuming that each subgrid is typically one-quarter the size of its parent grid, the search can be started from the lowest level based on the geographic coordinates of the target spatial object to determine the geographic grid covered by the target spatial object. The search can then be performed to find a geographic grid covered by the target object from the four subgrids corresponding to the next level above that geographic grid. Completely overlapping grids are not recorded repeatedly until the geographic grids at different levels covered by the target spatial object are determined. The at least one geographic grid can have different resolutions.For ease of understanding, Figure 2 shows a schematic diagram of geographic grid division. Rectangular boxes of different sizes represent geographic grids of different resolutions. Figure 2 illustrates eight geographic grids. As can be seen from Figure 2, geographic grid ⑧ is a subgrid of geographic grid ①. Furthermore, Figure 2 illustrates the overlay of three spatial objects of regular or irregular shapes. Spatial object 1 covers geographic grids ①, ②, ③, ④, ⑤, ⑥, and ⑦. Because geographic grid ⑧ completely overlaps with geographic grid ①, spatial object 1 does not need to record geographic grid ⑧ repeatedly. Spatial object 2 covers geographic grids ② and ③. Spatial object 3 covers geographic grid ⑧. In the disclosed embodiments, a target spatial object may refer to an unindexed spatial object in the database. Alternatively, the target spatial object may refer to a newly written spatial object. In a database, spatial objects can be stored in a data table structure or a key-value structure. The data table structure is obtained based on the conversion of the database's logical table. In the logical table, each spatial object is represented by a row of data, that is, a data record corresponding to each spatial object. Each row of data includes at least two columns. The at least two columns may include a primary key column for uniquely identifying a data record and a column for storing spatial geometry information. In addition, some attribute information columns may also be included, etc., which are not limited in this disclosure. Spatial geometric information is used to describe the geographic location of spatial objects and can use the Geometry type (a two-dimensional spatial geometry type used to describe the data type of spatial objects such as points, lines, and surfaces in two-dimensional space). A data table can be a specific implementation of a logical table on a storage medium, used to actually store data. Furthermore, based on the logical table structure, data records, that is, each row of data can be converted into a key-value storage result when written to the storage medium. Each row of data can be converted into (total number of columns - 1) key-value pairs for storage. The key of each key-value pair is the primary key + the name of the non-primary key column of the row of data, and the value can be the value corresponding to the non-primary key column. Optionally, the database can adopt a distributed architecture, and the above-mentioned data table or key-value storage structure can be divided into multiple data partitions for storage. Of course, spatial objects in the database can also be stored using other storage structures. This is merely an example and the present disclosure is not limited to this.

[0003] 102: Query a grid index to determine a grid record corresponding to at least one geographic grid.

[0004] 103: Update the grid record based on the primary key of the target spatial object. The grid index is used to determine, for the query object in the query command, a target primary key set consisting of values ​​in grid records corresponding to at least one geographic grid covered by the query object, and execute the query operation based on the target primary key set. The grid index is a database index established based on the geographic grids covered by the spatial object in the database. The grid index may include grid records corresponding to multiple geographic grids divided into the target area. Each grid record may be a key-value pair with the grid code of the geographic grid as the key and the primary key set consisting of the primary keys of at least one spatial object corresponding to the geographic grid as the value. Each geographic grid may record the spatial objects covering it in reverse order, so each geographic grid may correspond to one or more spatial objects. The grid code uniquely identifies the geographic grid. Depending on the encoding method, the code order of different geographic grids may vary. The grid records may be stored in order of the codes. The value in the grid record is the set consisting of the primary keys of at least one spatial object corresponding to the geographic grid. The grid index can be preconfigured, and the key in the grid record included in it is the grid code. Initially, the value in the grid record can be empty and is continuously updated as spatial objects are written. In the disclosed embodiment, after determining at least one geographic grid covered by the target spatial object, the grid index can be queried to retrieve the grid records corresponding to the at least one geographic grid. The primary key of the target spatial object can then be added to the value of the grid record to update the grid record. Therefore, for a query command, the grid index can be first searched to determine at least one grid record corresponding to the at least one geographic grid covered by the query object. The primary key in each grid is then read, and the components in the at least one grid record are combined into a target primary key set. The query operation can then be executed based on the target primary key set. Optionally, the query operation can be executed for data records that match the target primary key set. Specifically, based on the target object's primary key, the value in the grid record can be updated, and the target object's primary key can be added to the set of values ​​that constitute the target primary key set. The technical solutions of the embodiments of this disclosure enable data filtering based on grid indexes, thereby improving data query efficiency without having to traverse the entire database. Furthermore, grid indexes are constructed and implemented based on multi-level geographic grids of varying resolutions, enabling each divided geographic grid to effectively represent spatial objects of any type, thereby improving query performance and ensuring query accuracy. Furthermore, a single geographic grid in a grid index can record multiple spatial objects, significantly reducing storage space compared to one-to-one encoding methods.In some embodiments, to improve grid index write efficiency, after updating the grid record based on the primary key of the target spatial object, the method may further include: writing the updated grid record to memory; and, if the storage capacity in the memory satisfies a write condition, writing the multiple grid records stored in the memory to a storage medium. Querying the grid index to determine the grid record for at least one geographic grid may include: reading the grid record for at least one geographic grid from the grid index stored in the storage medium. Specifically, the grid record may be read from the grid index and updated. The updated grid record may first be written to memory, and if the storage capacity in the memory satisfies the write condition, the multiple grid records cached in the memory may then be written to the storage medium. Since the memory may already contain grid records, the updated grid record may be written to the memory and merged with the grid records already written to the memory. This merge may involve grid records corresponding to the same geographic grid, retaining the most recently updated grid record to ensure data real-time performance. By writing to memory first and then to the storage medium, writing to memory can improve write efficiency by leveraging the higher memory throughput efficiency. The storage medium may be a persistent storage medium, such as a disk, solid-state drive (SSD), network-attached storage (NAS), or storage area network (SAN), and this disclosure is not limited thereto. The write condition may, for example, be that the number of grid records reaches a predetermined number. The multiple grid records stored in the memory may be written to the storage medium as a new file. Furthermore, to further improve query efficiency, a spatial index tree corresponding to the multiple grid records may be constructed. This spatial index tree may be, for example, an R-tree, a quadtree, or an R+ tree. Using the spatial index tree, a specific geographic grid can be quickly located. Therefore, in some embodiments, writing the multiple grid records stored in the memory to the storage medium may include: saving the multiple grid records stored in the memory to a key-value storage structure; performing spatial partitioning based on the geographic coordinates of the geographic grids in the multiple grid records to construct a spatial index tree; and generating a file based on the spatial index tree and the key-value storage structure for writing to the storage medium. A spatial index tree can be generated by spatially partitioning the geographic grids based on the geographic coordinates of the geographic grids corresponding to the multiple grid records. The geographic coordinates can be longitude coordinates or latitude coordinates, etc., which is not limited in this disclosure. The spatial index tree can be a balanced spatial index tree that satisfies the following tree conditions, although this disclosure is not limited thereto. The spatial index tree can have any tree structure. The tree conditions can include:

[0005] 1) The tree must have at least two layers, and the maximum number of layers cannot exceed the threshold given by the system.

[0006] 2) Balanced tree properties: Let the maximum out-degree of the tree be m. Then, except for the root and leaf nodes, every other node has at least [ceil(m / 2)] children, and all leaf nodes appear on the same level. Here, m is a positive integer, and ceil is used to round upwards. Return the smallest integer greater than or equal to the given number (m / 2).

[0007] 3) Each node represents a spatial range. A grid record is logically stored in the smallest node (intermediate node or leaf node) that can contain it. This can actually be stored in a key-value storage structure, where the node actually stores the storage address of the grid record in the key-value storage structure. Therefore, optionally, the file stored in the storage medium may include a file header, a spatial index tree, and a key-value storage structure; the file header includes a pointer to the root node of the spatial index tree; the spatial index tree includes a root node, intermediate nodes, and leaf nodes; the root node, intermediate nodes, and leaf nodes each store their corresponding spatial range and pointers to their child nodes; and the leaf node or intermediate node stores a pointer to the corresponding grid record, i.e., the storage address. In the key-value storage structure, the grid records can be arranged in coding order. That is, saving the multiple grid records stored in memory to the key-value storage structure may include: saving the multiple grid records to the key-value storage structure in grid coding order. By arranging them in coding order, geographic grids can be quickly located, effectively improving grid retrieval efficiency. For ease of understanding, taking the geographic grid coverage of spatial objects 1 through 3 shown in FIG2 as an example, assume that the geographic coordinates of these eight geographic grids range from y0 to y5, with the maximum geographic coordinate being y5 and the minimum being y0. The geographic coordinates here may be longitude coordinates, although latitude coordinates may also be used for spatial division. FIG3 shows a spatial index tree constructed based on these eight geographic grids, although the present disclosure is not limited thereto. The key-value storage structure corresponding to these eight geographic grids, arranged in a coding order, may be as shown in Table 1 below, where the coding order is not limited in this disclosure. Table 1: In some embodiments, since a geographic grid may record multiple spatial objects, the median value of the grid record corresponding to the geographic grid is also a primary key list formed by the primary keys of the multiple spatial objects. To prevent a single row of data from being too large, which could affect query read efficiency, the method may optionally include: if the primary key list corresponding to any geographic grid exceeds a first threshold, splitting the primary key list into multiple groups, each of which forms multiple sets of key-value pairs with the geographic grid. For ease of understanding, as shown in Table 2 below, assuming that the primary key list corresponding to geographic grid 2 exceeds the first threshold, it can be split into two groups of key-value pairs for storage. Table 2: The grid record corresponding to the geographic grid may include multiple sets of key-value pairs, with the median value of each key-value pair being a group of primary keys. The key may be the grid code of the geographic grid, or may be the grid code of the geographic grid plus a group identifier, such as '0' or '1' in the table. The group identifier can be used to identify each key-value pair corresponding to the geographic grid. To further improve query and write efficiency, a distributed storage structure can be used to store grid indexes. Therefore, the grid record corresponding to the at least one geographic grid covered by the target spatial object can be read from the index partition corresponding to the at least one geographic grid. Therefore, in some embodiments, reading the grid record of the at least one geographic grid from the grid index stored in the storage medium may include: determining the index partition corresponding to the at least one geographic grid in the storage medium; and reading the grid record corresponding to the at least one geographic grid from the file stored in the index partition. In some embodiments, when grid records are first written to memory, the memory may be divided into multiple memory partitions for storage. Writing the updated grid record to the memory may include writing the updated grid record to the corresponding memory partition. Writing the multiple grid records stored in the memory to the storage medium may include writing the multiple grid records stored in the memory partition to the corresponding index partition in the storage medium. Metadata may be generated based on the storage status of the index partition and the memory partition. The metadata records the storage status of the index partition or memory partition and may include, for example, the file address corresponding to the index partition and the range of the grid records. Thus, the index partition corresponding to at least one geographic grid in the storage medium can be determined by searching the metadata, and the grid record corresponding to the at least one geographic grid can be read from the file stored in the index partition. To facilitate understanding, FIG4 illustrates the association between metadata, index partitions, and files. Each index partition corresponds to a corresponding memory partition, and the memory partition stores the latest data corresponding to the index partition. As shown in FIG4 , a file may include a file header, a spatial index tree, and a key-value storage structure. The file header includes a pointer to the root node of the spatial index tree. The spatial index tree includes a root node, intermediate nodes, and leaf nodes. The root node, intermediate nodes, and leaf nodes each store a corresponding spatial range and a pointer to their child nodes. A leaf node or intermediate node stores a pointer to a corresponding grid record, i.e., a storage address. FIG4 shows, for example, grid records pointing to geographic grid 1, geographic grid 5, and geographic grid 6. The key-value storage structure in FIG4 is described using Table 1 above as an example.In some embodiments, as described above, a spatial index tree can be constructed for the geographic grid corresponding to each grid record in each file. Reading the grid record corresponding to at least one geographic grid from the file stored in the index partition includes: searching the spatial index tree in the file to determine the storage address corresponding to the at least one geographic grid; and reading the grid record corresponding to the at least one geographic grid from the key-value storage structure of the file based on the storage address. For example, the spatial index tree query process, still taking the spatial index tree shown in FIG3 as an example, can be recursively traversed downwards, starting from the root node, based on the geographic coordinates of any geographic grid. At each node, filtering is performed based on the spatial relationship (e.g., intersection, inclusion, etc.) between the geographic coordinates and the corresponding spatial range until a final query result is obtained. In some embodiments, since the number of files may increase, to reduce the number of files, the method may further include: when the number of files in the index partition reaches a second threshold, performing a file merge operation on the files in the index partition, and merging the spatial index trees corresponding to the multiple merged files, with the merged spatial index tree serving as the merged spatial index tree corresponding to the merged files. In other words, multiple small files can be merged into a large file, and the spatial index trees corresponding to the multiple small files can also be merged to form the spatial index tree corresponding to the large file. Of course, the corresponding spatial index tree can also be regenerated based on the large file, and this process is consistent with the spatial index tree merging process. The file merging operation may include deduplication and sorting. Specifically, the grid records in multiple files are reordered according to the encoding order, and the most recently updated grid records corresponding to the same geographic grid are retained to achieve deduplication, thereby obtaining the merged file. Figure 5 is a flowchart of an embodiment of a data query method provided by an embodiment of the present disclosure. The technical solution of this embodiment is executed by a query server. This embodiment introduces the technical solution of the present disclosure from the perspective of data query. The method may include the following steps:

[0008] 501: Obtain a query command and determine the object to be queried. The query command can be provided by the client, for example, a query statement sent by the client. The query command can include search conditions to determine the object to be queried; it can also include filter conditions to limit the query scope. In practical scenarios, for example, if one wants to query roads in a certain area, that area is also the object to be queried; if one wants to query the area to which a certain location belongs, that location is also the object to be queried. The search conditions can include geographic location information, such as geographic coordinates. 502: Determine at least one geographic grid covered by the object to be queried. The at least one geographic grid covered by the object to be queried can be determined based on the geographic location information of the object to be queried. It should be noted that the at least one geographic grid covered by the object to be queried is determined in the same manner as the at least one geographic location covered by the target spatial object in the corresponding embodiments described above, and will not be repeated here. The geographic grids are obtained by dividing the target area. The division method has been described previously and will not be repeated here. The at least one geographic grid covered by the object to be queried can have different resolutions.

[0009] 503: Query the grid record corresponding to the at least one geographic grid from the grid index to obtain a target primary key set consisting of the values ​​in the grid record corresponding to the at least one geographic grid. After determining the at least one geographic grid covered by the query object, the grid record corresponding to the at least one geographic grid can be queried from the grid index. A grid record includes a key-value pair with the grid code of a geographic grid as the key and a set of at least one spatial object primary key corresponding to the geographic grid as the value. A geographic grid can correspond to one or more grid records, and multiple grid records are obtained by grouping the primary key list corresponding to the geographic grid. The grid index is generated by updating the corresponding grid record based on the at least one geographic grid covered by the target spatial object in the database and its primary key. The generation or updating of the grid index and grid record can be found in the previous embodiments and will not be repeated here. First, at least one geographic grid covered by the query object can be encoded as a grid code. Based on the grid code, the grid records hit by the query are searched from the grid index. This results in the values ​​in the grid records, i.e., a set of at least one primary key. The primary key sets in the at least one grid record hit by the at least one geographic grid covered by the query object are then merged to obtain the target primary key set. In practical applications, the query server may include an optimizer and an executor. The optimizer generates query conditions for the grid index based on the at least one geographic grid, and the executor performs queries from the grid index according to the query conditions.

[0010] 504: Execute the query operation corresponding to the query command based on the target primary key set. The target primary key set can limit the query scope, allowing the query operation to be executed within the query scope corresponding to the target primary key set. For example, the query operation can be executed for data records matched by the target primary key set. Optionally, the filter conditions in the query command can be updated based on the target primary key set, removing portions of the filter conditions that do not match the target primary key set. In practical applications, the optimizer can generate a query plan corresponding to the query command, remove portions of the query plan that do not match the target primary key set, and the executor executes the query plan to implement the query operation. A query plan is an execution strategy formulated before query execution, detailing the steps and sequence for retrieving data from the database. In the disclosed embodiments, the use of grid indexes can achieve data filtering, thereby improving data query efficiency without traversing all database data. The grid index is constructed and implemented based on multi-level geographic grids of varying resolutions, enabling the divided geographic grids to effectively represent spatial objects of any object type, thereby improving query performance and ensuring query accuracy. Furthermore, a single geographic grid in the grid index can record multiple spatial objects, effectively reducing storage space. In some embodiments, since grid records can be written to memory before being written to a storage medium, grid records will also exist in memory. Therefore, the aforementioned steps include querying grid records for at least one geographic grid from the grid index to obtain a target primary key set; and querying grid records for at least one geographic grid from both the memory and the grid index to obtain a target primary key set. Optionally, grid records for at least one geographic grid can be read from the grid index stored in the storage medium and from the memory in parallel. In some embodiments, querying grid records for at least one geographic grid from the memory and the grid index to obtain a target primary key set may include: reading corresponding grid records for at least one geographic grid from memory; reading corresponding grid records for at least one geographic grid from the grid index; and merging the grid records obtained from the memory with those obtained from the grid index to obtain grid records for the at least one geographic grid. Since the latest data is usually stored in the memory, the merging process may include retaining only the grid records obtained from the memory for the same geographic grid.In some embodiments, since a grid index can be stored in a distributed manner, that is, the grid index is distributedly stored in the form of files across multiple index partitions, querying the grid records of at least one geographic grid from the grid index to obtain the target primary key set may include: determining the index partition corresponding to the at least one geographic grid; and reading the grid records corresponding to the at least one geographic grid from the file stored in the index partition to obtain the target primary key set. In some embodiments, querying the grid records corresponding to the at least one geographic grid from memory may include querying the grid records corresponding to the at least one geographic grid from the memory partition corresponding to the at least one geographic grid. The memory partition and index partition corresponding to the at least one geographic grid can be determined by querying metadata. The metadata records the storage status of the index partition or memory partition, and may include, for example, the file address corresponding to the index partition or memory partition, as well as the grid record range. In some embodiments, a spatial index tree may be created for each file in an index partition. Each file may include a file header, a spatial index tree, and a key-value storage structure. The file header includes a pointer to the root node of the spatial index tree. The spatial index tree includes a root node, intermediate nodes, and leaf nodes. The root node, intermediate nodes, and leaf nodes each store their corresponding spatial ranges and pointers to their child nodes. A leaf node or intermediate node stores a pointer to a corresponding grid record, i.e., a storage address. The spatial index tree construction process can be described in detail in the corresponding embodiments above. Reading a grid record corresponding to at least one geographic grid from a file stored in the index partition to obtain a target primary key set may include: searching the spatial index tree in the file stored in the index partition to determine the storage address of the at least one grid record; and obtaining the target primary key set in the at least one grid record from the key-value storage structure of the file based on the storage address. It should be noted that the spatial index tree query process can, for example, be based on the geographic coordinates of any geographic grid, recursively traversing the tree hierarchy starting from the root node. At each node, filtering is performed based on the spatial relationship between the geographic coordinates and the corresponding spatial range (such as intersection, inclusion, etc.) until the final query result is obtained.FIG6 is a schematic diagram of a scenario interaction in a practical application of an embodiment of the present disclosure. The technical solutions of the embodiments of the present disclosure can be applied to a database system. The database system may include, for example, a client 601, a query server 602, and a storage service 603 in FIG6 . The client 601, query server 602, and storage service 603 may be deployed on the same physical machine. Alternatively, the client 601 may be a front-end device, while the query server 602 and storage service 603 may be back-end devices. The query server 602 and storage service 603 may be deployed on the same physical machine, or on different physical machines, which is not limited in this disclosure. The storage service 603 provides a storage medium to provide persistent storage services, while the query server 602 may provide memory, etc. In FIG6 , the data partition 60 stores spatial objects in the form of files, and the index partition 61 stores grid indexes in the form of files. For example, the file format in the index partition can be seen in FIG4 . The files in the data partition are not limited in this disclosure. Based on a write command from client 601, query server 602 can write the object to be written to the storage medium provided by storage server 603. The object to be written can be first written to the corresponding memory partition and then to the corresponding data partition, and the corresponding metadata can be updated based on the write operation. Data can be stored in a data partition using a data table, a key-value storage structure, or the like, which is not limited in this disclosure. The object to be written can serve as a target spatial object. For this target spatial object, the geographic grid it covers can be first determined. The index partition corresponding to the geographic grid can be determined by querying metadata, and the matching grid record can be read from it. The grid record can then be updated based on the primary key of the target spatial object. The updated grid record is first written to the corresponding memory partition and then to the corresponding index partition, and the corresponding metadata can be updated simultaneously. In response to the query command sent by client 601, query server 602 can utilize an optimizer and executor to search for corresponding grid records from memory partitions and index partitions based on at least one geographic grid covered by the query object, obtain a target primary key set, and then update the query plan based on the target primary key set, thereby executing the query plan and achieving the query objective. This query of files in the index partition can further improve query efficiency through the spatial index tree. The technical solution of the disclosed embodiment calculates the multi-level nested geographic grids covered by each spatial object. Geographic grids of different precision levels are more flexible to use than fixed-precision grids, effectively reflecting the size and shape of spatial objects and providing better filtering effects on irregular surface and line objects.Furthermore, a grid index can be constructed by combining a spatial index tree with key-value storage of grid records. This distributed storage approach allows data to be written to memory first and then persisted, resulting in high query and write efficiency and minimal storage space usage. During queries, the spatial index tree of metadata and index files can be used to quickly filter out most non-compliant grid records. The ordered nature of grid codes can then be leveraged to quickly locate eligible grid records, effectively improving grid retrieval efficiency. This reduces table scans and computational complexity without changing the data table structure, achieving high query efficiency. During writes, data can be written to memory first and then merged into index partitioned files. This leverages the higher throughput efficiency of memory to effectively improve write efficiency. During storage, each geographic grid can record multiple spatial objects, stored as a key-value pair. This effectively compresses storage space and reduces data redundancy compared to one-to-one coded indexes. By combining the spatial index tree with geographic grids, the number of levels and nodes in the spatial index tree can be controlled, reducing storage space. Figure 7 is a schematic diagram of an embodiment of a data processing device provided by the present disclosure. The device may include: a first determination module 701 for determining at least one geographic grid covered by a target spatial object in a database; the geographic grid is obtained by dividing a target area; a first query module 702 for querying a grid index to determine a grid record corresponding to the at least one geographic grid; the grid record includes a key-value pair with a grid code as a key and a primary key set of at least one spatial object as a value; an index update module 702 for updating the grid record based on the primary key of the target spatial object; the grid index is used to determine the primary key set corresponding to the at least one geographic grid covered by the query object in a query command, and to perform a query operation based on the primary key set. In some embodiments, the method may further include: a data writing module for writing the updated grid record to a memory; and, upon detecting that the storage capacity of the memory satisfies a write condition, writing the multiple grid records stored in the memory to a storage medium. In some embodiments, the data writing module writes the multiple grid records stored in the memory to the storage medium, including: saving the multiple grid records stored in the memory to a key-value storage structure; performing spatial division based on the geographic coordinates of the geographic grids in the multiple grid records to construct a spatial index tree; and generating a file based on the spatial index tree and the key-value storage structure to write to the storage medium.In some embodiments, the file includes a file header, the spatial index tree, and the key-value storage structure; the file header includes a pointer to a root node of the spatial index tree; the spatial index tree includes the root node, intermediate nodes, and leaf nodes; the root node, the intermediate nodes, and the leaf nodes each store a corresponding spatial range and a pointer to its child node; and the leaf node or the intermediate node stores a pointer to a corresponding grid record. In some embodiments, the data writing module saving the multiple grid records stored in the memory to the key-value storage structure includes: saving the multiple grid records to the key-value storage structure in grid code order. In some embodiments, the data writing module is further configured to, when a primary key list corresponding to any geographic grid reaches a first number threshold, split the primary key list into multiple groups to form multiple key-value pairs with the geographic grid. In some embodiments, the data writing module writing the updated grid record to the memory includes: writing the updated grid record to the corresponding memory partition; and the data writing module writing the multiple grid records stored in the memory to the storage medium includes: writing the multiple grid records stored in the memory partition to the corresponding index partition in the storage medium. In some embodiments, the apparatus may further include: a file merging module configured to, when the number of files in the index partition reaches a second threshold, perform a file merging operation on the files in the index partition, and merge the spatial index trees corresponding to the multiple merged files, so that the merged spatial index tree serves as the spatial index tree corresponding to the merged file. In some embodiments, the apparatus may further include: a grid partitioning module configured to partition the target area into multiple levels according to different resolutions to obtain a multi-level nested geographic grid. The data processing apparatus shown in FIG7 can execute the data processing method described in the embodiment shown in FIG1 , and its implementation principles and technical effects will not be further described. The specific manner in which the various modules and units of the data processing apparatus in the above-mentioned embodiments perform operations has been described in detail in the embodiments related to the method and will not be elaborated upon here.FIG8 is a schematic diagram of the structure of an embodiment of a data query device provided by an embodiment of the present disclosure. The device may include: a second determination module 801 for obtaining a query command and determining an object to be queried; a third determination module 802 for determining at least one geographic grid covered by the object to be queried; the geographic grid being obtained by dividing a target area; a second query module 803 for querying grid records of the at least one geographic grid from a grid index to obtain a primary key set; the grid record comprising a key-value pair with the grid code of any geographic grid as a key and the primary key set of at least one spatial object as a value; the grid index updating and generating corresponding grid records based on the at least one geographic grid covered by the target spatial object in a database and its primary key; and a third query module 804 for executing a query operation corresponding to the query command in combination with the primary key set. In some embodiments, the second query module specifically queries grid records of the at least one geographic grid from memory and the grid index to obtain the primary key set. In some embodiments, the second query module specifically reads a corresponding grid record from memory based on the at least one geographic grid; reads a corresponding grid record from the grid index based on the at least one geographic grid; and merges the grid record obtained from the memory with the grid record obtained from the grid index to obtain a grid record for the at least one geographic grid. In some embodiments, the grid index is distributed across multiple index partitions in the form of a file. The second query module may specifically determine the index partition corresponding to the at least one geographic grid; read the grid record corresponding to the at least one geographic grid from the file stored in the index partition to obtain a primary key set. In some embodiments, the file includes a file header, a spatial index tree, and a key-value storage structure; the spatial index tree is constructed based on the geographic grids in the multiple grid records corresponding to the file; the multiple grid records are stored in the key-value storage structure; the second query module may specifically search the spatial index tree in the file stored in the index partition to determine the storage address of the at least one grid record; and obtain the primary key set in the at least one grid record from the key-value storage structure of the file based on the storage address. In some embodiments, the third query module may execute the query operation corresponding to the query command within the query range corresponding to the primary key set. The data query device shown in FIG8 can execute the data query method described in the embodiment shown in FIG5 , and its implementation principles and technical effects are not further described. The specific manner in which the various modules and units of the data processing device in the above embodiment execute operations has been described in detail in the embodiment related to the method and will not be elaborated on here.The present disclosure also provides a computing device, as shown in FIG9 . The computing device may include a storage component 901 and a processing component 902. The storage component 901 stores one or more computer instructions, wherein the one or more computer instructions are called and executed by the processing component 902 to implement the data processing method described in the embodiment shown in FIG1 or the data query method described in the embodiment shown in FIG5 . Of course, the computing device may also include other components, such as an input / output interface, a display component, and a communication component. The input / output interface provides an interface between the processing component and peripheral interface modules, which may be output devices, input devices, etc. The communication component is configured to facilitate wired or wireless communication between the computing device and other devices. The processing component may include one or more processors to execute computer instructions to complete all or part of the steps in the above-described method. Of course, the processing component can also be implemented as one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components, to perform the above-mentioned methods. The storage component is configured to store various types of data to support operations on the terminal. The storage component can be implemented by any type of volatile or non-volatile storage device, or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disks, or optical disks. It should be noted that the computing device described above can be a physical device or an elastic computing host provided by a cloud computing platform. It can be implemented as a distributed cluster consisting of multiple servers or terminal devices, or as a single server or a single terminal device. The present disclosure also provides a computer-readable storage medium storing a computer program. When executed by a processing component, the computer program can implement the data processing method described in the embodiment shown in FIG1 or the data query method described in the embodiment shown in FIG5 . The computer-readable medium can be included in the electronic device described in the above embodiments, or it can exist independently and not be incorporated into the electronic device.The present disclosure also provides a computer program product comprising a computer program / instructions carried on a computer-readable storage medium. When executed by a processing component, the computer program / instructions can implement the data processing method described in the embodiment shown in FIG1 or the data query method described in the embodiment shown in FIG5 . In such an embodiment, the computer program can be downloaded and installed from a network and / or installed from a removable medium. When executed by a processor, the computer program performs the various functions defined in the system of the present disclosure. It should be noted that the present disclosure may involve the use of user data. In actual applications, user-specific personal data can be used in the solutions described herein, subject to compliance with applicable laws and regulations of the country (e.g., with explicit user consent, effective user notification, etc.), and within the scope permitted by applicable laws and regulations. Those skilled in the art will clearly understand that, for ease of description and brevity, the specific operating procedures of the systems, devices, and units described above can refer to the corresponding procedures in the aforementioned method embodiments and will not be further described here. The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one location or distributed across multiple network units. Some or all of these modules may be selected based on actual needs to achieve the objectives of the present embodiments. Those skilled in the art will be able to understand and implement the present embodiments without inventive effort. Through the description of the above embodiments, those skilled in the art will clearly understand that each embodiment can be implemented using software plus a required general-purpose hardware platform, or alternatively, hardware. Based on this understanding, the essence of the above technical solutions, or the portion that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a magnetic disk, or an optical disk, and includes instructions for enabling a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or portions thereof. Finally, it should be noted that the above embodiments are merely illustrative of the technical solutions of the present disclosure and are not intended to limit the present disclosure. Although the present disclosure has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they may modify the technical solutions described in the aforementioned embodiments or replace some of the technical features therein with equivalents. However, such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present disclosure.

Claims

Claims 1. A data processing method, comprising: Determining at least one geographic grid covered by the target spatial object in the database; The geographic grid is obtained by dividing the target area; Querying a grid index to determine a grid record corresponding to the at least one geographic grid; the grid record includes a key-value pair with a grid code as a key and a primary key set of at least one spatial object as a value; The grid record is updated according to the primary key of the target spatial object. The grid index is used to determine, for the object to be queried in the query command, a target primary key set consisting of values ​​in grid records corresponding to at least one geographic grid covered by the object to be queried, and the query operation is performed in combination with the target primary key set.

2. The method according to claim 1, after updating the grid record according to the primary key of the target spatial object, the method further comprises: Write the updated grid records into memory; When it is detected that the storage capacity of the memory satisfies a writing condition, the plurality of grid records stored in the memory are written into a storage medium.

3. The method according to claim 2, wherein writing the plurality of grid records stored in the memory into the storage medium comprises: Saving the plurality of grid records stored in the memory to a key-value storage structure; Performing spatial partitioning based on the geographic coordinates of the geographic grids in the plurality of grid records to construct a spatial index tree; Based on the spatial index tree and the key-value storage structure, a file is generated to be written into the storage medium.

4. The method according to claim 3, wherein the file comprises a file header, the spatial index tree, and the key-value storage structure; the file header comprises a pointer to a root node of the spatial index tree; The spatial index tree includes the root node, intermediate nodes, and leaf nodes; the root node, the intermediate nodes, and the leaf nodes respectively store their corresponding spatial ranges and pointers to their child nodes; The leaf node or the intermediate node stores a pointer pointing to a corresponding grid record.

5. The method according to claim 3 or 4, wherein saving the plurality of grid records stored in the memory to a key-value storage structure comprises: The plurality of grid records are stored in a key-value storage structure according to the grid coding order.

6. The method according to claim 5, further comprising: The primary key list corresponding to any geographic grid reaches a first quantity threshold; The primary key list is split into multiple groups to form multiple groups of key-value pairs with the geographic grid.

7. The method according to any one of claims 2 to 6, wherein writing the updated grid record into the memory comprises: Write the updated grid records into the corresponding memory partition; Writing the plurality of grid records stored in the memory into the storage medium includes: writing the plurality of grid records stored in the memory partition into a corresponding index partition in the storage medium.

8. The method according to claim 7, further comprising: When the number of files in the index partition reaches a second number threshold, a file merging operation is performed on the files in the index partition, and spatial index trees corresponding to the merged multiple files are merged, so that the merged spatial index tree is used as the spatial index tree corresponding to the merged file.

9. The method according to any one of claims 1 to 8, further comprising: The target area is divided into multiple levels according to different resolutions to obtain a multi-level nested geographic grid.

10. A data query method, comprising: Get the query command and determine the object to be queried; Determining at least one geographical grid covered by the object to be queried; The geographic grid is obtained by dividing the target area; Querying a grid record corresponding to the at least one geographic grid from a grid index to obtain a target primary key set consisting of values ​​in the grid record corresponding to the at least one geographic grid; the grid record includes a key-value pair with a grid code of any geographic grid as a key and a primary key set of at least one spatial object as a value; the grid index updates and generates corresponding grid records based on at least one geographic grid covered by the target spatial object in a database and its primary key; and executing a query operation corresponding to the query command in combination with the target primary key set.

11. The method according to claim 10, wherein the grid records of the at least one geographic grid are queried from a grid index to obtain a target primary key set; and the grid records of the at least one geographic grid are queried from a memory and the grid index to obtain a target primary key set.

12. The method according to claim 11, wherein querying the grid records of the at least one geographic grid from the memory and the grid index to obtain the target primary key set comprises: Based on the at least one geographic grid, reading a corresponding grid record from a memory; Based on the at least one geographic grid, reading a corresponding grid record from the grid index; The grid obtained from the memory is merged with the grid record obtained from the grid index to obtain the grid record of the at least one geographic grid.

13. The method according to claim 11, wherein the grid index is distributedly stored in a plurality of index partitions in a file format, and the querying the grid records of the at least one geographic grid from the grid index to obtain a target primary key set comprises: Determining an index partition corresponding to the at least one geographic grid; The grid record corresponding to the at least one geographic grid is read from the file stored in the index partition to obtain a target primary key set.

14. The method according to claim 13, wherein the file comprises a file header, a spatial index tree, and a key-value storage structure; the spatial index tree is constructed based on geographic grids in a plurality of grid records corresponding to the file; and the plurality of grid records are stored as the key-value storage structure; The reading of the grid record corresponding to the at least one geographic grid from the file stored in the index partition to obtain the target primary key set includes: Searching a spatial index tree in a file stored in the index partition to determine a storage address of the at least one grid record; Based on the storage address, a target primary key set in the at least one grid record is obtained from the key-value storage structure of the file.

15. The method according to any one of claims 10 to 14, wherein the executing the query operation corresponding to the query command in combination with the target primary key set comprises: Execute the query operation corresponding to the query command within the query range corresponding to the target primary key set.

16. A computing device comprising a processing component and a storage component; the storage component storing one or more computer instructions; the one or more computer instructions being called and executed by the processing component to implement the data processing method according to any one of claims 1 to 9 or the data query method according to any one of claims 10 to 14.

17. A computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processing component, the data processing method according to any one of claims 1 to 9 or the data processing method according to any one of claims 10 to 14 is implemented. The data query method described in item .

18. A computer program product, comprising a computer program / instruction, wherein when the computer program / instruction is executed by a processing component, the computer program / instruction implements the data processing method according to any one of claims 1 to 9 or the data query method according to any one of claims 10 to 14. 16

Images