A method for query optimization of spatial objects

Abstract

The present application relates to a kind of query optimization methods of spatial object, belong to spatial object query technical field, solve the problem of slow query in prior art.Method includes: obtaining historical query data;Based on historical query data, the query frequency and average query interval time of each spatial object at each time step are calculated;Based on the query frequency and average query interval time of each spatial object at each time step, training sample set is constructed;Hot area prediction and multi-step query frequency prediction of spatial object are constructed for multi-task prediction network;Based on training sample set, the multi-task prediction network is trained to obtain hot prediction network;The multi-step historical data of each spatial object in the to-be-predicted region before to-be-predicted time is input into the hot prediction network, and the prediction result of each to-be-predicted region is obtained;According to the prediction result of each to-be-predicted region, spatial object data is preloaded. Efficient query is realized.

Description

Technical Field

[0001] This invention relates to the field of spatial object query technology, and in particular to a method for optimizing spatial object queries. Background Technology

[0002] In today's digital age, spatial object data has extremely important application value in many fields, such as Geographic Information Systems (GIS), map navigation services, geofencing monitoring, spatial data analysis, intelligent transportation systems, and urban planning and management. These applications typically require fast and efficient query operations on large amounts of spatial object data to meet users' real-time or high-efficiency needs.

[0003] Spatial object data mainly includes geometric elements such as points, lines, surfaces, and their combinations, used to represent entities with spatial location and shape characteristics. For example, a point can represent a specific geographical location (such as a landmark building, base station location, etc.), a line can represent linear features such as roads and rivers, and a surface can represent a region (such as an administrative division, lake, or land parcel).

[0004] With the continuous development of satellite remote sensing technology, geographic surveying and mapping technology, and geographic information system (GIS) data acquisition technology, spatial object data is experiencing explosive growth. For example, a geographic information system of a large city may contain hundreds of millions of spatial object data points. This massive amount of data poses a huge challenge to query operations. When a user initiates a query request, the system needs to filter spatial objects that meet the query criteria from the massive amount of data. This involves a large number of data reading and processing operations, resulting in long query response times. Summary of the Invention

[0005] Based on the above analysis, the embodiments of the present invention aim to provide a spatial object query optimization method to solve the problem of slow spatial object query in the present invention.

[0006] On one hand, embodiments of the present invention provide a method for optimizing queries of spatial objects, comprising the following steps:

[0007] Obtain historical query data; calculate the query frequency and average query interval for each spatial object at each time step based on the historical query data; construct a training sample set based on the query frequency and average query interval for each spatial object at each time step;

[0008] A multi-task prediction network is constructed, which is used for hotspot region prediction and multi-step query frequency prediction of spatial objects; the hotspot prediction network is obtained by training the multi-task prediction network based on the training sample set.

[0009] The historical data of the spatial objects in each region to be predicted before the time to be predicted are input into the popular prediction network to obtain the prediction result for each region to be predicted.

[0010] Spatial object data is preloaded based on the prediction results for each region to be predicted.

[0011] Based on further improvements to the above technical solution, a training sample set is constructed based on the query frequency and average query interval of each spatial object at each time step, including:

[0012] For each region, the feature data for each time step includes the meteorological data for that time step, the object type, geographic type, number of queries, and average query interval for each spatial object in that region;

[0013] For each region, T IN The feature data of each time step is used as the input data for one sample, with the T... IN T after one time step OUT Within each time step, a training sample set is constructed using labels such as whether the area is a popular area and the query frequency of each spatial object.

[0014] Based on further improvements to the above technical solution, the training loss is calculated using the following formula:

[0015]

[0016] Where N represents the number of samples in the current training batch, y i p represents the label indicating whether the i-th sample is a popular area. i T represents the predicted probability that the i-th sample is a popular area. OUT z represents the number of time steps for prediction. i,t This represents the actual query frequency data for each spatial object of the i-th sample at time step t. Let α represent the predicted query frequency of each spatial object for the i-th sample at time step t, where α and β represent weights.

[0017] Based on further improvements to the above technical solution, the multi-task prediction network includes a first prediction network and a second prediction network:

[0018] The first prediction network is used to predict popular areas based on the input data; the second prediction network is used to predict the multi-step query frequency of spatial objects based on the input data.

[0019] The second prediction network includes:

[0020] The coding network, consisting of multiple encoders connected in sequence, is used to extract multi-level features from the input data based on an autocorrelation mechanism;

[0021] The decoding network, consisting of multiple decoders connected in sequence, is used to decode the features extracted by the encoding network to obtain the query frequency prediction results.

[0022] Based on the further improvement of the above technical solution, each encoder includes two sequence decomposition modules for sequence decomposition, and feature extraction is performed in the following manner:

[0023]

[0024] in, This represents the periodic term obtained by the first sequence decomposition module of the l-th encoder. This represents the periodic term obtained by the second sequence decomposition module of the l-th encoder. This represents the output data of the (l-1)th encoder. This represents the output data of the l-th encoder, FFN(·) represents the feedforward layer, AC1(·) represents calculating the first autocorrelation, and SD(·) represents sequence decomposition. When l=1, X represents the input data X of the coding network en When l>1, This represents the output data of the (l-1)th encoder.

[0025] Based on further improvements to the above technical solution, each decoder includes three sequence decomposition modules for sequence decomposition; decoding is performed in the following manner:

[0026]

[0027] in, This represents the periodic term obtained by the i-th sequence decomposition module of the l-th decoder. This represents the trend term obtained by the i-th sequence decomposition module of the l-th decoder. Let represent the output data of the Nth encoder, FFN(·) represent the feedforward layer, Ac1(·) represent calculating the first autocorrelation, AC2(·,·) represent calculating the second autocorrelation, and SD(·) represent sequence decomposition. This represents the trend term output by the l-th decoder. This represents the periodic term output by the l-th decoder. This represents the weight of the trend term obtained by the i-th sequence decomposition module of the l-th decoder;

[0028] When l>1, This represents the periodic term output by the (l-1)th decoder. This represents the trend term output by the (l-1)th decoder; when l=1, Represents the initial periodic term. This represents the initial trend term.

[0029] Based on further improvements to the above technical solution, the initial period term and initial trend term are calculated using the following formula:

[0030]

[0031] Where Concat(·) represents concatenation, and SD(·) represents sequence decomposition. X represents the input data X of the coding network en The The sequence consisting of the first element up to the i-th element, where i represents the input data X. en The length of X, where X0 represents a sequence filled with 0s, X Mean Indicates by A sequence filled with the mean.

[0032] Based on further improvements to the above technical solution, the prediction result is obtained using the following formula:

[0033]

[0034] Among them, X P This represents the prediction result of the multi-task prediction network. This represents the periodic term output by the Mth decoder. This represents the trend term output by the Mth decoder, where M represents the number of decoders, and W... s and W T This represents the weight matrix.

[0035] Based on further improvements to the above technical solution, the first autocorrelation is calculated using the following method:

[0036]

[0037] τ1,…,τ k =argTopk(R Q,K (τ))

[0038]

[0039] Q = W Q X2

[0040] K = W K X2

[0041] V = W V X2

[0042] Where AC1(X2) represents the calculation of the first autocorrelation of sequence X2, τ represents the period length, τ1,…,τ kR represents the selected k period lengths, argTopk(·) represents taking the k period lengths corresponding to the top k autocorrelation coefficients, and R Q,K R(τ) represents the autocorrelation coefficient calculated for all period lengths, and R(τ1) represents the autocorrelation coefficient corresponding to the first period length. k ) represents the autocorrelation coefficient corresponding to the k-th period length, and SoftMax(·) represents the softmax function. This represents the inverse fast Fourier transform. Represents the Fast Fourier Transform. W represents the conjugate of the Fast Fourier Transform. Q W represents the query coefficient matrix. K W represents the bond coefficient matrix. V Roll(V,τ) represents the coefficient matrix. i ) indicates that τ is applied to V. i Delay.

[0043] Based on further improvements to the above technical solution, the second autocorrelation is calculated using the following method:

[0044]

[0045] τ1,…,τ k =argTopk(R Q,K (τ))

[0046]

[0047] Q = W Q X2

[0048] K = W K X2

[0049]

[0050] in, This indicates the calculation of the second autocorrelation of sequence X2, where τ represents the period length, τ1,…,τ k R represents the selected k period lengths, argTopk(·) represents taking the k period lengths corresponding to the top k autocorrelation coefficients, and R Q,K R(τ) represents the autocorrelation coefficient calculated for all period lengths, and R(τ1) represents the autocorrelation coefficient corresponding to the first period length. k ) represents the autocorrelation coefficient corresponding to the k-th period length, and SoftMax(·) represents the softmax function. This represents the inverse fast Fourier transform. Represents the Fast Fourier Transform. W represents the conjugate of the Fast Fourier Transform.Q W represents the query coefficient matrix. K W represents the bond coefficient matrix. V Roll(V,τ) represents the coefficient matrix. i ) indicates that τ is applied to V. i Delay This represents the output data of the Nth encoder.

[0051] Compared with existing technologies, this invention constructs a training sample set based on historical query data and trains a multi-task prediction network based on the training sample set, thereby achieving simultaneous prediction of the query frequency of popular areas and spatial objects. By inputting multi-step historical data of spatial objects in each area to be predicted before the prediction time into the popular prediction network, the prediction result of each area to be predicted is obtained. Then, the spatial object data is preloaded according to the prediction result of each area to be predicted, thereby loading data in advance during off-peak periods, avoiding peak congestion, and improving query efficiency.

[0052] In this invention, the above-described technical solutions can be combined with each other to achieve more preferred combinations. Other features and advantages of this invention will be set forth in the following description, and some advantages may become apparent from the description or be learned by practicing the invention. The objects and other advantages of this invention can be realized and obtained from what is particularly pointed out in the description and drawings. Attached Figure Description

[0053] The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. Throughout the drawings, the same reference numerals denote the same parts.

[0054] Figure 1 This is a flowchart of a spatial object query optimization method according to an embodiment of the present invention. Detailed Implementation

[0055] Preferred embodiments of the present invention will now be described in detail with reference to the accompanying drawings, which form part of this application and are used together with the embodiments of the present invention to illustrate the principles of the present invention, but are not intended to limit the scope of the present invention.

[0056] A specific embodiment of the present invention discloses a query optimization method for spatial objects, such as... Figure 1 As shown, it includes the following steps:

[0057] S1. Obtain historical query data; calculate the query frequency and average query interval for each spatial object at each time step based on the historical query data; construct a training sample set based on the query frequency and average query interval for each spatial object at each time step.

[0058] S2. Construct a multi-task prediction network, which is used for hotspot region prediction and multi-step query frequency prediction of spatial objects; train the multi-task prediction network based on the training sample set to obtain the hotspot prediction network;

[0059] S3. Input the multi-step historical data of the spatial objects in each region to be predicted before the time to be predicted into the popular prediction network to obtain the prediction result for each region to be predicted.

[0060] S4. Preload the spatial object data based on the prediction results for each region to be predicted.

[0061] Compared with existing technologies, the spatial object query optimization method provided in this embodiment constructs a training sample set based on historical query data, and trains a multi-task prediction network based on the training sample set, thereby achieving simultaneous prediction of popular areas and spatial object query frequencies. By inputting multi-step historical data of spatial objects in each region to be predicted before the prediction time into the popular prediction network, the prediction results of each region to be predicted are obtained. Then, the spatial object data is preloaded according to the prediction results of each region to be predicted, thereby loading data in advance during off-peak periods, avoiding peak congestion, and improving query efficiency.

[0062] Spatial objects include three types: points, lines, and polygons. The data stored for spatial objects includes their coordinates and attributes. Attribute information includes, for example, the spatial object's name, unique identifier, classification code, type description (such as road name, school type, administrative region code, etc.), and measurement information (such as area, length, etc.).

[0063] Information about spatial objects is stored in a database. When a user queries, the corresponding data is retrieved from the database and returned to the user. However, spatial vector data is large in volume, and frequent queries can lead to slow query speeds and low efficiency.

[0064] With the development of artificial intelligence such as deep learning and reinforcement learning, it has become possible to perform data prediction by training deep learning networks. To improve query efficiency, this invention trains a multi-task prediction network to predict the query frequency of popular regions and spatial vector objects. During off-peak query periods, spatial object data is preloaded into the RegionServer memory or relevant cache based on the prediction results, thereby reducing latency for high-frequency queries, avoiding performance fluctuations caused by sudden high loads, and improving query efficiency.

[0065] During implementation, historical query data can be extracted from the log file. The query data includes the spatial object queried each time and the query time.

[0066] Calculate the query frequency and average query interval for each spatial object at each time step based on historical query data.

[0067] In practice, an hour is a time step.

[0068] Meteorological data, such as temperature, humidity, and wind speed.

[0069] A training sample set is constructed based on the query frequency and average query interval of each spatial object at each time step, including:

[0070] For each region, the feature data for each time step includes the meteorological data for that time step, the object type, geographic type, number of queries, and average query interval for each spatial object in that region;

[0071] For each region, T IN The feature data of each time step is used as the input data for one sample, with the T... IN T after one time step OUT Within each time step, a training sample set is constructed using labels such as whether the area is a popular area and the query frequency of each spatial object.

[0072] Spatial objects can be categorized into point, line, and polygon types. Geographic types represent real-world entity types, such as land use types and building types.

[0073] For each region, the feature data for each time step includes the meteorological data for that time step, the object type, geographic type, number of queries, and average query interval for each spatial object in that region, which constitute the feature sequence for that region.

[0074] If the time step of the sample's input data is T IN For each region, a sliding window is used to extract data from the feature sequence. Let T... IN The feature data of each time step is used as the input data for one sample, with the T... IN T after one time step OUT Within each time step, whether the region is a popular region and the query frequency of each spatial object at each time step are used as labels to obtain a sample, thereby constructing a training sample set.

[0075] Construct a multi-task prediction network for predicting popular regions and spatial objects using multi-step query frequency prediction.

[0076] Specifically, the multi-task prediction network includes a first prediction network and a second prediction network:

[0077] The first prediction network is used to predict popular areas based on the input data; the second prediction network is used to predict the multi-step query frequency of spatial objects based on the input data.

[0078] In practice, the first prediction network can adopt an existing CNN network structure.

[0079] In implementation, the second prediction network includes: an encoding network, comprising multiple encoders connected in sequence, used for multi-level feature extraction of the input data based on an autocorrelation mechanism;

[0080] The decoding network, consisting of multiple decoders connected in sequence, is used to decode the features extracted by the encoding network to obtain the prediction result.

[0081] Specifically, each encoder includes two sequence decomposition modules for sequence decomposition, and features are extracted using the following methods:

[0082]

[0083] in, This represents the periodic term obtained by the first sequence decomposition module of the l-th encoder. This represents the periodic term obtained by the second sequence decomposition module of the l-th encoder. This represents the output data of the (l-1)th encoder. This represents the output data of the l-th encoder, FFN(·) represents the feedforward layer, AC1(·) represents calculating the first autocorrelation, and SD(·) represents sequence decomposition. When l=1, X represents the input data X of the coding network en When l>1, This represents the output data of the (l-1)th encoder.

[0084] To learn complex time patterns in long-term forecasting environments, sequence decomposition is used to break down the input data into periodic and trend terms.

[0085] Specifically, the sequence decomposition method is as follows:

[0086] X T =AvgPoll(Padding(X1))

[0087] X S =X1-X T

[0088] Where X1 represents the input data for sequence decomposition, X T X represents the trend term obtained from sequence decomposition. S The periodic term represents the sequence decomposition, AvgPoll(·) represents average pooling, and Padding(·) represents the padding function.

[0089] In practice, the encoder only passes the periodic terms as output to the next encoder, thus retaining only the smooth periodic terms and gradually eliminating the trend terms, which helps the decoder refine the prediction results.

[0090] The encoding network can use N encoders, with the output data of the last encoder being... It is transmitted to the decoding network for decoding.

[0091] During implementation, the decoder decodes the trend and periodic terms of the input data separately. For the periodic terms, an autocorrelation mechanism is used to perform dependency mining based on the periodicity of the sequence and to aggregate subsequences with similar processes. For the trend terms, a stepwise accumulation method is used to extract trend information.

[0092] Specifically, each decoder includes three sequence decomposition modules for sequence decomposition, and decoding is performed in the following manner:

[0093]

[0094] in, This represents the periodic term obtained by the i-th sequence decomposition module of the l-th decoder. This represents the trend term obtained by the i-th sequence decomposition module of the l-th decoder. Let represent the output data of the Nth encoder, FFN(·) represent the feedforward layer, AC1(·) represent calculating the first autocorrelation, AC2(·,·) represent calculating the second autocorrelation, and SD(·) represent sequence decomposition. This represents the trend term output by the l-th decoder. This represents the periodic term output by the l-th decoder. This represents the weight of the trend term obtained by the i-th sequence decomposition module of the l-th decoder;

[0095] When l>1, This represents the periodic term output by the (l-1)th decoder. This represents the trend term output by the (l-1)th decoder; Represents the initial periodic term. This represents the initial trend term.

[0096] During implementation, the initial period term and initial trend term are calculated using the following formulas:

[0097]

[0098] Where Concat(·) represents concatenation, and SD(·) represents sequence decomposition. X represents the input data X of the coding network en The The sequence consisting of the first element up to the i-th element, where i represents the input data X. en The length of X, where X0 represents a sequence filled with 0s, X Mean Indicates by A sequence filled with the mean.

[0099] During implementation, X0 and X Mean The length is T OUT The upcoming prediction of T OUT The data from each time step is concatenated with the latter half of the input sequence, and X0 is used as the value to be predicted for T. OUT The initial value of the periodic term of the sequence at each time step, X Mean As for T to be predicted OUT The initial value of the trend term for the sequence at each time step.

[0100] The sum of the periodic and trend terms output by the last decoder is the model's prediction result. Where X P This represents the prediction result of the multi-task prediction network. This represents the periodic term output by the Mth decoder. This represents the trend term output by the Mth decoder, where M represents the number of decoders, and W... s and W T This represents the weight matrix.

[0101] During implementation, the first autocorrelation is calculated using the following method:

[0102]

[0103] τ1,…,τ k =argTopk(R Q,K (τ))

[0104]

[0105] Q = W Q X2

[0106] K = W K X2

[0107] V = W V X2

[0108] Where AC1(X2) represents the calculation of the first autocorrelation of sequence X2, τ represents the period length, τ1,…,τ k R represents the selected k period lengths, argTopk(·) represents taking the k period lengths corresponding to the top k autocorrelation coefficients, and R Q,K R(τ) represents the autocorrelation coefficient calculated for all period lengths, and R(τ1) represents the autocorrelation coefficient corresponding to the first period length. k ) represents the autocorrelation coefficient corresponding to the length of the kth period, and SoftMax(·) represents the softmax function. This represents the inverse fast Fourier transform. Represents the Fast Fourier Transform. W represents the conjugate of the Fast Fourier Transform. Q W represents the query coefficient matrix. K W represents the bond coefficient matrix. V Roll(V,τ) represents the coefficient matrix. i ) indicates that τ is applied to V. i Delay.

[0109] In implementation, for the input sequence X2, Q, K, and V are obtained by querying the coefficient matrix, key coefficient matrix, and value coefficient matrix. The period length is 1 ≤ τ ≤ L, where L represents the length of the input sequence. Based on the Wiener-Khinchin theorem, the autocorrelation coefficients R for all period lengths are obtained through Fast Fourier Transform. Q,K (τ), the most likely k period lengths are selected using argTopk(·) to mine periodic dependencies. In practice, the autocorrelation coefficients of the selected Top k are normalized using softmax, and the normalized results are used as weights to weight the time-delay series (element-by-element multiplication) to obtain the final autocorrelation result.

[0110] During implementation, τ i The first τ delay is the time delay of the sequence. i The data of the length is moved to the end of the sequence.

[0111] In implementation, the autocorrelation mechanism in both sequence decomposition modules of the encoder uses the first autocorrelation mechanism. The autocorrelation mechanism in the first sequence decomposition module of the decoder uses the first autocorrelation mechanism, while the autocorrelation mechanism in the second sequence decomposition module uses the second autocorrelation mechanism.

[0112] During implementation, the second autocorrelation is calculated using the following method:

[0113]

[0114] τ1,…,τ k =argTopk(R Q,K (τ))

[0115]

[0116] Q = W Q X2

[0117] K = W K X2

[0118]

[0119] in, This indicates the calculation of the second autocorrelation of sequence X2, where τ represents the period length, τ1,…,τ kR represents the selected k period lengths, argTopk(·) represents taking the k period lengths corresponding to the top k autocorrelation coefficients, and R Q,K R(τ) represents the autocorrelation coefficient calculated for all period lengths, and R(τ1) represents the autocorrelation coefficient corresponding to the first period length. k ) represents the autocorrelation coefficient corresponding to the k-th period length, and SoftMax(·) represents the softmax function. This represents the inverse fast Fourier transform. Represents the Fast Fourier Transform. W represents the conjugate of the Fast Fourier Transform. Q W represents the query coefficient matrix. K W represents the bond coefficient matrix. V Roll(V,τ) represents the coefficient matrix. i ) indicates that τ is applied to V. i Delay This represents the output data of the Nth encoder.

[0120] The second autocorrelation follows the same principle as the first autocorrelation, except that V is obtained by projecting the output data of the encoding network.

[0121] To enable the decoder to dynamically and directly retrieve the most relevant periodic support from the complete historical encoding information at each prediction step, a stable, reliable, and noise-free "knowledge base" is provided for each prediction step. Therefore, the autocorrelation mechanism of the decoder's second sequence decomposition module adopts the second autocorrelation, which calculates the cross-correlation between the current decoder input sequence (including some predicted values) and the complete historical feature sequence output by the encoder. The aggregated output represents the periodic information most relevant to the current prediction point retrieved from the global historical features. Each prediction step directly retrieves the global features from the encoder. By retrieving information from the past to guide the current step, the chain of prediction error propagation between time steps is broken. Even if the prediction at a certain step is slightly off, the prediction of the next step is still mainly based on clean historical features, which will not lead to the amplification of the deviation.

[0122] The multi-task prediction network is trained based on the constructed training sample set to obtain a well-trained multi-task prediction network.

[0123] During implementation, the loss between the predicted results and the real data is calculated, and the parameters of the network model are updated according to the loss until the loss accuracy reaches a preset threshold or the training reaches a preset number of times, thus completing the training and obtaining a trained multi-task prediction network.

[0124] The training loss is calculated using the following formula during implementation:

[0125]

[0126] Where N represents the number of samples in the current training batch, y i p represents the label indicating whether the i-th sample is a popular area. i T represents the predicted probability that the i-th sample is a popular area. OUT z represents the number of time steps for prediction. i,t This represents the actual query frequency data for each spatial object of the i-th sample at time step t. Let represent the predicted query frequency of each spatial object for the i-th sample at time step t, where α and β represent weights, and ||·|2 represents the 2-norm of the matrix.

[0127] During implementation, the spatial objects of each region to be predicted will be located at time T before the time to be predicted. IN The feature data of each step is input into the popular prediction network to obtain the T-value of each region to be predicted. Out The prediction results for whether a region is a hot area within each time step, and the prediction results for the query frequency of each spatial object at each time step.

[0128] Spatial object data is preloaded based on the prediction results for each region to be predicted.

[0129] During implementation, spatial object data is stored in a database. If the area to be predicted is a popular area, the index of the spatial vector objects for that area is preloaded into memory. Based on the query frequency prediction results for each spatial object at each time step, popular spatial objects at each time step are identified. Before that time step arrives, the coordinates and attributes of these popular objects are loaded into memory, thereby improving query efficiency.

[0130] For example, at the s-th time step, if the query frequency of a spatial object exceeds a preset threshold, then that spatial object is considered a popular spatial object at the s-th time step. Before the s-th time step arrives, the coordinates and attributes of the popular spatial objects at that time step are loaded into memory.

[0131] In one specific embodiment of the present invention, spatial object data is stored in an HBase database. The spatial object data stored in the HBase database is preloaded based on the prediction results for each region to be predicted. HBase, as a distributed NoSQL database, supports massive data storage. To further improve query efficiency, the present invention performs multi-level encoding on spatial objects, and stores the spatial object data in the HBase database based on this multi-level encoding, thus achieving distributed storage of spatial vector data.

[0132] Specifically, spatial objects are encoded at multiple levels, including:

[0133] The Geohash code is calculated based on the coordinates of each spatial object and serves as the first-level code for the spatial object; each Geohash code prefix corresponds to a region.

[0134] For each primary code, the S2 code is calculated based on the coordinates of each spatial object corresponding to that primary code, and is used as the secondary code for each spatial object corresponding to that primary code.

[0135] For each secondary code, the Morton code is calculated based on the coordinates of each spatial object corresponding to that secondary code, and used as the tertiary code for that spatial object.

[0136] In implementation, Geohash encoding is used for coarse-grained encoding to obtain the first-level code of spatial objects, realizing geospatial partitioning. Prefix matching is then used to quickly locate the target area, which serves as the prefix encoding matching range for HBase's Region spatial pre-partitioning.

[0137] In implementation, S2 encoding is used to obtain the secondary encoding of spatial objects. The S2Geometry library is used to generate the secondary encoding of spatial objects.

[0138] In implementation, Morton encoding is used to obtain a three-level code for spatial objects. Morton encoding supports nanometer-level precision positioning. Storing precise coordinate codes of spatial objects under the same GeoHash grid can achieve millimeter-level precision. When used in conjunction with Geohash, it can achieve Geohash range filtering and precise Morton matching.

[0139] During implementation, a unique three-level spatial grid composite code is determined for each spatial object through multi-level spatial grid coding, and this code is used as the row key value for storing the spatial object in HBase.

[0140] The dynamic multi-level adaptive spatial grid encoding of spatial objects mainly functions to dynamically reduce the spatial object to a one-dimensional composite spatial code based on the spatial range it represents, achieving a unique correspondence between spatial objects and spatial codes. This is used for the subsequent design of HBase spatial vector data storage models. The overall composite encoding is used for the row key design of spatial objects. The first-level encoding is used for spatial pre-partitioning design and spatial range representation based on the Geohash encoding prefix, the second-level encoding is used for the spherical spatial object representation based on S2, and the third-level encoding is used for the accurate positioning of the spatial object entity coordinates.

[0141] Specifically, based on multi-level encoding of spatial objects, spatial objects are stored in the HBase database, including:

[0142] Use the first-level code of the spatial object as the partition key to perform spatial pre-partitioning of the HBase Region;

[0143] For each spatial pre-partition, a local index is built based on the secondary encoding within that partition;

[0144] The multi-level encoding of spatial objects serves as the row key value of the spatial objects, while the coordinates and attribute information of the spatial objects serve as column families, and the spatial objects are stored in the HBase database.

[0145] Because HBase uses automatic partitioning by default, spatial vector data storage may result in spatially adjacent data that is not physically adjacent. This often leads to cross-partition access during spatial range retrieval, resulting in low performance. Therefore, the Geohash encoding in the row key is used for HBase Region space pre-partitioning.

[0146] The composite encoding consisting of multi-level encoding of spatial objects serves as the row key value of spatial objects, and the coordinates and attribute information of spatial objects are used as column families to realize the storage of spatial objects in the HBase database.

[0147] Based on the above row key and spatial pre-partitioning design, a hierarchical hybrid spatial index is constructed, which is reflected in the global index and local index in terms of architecture, and in the hybrid index in terms of spatial grid encoding and tree index based on R*-Tree.

[0148] Global Index: In implementation, a global index can be created using R*-Tree. The index object is a spatial rectangle represented by the partition key encoding (first-level Geohash encoding) of each Region partition.

[0149] Local index: A local index (based on the second-level encoding S2 encoding) is built and maintained within each partition. The index object is the space rectangle corresponding to the second-level encoding of each row key in the Region partition.

[0150] Finally, precise positioning of fine-grained spatial objects can be achieved based on the Marton coding index.

[0151] For index persistence, consider maintaining global indexes in memory and local indexes on high-speed SSD disks.

[0152] In practice, in order to balance the granularity of spatial partitioning and the dispersion of data distribution, and to make the quantity within each partition relatively balanced, thereby improving query efficiency, this invention determines the length of each level of encoding based on the degree of data dispersion.

[0153] Specifically, the length of the first-level code is calculated using the following formula:

[0154]

[0155] Where E represents the total number of space objects, R represents the average number of storage bytes occupied by each space object, and C represents the optimal performance capacity of the Region. L1 indicates rounding up, and L1 represents the character length space object of the first-level encoding.

[0156] Each letter or number in Geohash (such as the 'w' in wx4g0e) is composed of 5 bits (2 5 =32, base32), these 5 bits can have 32 different combinations.

[0157] Level 1 coding corresponds to a larger region, and the coding length is determined by the total number of spatial objects and the region capacity, thus avoiding uneven region fragmentation. Level 1 grid is a grid divided according to level 1 coding.

[0158] Specifically, for a first-level code, the length of the second-level code for each spatial object corresponding to that first-level code is calculated using the following formula:

[0159]

[0160] Where E2 represents the total number of spatial objects corresponding to the first-level code, L1 represents the character length of the first-level code, L2 represents the length of the second-level code, and K represents the number of spatial vectors expected to be stored in each second-level grid. This indicates the average number of secondary grids covered by the spatial object corresponding to the primary code.

[0161] For each first-level code, calculate the length of the second-level codes of all spatial vector objects corresponding to that first-level code according to the above formula.

[0162] A two-level grid is a grid divided according to a two-level code.

[0163] In implementation, a base length, such as L, is taken for the two-level encoding. 2-base =20, calculate the coverage of each spatial object corresponding to this first-level encoding by L 2-base The number of grids is obtained. During implementation, the intersection of the bounding rectangle of the spatial object and the grid is calculated. If the intersection is not empty, the spatial object covers the secondary grid.

[0164] During implementation, the number of spatial vectors expected to be stored in each secondary grid is determined based on the range of spatial objects, for example, 1000. The above formula ensures that the number of object instances stored in each secondary grid is close to the target value K, thus balancing the query load.

[0165] During implementation, to avoid query hotspot issues, for each primary code, if the standard deviation of the coordinates of its corresponding spatial object is greater than a preset threshold, the length of the secondary code of its corresponding spatial vector object is adjusted based on the dispersion.

[0166] During implementation, the following formula should be used for correction:

[0167]

[0168] L ′ 2 represents the length of the corrected second-level code, σ represents the standard deviation of the coordinates of the spatial object corresponding to the first-level code, and θ represents the preset threshold. This indicates rounding down to the nearest integer.

[0169] During implementation, for each first-level code, the standard deviation of the coordinates of its corresponding spatial object is calculated to obtain σ.

[0170] It should be noted that the length of the second-level code is the number of bits in the code.

[0171] A tertiary code is a unique identifier for spatial objects within the same secondary code grid. The length of a tertiary code is determined by the maximum number of objects that may appear within the grid corresponding to that secondary code.

[0172] Assume D cell2 Let this be the number of spatial objects corresponding to the grid with the largest number of spatial objects among all secondary grids. Then the length of the tertiary code for the spatial object corresponding to this secondary code is:

[0173]

[0174] Given the total number of objects within the primary grid, the average number of secondary grids covered by spatial objects within that partition, and the total number of secondary grids, the average number of spatial objects contained in each secondary grid is:

[0175] Therefore, the length of the tertiary code of the spatial object corresponding to the secondary code can also be calculated using the following formula:

[0176]

[0177] Among them, E p Indicates the number of first-level codes. L2 represents the number of secondary grids that a spatial object covers on average, L2 represents the length of the secondary code, and δ represents the unevenness factor.

[0178] In practice, δ can be set to 5-20, depending on the distribution of the objects.

[0179] The length of a three-level code is the number of bits in the code.

[0180] By determining the encoding length based on the spatial object, the storage distribution of spatial vector data is made more uniform, avoiding query hotspots and improving query efficiency.

[0181] For spatial objects stored in the HBase database, once a popular region is predicted, the local index within the HBase Region partition corresponding to that popular region is loaded into memory, thereby improving the query efficiency of that popular region.

[0182] Based on the query frequency prediction results for each spatial object at each time step, the coordinates and attributes of popular spatial objects are loaded into the RegionServer memory or cache before each time step. This allows relevant index grid data, spatial object data, etc., to be preloaded into the RegionServer memory or cache before peak periods or predicted hot areas appear, thereby improving query performance and reducing latency.

[0183] Those skilled in the art will understand that all or part of the processes of the methods described in the above embodiments can be implemented by a computer program instructing related hardware, and the program can be stored in a computer-readable storage medium. The computer-readable storage medium may be a disk, optical disk, read-only memory, or random access memory, etc.

[0184] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention.

Claims

1. A query optimization method for spatial objects, characterized in that, Includes the following steps: Retrieve historical query data; Calculate the query frequency and average query interval for each spatial object at each time step based on historical query data; A training sample set is constructed based on the query frequency and average query interval of each spatial object at each time step; A multi-task prediction network is constructed, which is used for hotspot region prediction and multi-step query frequency prediction of spatial objects; The popular prediction network is obtained by training the multi-task prediction network based on the training sample set. The historical data of spatial objects in each region to be predicted up to the time of prediction are input into the popular prediction network to obtain the prediction result for each region to be predicted; the prediction result for each region to be predicted includes the historical data of each region before the time of prediction. The prediction results of whether it is a popular area within each time step, and the prediction results of the query frequency of each spatial object in each time step; Spatial object data is preloaded based on the prediction results for each region to be predicted; A training sample set is constructed based on the query frequency and average query interval of each spatial object at each time step, including: For each region, the feature data for each time step includes the meteorological data for that time step, the object type, geographic type, query frequency, and average query interval for each spatial object in that region; For each region, with The feature data at each time step is used as the input data for one sample. After a time step Within each time step, whether the region is a popular region and the query frequency of each spatial object in the region at each time step are used as sample labels to obtain a sample, and a training sample set is constructed.

2. The spatial object query optimization method according to claim 1, characterized in that, The training loss is calculated using the following formula: Where N represents the number of samples in the current training batch. The label indicating whether the i-th sample is a popular area. This represents the predicted probability that the i-th sample is a popular area. Indicates the number of time steps in the prediction. This represents the actual query frequency data for each spatial object of the i-th sample at time step t. This represents the predicted query frequency of each spatial object for the i-th sample at time step t. and Indicates weight, This represents the 2-norm of a matrix.

3. The spatial object query optimization method according to claim 1, characterized in that, The multi-task prediction network includes a first prediction network and a second prediction network: The first prediction network is used to predict popular areas based on the input data; the second prediction network is used to predict the multi-step query frequency of spatial objects based on the input data. The second prediction network includes: The coding network, consisting of multiple encoders connected in sequence, is used to extract multi-level features from the input data based on an autocorrelation mechanism; The decoding network, consisting of multiple decoders connected in sequence, is used to decode the features extracted by the encoding network to obtain the query frequency prediction results.

4. The spatial object query optimization method according to claim 3, characterized in that, Each encoder includes two sequence decomposition modules for sequence decomposition, and feature extraction is performed using the following methods: in, Indicates the first The periodic term obtained from the first sequence decomposition module of the encoder. Indicates the first The periodic term obtained by the second sequence decomposition module of the encoder. Indicates the first The output data of the encoder Indicates the first The output data of the encoder Indicates feedforward layer, This indicates the calculation of the first autocorrelation. Indicates sequence decomposition. At 1 o'clock, Represents the input data of the coding network , At 1 o'clock, Indicates the first The output data of each encoder.

5. The spatial object query optimization method according to claim 4, characterized in that, Each decoder includes three sequence decomposition modules for sequence decomposition; Decoding is performed in the following manner: in, Indicates the first The periodic term obtained by the i-th sequence decomposition module of the decoder. Indicates the first The trend term obtained by the i-th sequence decomposition module of the decoder. This represents the output data of the Nth encoder. Indicates feedforward layer, This indicates the calculation of the first autocorrelation. This indicates the calculation of the second autocorrelation. Indicates sequence decomposition. Indicates the first Trend terms output by each decoder Indicates the first The periodic terms output by each decoder. Indicates the first The weights of the trend terms obtained by the i-th sequence decomposition module of the decoder; At 1 o'clock, Indicates the first The periodic terms output by each decoder. Indicates the first Trend terms output by each decoder; At 1 o'clock, Represents the initial periodic term. This represents the initial trend term.

6. The spatial object query optimization method according to claim 5, characterized in that, The initial period term and initial trend term are calculated using the following formulas: ， ； in, Indicates splicing, Indicates sequence decomposition. Represents the input data of the coding network The The sequence consisting of the first element to the i-th element, where i represents the input data. Length, This represents a sequence filled with 0s. Indicates by A sequence filled with the mean.

7. The spatial object query optimization method according to claim 5, characterized in that, The prediction results are obtained using the following formula: in, This represents the prediction result of the multi-task prediction network. Indicates the first The periodic terms output by each decoder. Indicates the first The trend term output by each decoder, where M represents the number of decoders. and This represents the weight matrix.

8. The spatial object query optimization method according to claim 4, characterized in that, The first autocorrelation is calculated as follows: in, Represents the computation sequence The first autocorrelation, Indicates the period length. This indicates the lengths of the selected k periods. This indicates taking the k period lengths corresponding to the top k autocorrelation coefficients. This indicates the calculation of the autocorrelation coefficient for all period lengths. This represents the autocorrelation coefficient corresponding to the length of the first period. This represents the autocorrelation coefficient corresponding to the length of the kth period. This represents the softmax function. This represents the inverse fast Fourier transform. Represents the Fast Fourier Transform. Represents the conjugate of the Fast Fourier Transform. This indicates a query coefficient matrix. Represents the bond coefficient matrix. Represents the value coefficient matrix, Indicates to performing on V Delay.

9. The spatial object query optimization method according to claim 5, characterized in that, The second autocorrelation is calculated as follows: in, Represents the computation sequence The second autocorrelation, Indicates the period length. This indicates the lengths of the selected k periods. This indicates taking the k period lengths corresponding to the top k autocorrelation coefficients. This indicates the calculation of the autocorrelation coefficient for all period lengths. This represents the autocorrelation coefficient corresponding to the length of the first period. This represents the autocorrelation coefficient corresponding to the length of the kth period. This represents the softmax function. This represents the inverse fast Fourier transform. Represents the Fast Fourier Transform. Represents the conjugate of the Fast Fourier Transform. This indicates a query coefficient matrix. Represents the bond coefficient matrix. Represents the value coefficient matrix, Indicates to performing on V Delay This represents the output data of the Nth encoder.

Images