A method of preloading spatial vector data

Abstract

The application relates to a space vector data preloading method, and belongs to the technical field of space vector data processing, which solves the problem of slow space vector data loading in the prior art. The method comprises the following steps: performing multilevel coding on a space vector object, constructing a space pre-partition and a local index of an HBase region, and storing the space vector object in an HBase database; inputting multi-step feature data of the space vector object in a to-be-predicted region before a to-be-predicted moment into a trained multi-step time sequence prediction network to obtain a query frequency prediction result of each space vector object in the to-be-predicted region at a plurality of future time steps; judging whether the to-be-predicted region is a popular region based on the query frequency prediction result; if the to-be-predicted region is a popular region, loading the local index in a partition corresponding to the to-be-predicted region into a memory; and loading the coordinates and attributes of popular space vector objects into the memory before each time step according to the query frequency prediction result. Efficient data loading and query are realized.

Description

Technical Field

[0001] This invention relates to the field of spatial vector data processing technology, and in particular to a method for preloading spatial vector data. Background Technology

[0002] In today's digital age, spatial vector data is widely used in many fields such as Geographic Information Systems (GIS), map services, and spatial data visualization. As an important type of spatial data, spatial vector data is used to represent information about geographic features with spatial location and geometric shape, such as points, lines, polygons, and combinations thereof. It covers multiple application scenarios including geography, surveying and mapping, urban planning, environmental protection, and resource management. Its efficient loading is of vital importance to the performance of related systems and user experience.

[0003] Currently, slow loading speeds are a problem when querying spatial vector data, severely hindering the development and promotion of related applications. For example, in traditional map browsing applications, when a user needs to view a detailed map of a specific area, the system typically needs to retrieve the spatial vector data of that area from the server in real time and render it. However, due to limitations such as large data volumes, network transmission latency, and server processing capabilities, this process often takes a long time, causing users to experience lag and choppy visuals while waiting for data to load, significantly impacting the user experience. In application scenarios with high real-time requirements, such as online navigation and geofencing monitoring, quickly and accurately acquiring spatial vector data is even more crucial. Delayed data loading can lead to serious problems such as delayed navigation commands and missed monitoring, thereby affecting the effectiveness and reliability of the application. Summary of the Invention

[0004] Based on the above analysis, the present invention aims to provide a method for preloading spatial vector data to solve the problem of slow loading of existing spatial vector data.

[0005] On one hand, embodiments of the present invention provide a method for preloading spatial vector data, comprising the following steps:

[0006] The spatial vector objects are encoded in multiple levels. Based on the multi-level encoding of the spatial vector objects, the spatial pre-partitioning and local index of HBaseRegion are constructed, and the spatial vector objects are stored in the HBase database.

[0007] Calculate the multi-step feature data of the spatial vector objects in the region to be predicted before the time to be predicted, input the multi-step feature data into the trained multi-step temporal prediction network, and obtain the query frequency prediction result of each spatial vector object in the region to be predicted in the future multiple time steps.

[0008] The prediction results of the query frequency of each spatial vector object in the region to be predicted in the future multiple time steps determine whether the region to be predicted is a popular region.

[0009] If the region to be predicted is a popular region, the local index within the HBase Region partition corresponding to the region to be predicted is loaded into memory; based on the query frequency prediction results of each spatial vector object at each time step, the coordinates and attributes of the popular spatial vector objects at that time step are loaded into memory before each time step.

[0010] Based on further improvements to the above technical solution, multi-level encoding is performed on spatial vector objects, including:

[0011] The Geohash code is calculated based on the coordinates of each spatial vector object and serves as the first-level code for the spatial vector object; each Geohash code prefix corresponds to a region.

[0012] For each primary code, the S2 code is calculated based on the coordinates of each spatial vector object corresponding to that primary code, and is used as the secondary code for each spatial vector object corresponding to that primary code.

[0013] For each secondary code, the Morton code is calculated based on the coordinates of each spatial vector object corresponding to that secondary code, and used as the tertiary code for that spatial vector object.

[0014] Based on further improvements to the above technical solution, the length of the first-level code is calculated using the following formula:

[0015]

[0016] Where E represents the total number of space vector objects, R represents the average number of storage bytes occupied by each space vector object, and C represents the optimal performance capacity of the Region. L1 indicates rounding up, and L1 represents the length of the first-level encoding character.

[0017] Based on further improvements to the above technical solution, the length of the second-level code for each spatial vector object corresponding to the first-level code is calculated using the following formula:

[0018]

[0019] Where E2 represents the total number of spatial vector objects corresponding to the first-level encoding, L1 represents the character length of the first-level encoding, L2 represents the length of the second-level encoding, and K represents the number of spatial vectors expected to be stored in each second-level grid. This indicates the average number of secondary grids covered by the spatial vector object corresponding to this primary encoding.

[0020] Based on further improvements to the above technical solution, the length of the second-level code is corrected using the following formula:

[0021]

[0022] L ′ 2 represents the length of the corrected second-level code, σ represents the average standard deviation of the coordinates of the spatial vector object corresponding to the first-level code, and θ represents the preset threshold. This indicates rounding down to the nearest integer.

[0023] Based on further improvements to the above technical solution, the length of the three-level code is calculated using the following formula:

[0024]

[0025] Among them, E p Indicates the number of first-level codes. L2 represents the number of secondary grids that the spatial vector object covers on average, L2 represents the length of the secondary code, and δ represents the non-uniformity factor.

[0026] Further improvements to the above technical solution involve constructing spatial pre-partitioning and local indexes for HBaseRegions based on multi-level encoding of spatial vector objects, and storing the spatial vector objects in the HBase database, including:

[0027] Spatial pre-partitioning of HBase Regions is performed using the first-level encoding of spatial vector objects as partition keys;

[0028] For each spatial pre-partition, a local index is built based on the secondary encoding within that partition;

[0029] The multi-level encoding of the spatial vector object serves as the row key value of the spatial vector object, and the coordinates and attribute information of the spatial vector object serve as the column family, storing the spatial vector object in the HBase database.

[0030] Based on further improvements to the above technical solution, the multi-step time series prediction network includes:

[0031] The coding network, consisting of multiple encoders connected in sequence, is used to extract multi-level features from the input data based on an autocorrelation mechanism;

[0032] The decoding network, consisting of multiple decoders connected in sequence, is used to decode the features extracted by the encoding network to obtain the prediction result.

[0033] Based on the further improvement of the above technical solution, each encoder includes two sequence decomposition modules for sequence decomposition, and feature extraction is performed in the following manner:

[0034]

[0035] in, This represents the periodic term obtained by the first sequence decomposition module of the l-th encoder. This represents the periodic term obtained by the second sequence decomposition module of the l-th encoder. This represents the output data of the (l-1)th encoder. This represents the output data of the l-th encoder, FFN(·) represents the feedforward layer, AC1(·) represents calculating the first autocorrelation, and SD(·) represents sequence decomposition. When l=1, X represents the input data X of the coding network en When l>1, This represents the output data of the (l-1)th encoder.

[0036] Based on further improvements to the above technical solution, each decoder includes three sequence decomposition modules for sequence decomposition; decoding is performed in the following manner:

[0037]

[0038] in, This represents the periodic term obtained by the i-th sequence decomposition module of the l-th decoder. This represents the trend term obtained by the i-th sequence decomposition module of the l-th decoder. Let represent the output data of the Nth encoder, FFN(·) represent the feedforward layer, AC1(·) represent calculating the first autocorrelation, AC2(·,·) represent calculating the second autocorrelation, and SD(·) represent sequence decomposition. This represents the trend term output by the l-th decoder. This represents the periodic term output by the l-th decoder. This represents the weight of the trend term obtained by the i-th sequence decomposition module of the l-th decoder;

[0039] When l>1, This represents the periodic term output by the (l-1)th decoder. This represents the trend term output by the (l-1)th decoder; when l=1, Represents the initial periodic term. This represents the initial trend term.

[0040] Compared with existing technologies, this invention stores spatial vector objects in an HBase database through multi-level encoding. It predicts the query frequency of each spatial vector object in the region to be predicted in the future at multiple time steps through a multi-step time series prediction network, thereby determining whether the region to be predicted will be a popular region in the future. If it is a popular region, the local index corresponding to the region to be predicted is loaded into memory in advance, and the relevant data of the popular spatial vector objects in the region to be predicted are also loaded into memory, thereby avoiding query congestion during peak periods and improving loading and query efficiency.

[0041] In this invention, the above-described technical solutions can be combined with each other to achieve more preferred combinations. Other features and advantages of this invention will be set forth in the following description, and some advantages may become apparent from the description or be learned by practicing the invention. The objects and other advantages of this invention can be realized and obtained from what is particularly pointed out in the description and drawings. Attached Figure Description

[0042] The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. Throughout the drawings, the same reference numerals denote the same parts.

[0043] Figure 1 This is a flowchart of a method for preloading spatial vector data according to an embodiment of the present invention. Detailed Implementation

[0044] Preferred embodiments of the present invention will now be described in detail with reference to the accompanying drawings, which form part of this application and are used together with the embodiments of the present invention to illustrate the principles of the present invention, but are not intended to limit the scope of the present invention.

[0045] A specific embodiment of the present invention discloses a method for preloading spatial vector data, such as... Figure 1 As shown, it includes the following steps:

[0046] S1. Perform multi-level encoding on spatial vector objects, construct spatial pre-partitions and local indexes of HBaseRegion based on the multi-level encoding of spatial vector objects, and store spatial vector objects in the HBase database;

[0047] S2. Calculate the multi-step feature data of the spatial vector objects in the region to be predicted before the time to be predicted, and input the multi-step feature data into the trained multi-step temporal prediction network to obtain the query frequency prediction result of each spatial vector object in the region to be predicted in the future multiple time steps.

[0048] S3. Determine whether the region to be predicted is a popular region based on the query frequency prediction results of each spatial vector object in the region to be predicted at multiple time steps in the future.

[0049] If the region to be predicted is a popular region, then load the local index within the HBase Region partition corresponding to the region to be predicted into memory; based on the query frequency prediction results of each spatial vector object at each time step, load the coordinates and attributes of the popular spatial vector objects of that time step into memory before each time step.

[0050] Compared with existing technologies, the spatial vector data preloading method provided in this embodiment constructs spatial pre-partitioning and local indexes of HBase Regions through multi-level coding, stores spatial vector objects in the HBase database, and predicts the query frequency of each spatial vector object in the region to be predicted in the future multiple time steps through a multi-step time series prediction network, thereby determining whether the region to be predicted will be a popular region in the future. If it is a popular region, the local index corresponding to the region to be predicted is preloaded into memory, and the relevant data of the popular spatial vector objects in the region to be predicted are also preloaded into memory, thereby avoiding query congestion during peak periods and improving loading and query efficiency.

[0051] Spatial vector objects include three types of objects: points, lines, and polygons. Stored spatial vector objects include their coordinates and attributes. Attribute information includes, for example, the spatial object's name, unique identifier, classification code, type description (such as road name, school type, administrative region code, etc.), and measurement information (such as area, length, etc.).

[0052] HBase, as a distributed NoSQL database, supports big data storage and can realize distributed storage of spatial vector objects.

[0053] In implementation, this invention performs multi-level encoding on spatial vector objects, constructs spatial pre-partitions and local indexes for HBase Regions based on the multi-level encoding of spatial vector objects, and stores spatial vector objects in the HBase database.

[0054] Specifically, spatial vector objects are encoded at multiple levels, including:

[0055] The Geohash code is calculated based on the coordinates of each spatial vector object and serves as the first-level code for the spatial vector object; each Geohash code prefix corresponds to a region.

[0056] For each primary code, the S2 code is calculated based on the coordinates of each spatial vector object corresponding to that primary code, and is used as the secondary code for each spatial vector object corresponding to that primary code.

[0057] For each secondary code, the Morton code is calculated based on the coordinates of each spatial vector object corresponding to that secondary code, and used as the tertiary code for that spatial vector object.

[0058] In implementation, Geohash encoding is used for coarse-grained encoding to obtain the first-level encoding of spatial vector objects, realizing geospatial partitioning. Target areas are quickly located through prefix matching. This encoding serves as the prefix encoding matching range for HBase's Region spatial pre-partitioning.

[0059] During implementation, S2 encoding is used to obtain the secondary encoding of spatial vector objects. The S2 Geometry library is used to generate the secondary encoding of spatial vector objects.

[0060] In implementation, Morton encoding is used to obtain a three-level encoding of spatial vector objects. Morton encoding supports nanometer-level precision positioning. Storing precise coordinate encodings of spatial objects under the same GeoHash grid can achieve millimeter-level precision. When used in conjunction with Geohash, it can achieve Geohash range filtering and precise Morton matching.

[0061] During implementation, a three-level spatial grid composite code is used to determine that each spatial vector object uniquely corresponds to a spatial grid, and this code is used as the row key value for storing the spatial object in HBase.

[0062] The dynamic multi-level adaptive spatial grid encoding of spatial objects mainly functions to dynamically reduce the spatial object to a one-dimensional composite spatial code based on the spatial range it represents, achieving a unique correspondence between spatial objects and spatial codes. This is used for the subsequent design of HBase spatial vector data storage models. The overall composite encoding is used for the row key design of spatial objects. The first-level encoding is used for spatial pre-partitioning design and spatial range representation based on the Geohash encoding prefix, the second-level encoding is used for the spherical spatial object representation based on S2, and the third-level encoding is used for the accurate positioning of the spatial object entity coordinates.

[0063] Specifically, spatial pre-partitioning and local indexing of HBase Regions are constructed based on multi-level encoding of spatial vector objects, and the spatial vector objects are stored in the HBase database, including:

[0064] Spatial pre-partitioning of HBase Regions is performed using the first-level encoding of spatial vector objects as partition keys;

[0065] For each spatial pre-partition, a local index is built based on the secondary encoding within that partition;

[0066] The multi-level encoding of the spatial vector object serves as the row key value of the spatial vector object, and the coordinates and attribute information of the spatial vector object serve as the column family, storing the spatial vector object in the HBase database.

[0067] Because HBase uses automatic partitioning by default, spatial vector data storage may result in spatially adjacent data that is not physically adjacent. This often leads to cross-partition access during spatial range retrieval, resulting in low performance. Therefore, the Geohash encoding in the row key is used for HBase Region space pre-partitioning.

[0068] The composite encoding consisting of multi-level encoding of spatial vector objects serves as the row key value of spatial vector objects, and the coordinates and attribute information of spatial vector objects are used as column families to realize the storage of spatial vector objects in the HBase database.

[0069] Based on the above row key and spatial pre-partitioning design, a hierarchical hybrid spatial index is constructed, which is reflected in the global index and local index in terms of architecture, and in the hybrid index in terms of spatial grid encoding and tree index based on R*-Tree.

[0070] Global Index: In implementation, a global index can be created using R*-Tree. The index object is a spatial rectangle represented by the partition key encoding (first-level Geohash encoding) of each Region partition.

[0071] Local index: A local index (based on the second-level encoding S2 encoding) is built and maintained within each partition. The index object is the space rectangle corresponding to the second-level encoding of each row key in the Region partition.

[0072] Finally, precise positioning of fine-grained spatial objects can be achieved based on the Marton coding index.

[0073] For index persistence, consider maintaining global indexes in memory and local indexes on high-speed SSD disks.

[0074] In practice, in order to balance the granularity of spatial partitioning and the dispersion of data distribution, and to make the quantity within each partition relatively balanced, thereby improving query efficiency, this invention determines the length of each level of encoding based on the degree of data dispersion.

[0075] Specifically, the length of the first-level code is calculated using the following formula:

[0076]

[0077] Where E represents the total number of space vector objects, R represents the average number of storage bytes occupied by each space vector object, and C represents the optimal performance capacity of the Region. L1 indicates rounding up, and L1 represents the length of the first-level encoding character.

[0078] Each letter or number in Geohash (such as the 'w' in wx4g0e) is composed of 5 bits (2 5=32, base32), these 5 bits can have 32 different combinations.

[0079] Level 1 coding corresponds to a larger region, and the coding length is determined by the total number of spatial objects and the region capacity, thus avoiding uneven region fragmentation. Level 1 grid is a grid divided according to level 1 coding.

[0080] Specifically, for a first-level code, the length of the second-level code for each spatial vector object corresponding to that first-level code is calculated using the following formula:

[0081]

[0082] Where E2 represents the total number of spatial vector objects corresponding to the first-level encoding, L1 represents the character length of the first-level encoding, L2 represents the length of the second-level encoding, and K represents the number of spatial vectors expected to be stored in each second-level grid. This indicates the average number of secondary grids covered by the spatial vector object corresponding to this primary encoding.

[0083] For each first-level code, calculate the length of the second-level codes of all spatial vector objects corresponding to that first-level code according to the above formula.

[0084] A two-level grid is a grid divided according to a two-level code.

[0085] In implementation, a base length, such as L, is taken for the two-level encoding. 2-base =20, calculate the coverage of each spatial vector object corresponding to this first-level encoding by L 2-base The number of grids is obtained. During implementation, the intersection of the bounding rectangle of the spatial vector object and the grid is calculated. If the intersection is not empty, the spatial vector object covers the secondary grid.

[0086] During implementation, the number of space vectors expected to be stored in each secondary grid is determined based on the range of the number of space vector objects, for example, 1000. The above formula ensures that the number of object instances stored in each secondary grid is close to the target value K, thus balancing the query load.

[0087] During implementation, to avoid query hotspot issues, for each primary code, if the standard deviation of the coordinates of its corresponding spatial vector object is greater than a preset threshold, the length of the secondary code of the spatial vector object is corrected based on the dispersion.

[0088] During implementation, the following formula should be used for correction:

[0089]

[0090] L ′2 represents the length of the corrected second-level code, σ represents the standard deviation of the coordinates of the spatial vector object corresponding to the first-level code, and θ represents the preset threshold. This indicates rounding down to the nearest integer.

[0091] During implementation, for each level-one code, the standard deviation of the coordinates of its corresponding spatial vector object is calculated to obtain σ.

[0092] It should be noted that the length of the second-level code is the number of bits in the code.

[0093] A tertiary code is a unique identifier for a spatial vector object within the same secondary code grid. The length of a tertiary code is determined by the maximum number of objects that may appear within the grid corresponding to that secondary code.

[0094] Assume D cell2 Let this be the number of spatial vector objects corresponding to the grid that contains the most spatial vector objects among all secondary grids. Then, the length of the tertiary code for the spatial vector object corresponding to this secondary code is:

[0095]

[0096] Given the total number of objects within the primary grid, the average number of secondary grids covered by spatial vector objects within that partition, and the total number of secondary grids, the average number of spatial vector objects contained in each secondary grid is:

[0097]

[0098] Therefore, the length of the tertiary code for the spatial vector object corresponding to the secondary code can also be calculated using the following formula:

[0099]

[0100] Among them, E p Indicates the number of first-level codes. L2 represents the number of secondary grids that the spatial vector object covers on average, L2 represents the length of the secondary code, and δ represents the non-uniformity factor.

[0101] In practice, δ can be set to 5-20, depending on the distribution of the objects.

[0102] The length of a three-level code is the number of bits in the code.

[0103] By determining the encoding length based on the spatial vector object, the storage distribution of spatial vector data is made more uniform, avoiding query hotspots and improving query efficiency.

[0104] With the development of artificial intelligence such as deep learning and reinforcement learning, more and more fields are using deep learning network models for data prediction. To further improve query efficiency, this invention calculates the query frequency of spatial vector objects using a trained multi-step temporal prediction network. This allows for the preloading of relevant index data for popular regions and spatial objects into the RegionServer memory or related cache during off-peak query periods, thereby reducing latency for high-frequency queries, avoiding performance fluctuations caused by sudden high loads, and improving query efficiency.

[0105] In practice, the trained multi-step temporal prediction network is obtained through the following steps:

[0106] A training sample set was constructed based on historical query data;

[0107] The multi-step time series prediction network is trained based on the training sample set to obtain a trained multi-step time series prediction network.

[0108] During implementation, a training sample set is constructed based on historical query data, specifically including:

[0109] Calculate the query frequency and average query interval for each spatial vector object at each time step based on historical query data;

[0110] For each region, the feature data for each time step includes the meteorological data for that time step, the object type, geographic type, number of queries, and average query interval for each spatial vector object in that region;

[0111] For each region, T IN The feature data of each time step is used as the input data for one sample, with the T... IN T after one time step OUT The query frequency of each spatial vector object within each time step is used as the label to construct a training sample set.

[0112] During implementation, historical query data can be extracted from the log file. The query data includes the spatial vector object queried each time and the query time.

[0113] Calculate the query frequency and average query interval for each spatial vector object at each time step based on historical query data.

[0114] In practice, an hour is a time step.

[0115] Meteorological data, such as temperature, humidity, and wind speed.

[0116] A region is defined by a single level-one code. A spatial vector object for a region is defined by all spatial vectors corresponding to the level-one code of that region.

[0117] Spatial vector objects can be categorized into point, line, and polygon types. Geographic types represent real-world entity types, such as land use types and building types.

[0118] For each region, the feature data for each time step includes the meteorological data for that time step, the object type, geographic type, number of queries, and average query interval for each spatial vector object in that region, which constitute the feature sequence for that region.

[0119] If the time step of the sample's input data is T IN For each region, a sliding window is used to extract data from the feature sequence. Let T... IN The feature data of each time step is used as the input data for one sample, with the T... IN T after one time step OUT Within each time step, the query frequency of each spatial vector object at each time step is used as the label to obtain a sample, thereby constructing a training sample set.

[0120] Construct a multistep query frequency prediction and multistep time series prediction network for spatial vector objects.

[0121] Specifically, multi-step time series prediction networks include:

[0122] The coding network, consisting of multiple encoders connected in sequence, is used to extract multi-level features from the input data based on an autocorrelation mechanism;

[0123] The decoding network, consisting of multiple decoders connected in sequence, is used to decode the features extracted by the encoding network to obtain the prediction result.

[0124] Specifically, each encoder includes two sequence decomposition modules for sequence decomposition, and features are extracted using the following methods:

[0125]

[0126] in, This represents the periodic term obtained by the first sequence decomposition module of the l-th encoder. This represents the periodic term obtained by the second sequence decomposition module of the l-th encoder. This represents the output data of the (l-1)th encoder. This represents the output data of the l-th encoder, FFN(·) represents the feedforward layer, AC1(·) represents calculating the first autocorrelation, and SD(·) represents sequence decomposition. When l=1, X represents the input data X of the coding network en When l>1, This represents the output data of the (l-1)th encoder.

[0127] To learn complex time patterns in long-term forecasting environments, sequence decomposition is used to break down the input data into periodic and trend terms.

[0128] Specifically, the sequence decomposition method is as follows:

[0129] X T =AvgPoll(Padding(X1))

[0130] X S =X1-X T

[0131] Where X1 represents the input data for sequence decomposition, X T X represents the trend term obtained from sequence decomposition. S The periodic term represents the sequence decomposition, AvgPoll(·) represents average pooling, and Padding(·) represents the padding function.

[0132] In practice, the encoder only passes the periodic terms as output to the next encoder, thus retaining only the smooth periodic terms and gradually eliminating the trend terms, which helps the decoder refine the prediction results.

[0133] The encoding network can use N encoders, with the output data of the last encoder being... It is transmitted to the decoding network for decoding.

[0134] During implementation, the decoder decodes the trend and periodic terms of the input data separately. For the periodic terms, an autocorrelation mechanism is used to perform dependency mining based on the periodicity of the sequence and to aggregate subsequences with similar processes. For the trend terms, a stepwise accumulation method is used to extract trend information.

[0135] Specifically, each decoder includes three sequence decomposition modules for sequence decomposition, and decoding is performed in the following manner:

[0136]

[0137] in, This represents the periodic term obtained by the i-th sequence decomposition module of the l-th decoder. This represents the trend term obtained by the i-th sequence decomposition module of the l-th decoder. Let represent the output data of the Nth encoder, FFN(·) represent the feedforward layer, AC1(·) represent calculating the first autocorrelation, AC2(·,·) represent calculating the second autocorrelation, and SD(·) represent sequence decomposition. This represents the trend term output by the l-th decoder. This represents the periodic term output by the l-th decoder. This represents the weight of the trend term obtained by the i-th sequence decomposition module of the l-th decoder;

[0138] When l>1, This represents the periodic term output by the (l-1)th decoder. This represents the trend term output by the (l-1)th decoder; Represents the initial periodic term. This represents the initial trend term.

[0139] During implementation, the initial period term and initial trend term are calculated using the following formulas:

[0140]

[0141] Where Concat(·) represents concatenation, and SD(·) represents sequence decomposition. X represents the input data X of the coding network en The The sequence consisting of the first element up to the i-th element, where i represents the input data X. en The length of X, where X0 represents a sequence filled with 0s, X Mean Indicates by A sequence filled with the mean.

[0142] During implementation, X0 and X Mean The length is T OUT The upcoming prediction of T OUT The data from each time step is concatenated with the latter half of the input sequence, and X0 is used as the value to be predicted for T. OUT The initial value of the periodic term of the sequence at each time step, X Mean As for T to be predicted OUT The initial value of the trend term for the sequence at each time step.

[0143] The sum of the periodic and trend terms output by the last decoder is the model's prediction result. Where X P This represents the prediction results of a multi-step temporal prediction network. This represents the periodic term output by the Mth decoder. This represents the trend term output by the Mth decoder, where M represents the number of decoders, and W... s and W T This represents the weight matrix.

[0144] During implementation, the first autocorrelation is calculated using the following method:

[0145]

[0146] τ1,…,τ k =argTopk(R Q,K (τ))

[0147]

[0148] Q = W Q X2

[0149] K = W K X2

[0150] V = W V X2

[0151] Where AC1(X2) represents the calculation of the first autocorrelation of sequence X2, τ represents the period length, τ1,…,τ k R represents the selected k period lengths, argTopk(·) represents taking the k period lengths corresponding to the top k autocorrelation coefficients, and R Q,K R(τ) represents the autocorrelation coefficient calculated for all period lengths, and R(τ1) represents the autocorrelation coefficient corresponding to the first period length. k ) represents the autocorrelation coefficient corresponding to the k-th period length, and SoftMax(·) represents the softmax function. This represents the inverse fast Fourier transform. Represents the Fast Fourier Transform. W represents the conjugate of the Fast Fourier Transform. Q W represents the query coefficient matrix. K W represents the bond coefficient matrix. V Roll(V,τ) represents the coefficient matrix. i ) indicates that τ is applied to V. i Delay.

[0152] In implementation, for the input sequence X2, Q, K, and V are obtained by querying the coefficient matrix, key coefficient matrix, and value coefficient matrix. The period length is 1 ≤ τ ≤ L, where L represents the length of the input sequence. Based on the Wiener-Khinchin theorem, the autocorrelation coefficients R for all period lengths are obtained through Fast Fourier Transform. Q,K (τ), the most likely k period lengths are selected using argTopk(·) to mine periodic dependencies. In practice, the autocorrelation coefficients of the selected Top k are normalized using softmax, and the normalized results are used as weights to weight the time-delay series (element-by-element multiplication) to obtain the final autocorrelation result.

[0153] During implementation, τ i The first τ delay is the time delay of the sequence. i The data of the length is moved to the end of the sequence.

[0154] In implementation, the autocorrelation mechanism in both sequence decomposition modules of the encoder uses the first autocorrelation mechanism. The autocorrelation mechanism in the first sequence decomposition module of the decoder uses the first autocorrelation mechanism, while the autocorrelation mechanism in the second sequence decomposition module uses the second autocorrelation mechanism.

[0155] During implementation, the second autocorrelation is calculated using the following method:

[0156]

[0157] τ1,…,τ k =argTopk(R Q,K (τ))

[0158]

[0159] Q = W Q X2

[0160] K = W K X2

[0161]

[0162] in, This indicates the calculation of the second autocorrelation of sequence X2, where τ represents the period length, τ1,…,τ k R represents the selected k period lengths, argTopk(·) represents taking the k period lengths corresponding to the top k autocorrelation coefficients, and R Q,K R(τ) represents the autocorrelation coefficient calculated for all period lengths, and R(τ1) represents the autocorrelation coefficient corresponding to the first period length. k ) represents the autocorrelation coefficient corresponding to the k-th period length, and SoftMax(·) represents the softmax function. This represents the inverse fast Fourier transform. Represents the Fast Fourier Transform. W represents the conjugate of the Fast Fourier Transform. Q W represents the query coefficient matrix. K W represents the bond coefficient matrix. V Roll(V,τ) represents the coefficient matrix. i ) indicates that τ is applied to V. i Delay This represents the output data of the Nth encoder.

[0163] The second autocorrelation follows the same principle as the first autocorrelation, except that V is obtained by projecting the output data of the encoding network.

[0164] To enable the decoder to dynamically and directly retrieve the most relevant periodic support from the complete historical encoding information at each prediction step, a stable, reliable, and noise-free "knowledge base" is provided for each prediction step. Therefore, the autocorrelation mechanism of the decoder's second sequence decomposition module adopts the second autocorrelation, which calculates the cross-correlation between the current decoder input sequence (including partial predicted values) and the complete historical feature sequence output by the encoder. The aggregated output represents the periodic information most relevant to the current prediction point retrieved from the global historical features. Each prediction step directly retrieves the global features from the encoder. By retrieving information from the past to guide the current step, the chain of prediction error propagation between time steps is broken. Even if the prediction at a certain step is slightly off, the prediction of the next step is still mainly based on clean historical features, which will not lead to the amplification of the deviation.

[0165] The multi-step time series prediction network is trained based on the constructed training sample set to obtain a well-trained multi-step time series prediction network.

[0166] During implementation, the loss between the predicted results and the real data is calculated, and the parameters of the network model are updated according to the loss until the loss accuracy reaches a preset threshold or the training reaches a preset number of times, thus completing the training and obtaining a trained multi-step time series prediction network.

[0167] During implementation, the spatial vector object of the region to be predicted is set to T before the time of prediction. IN The feature data of each step is input into the trained multi-step temporal prediction network to obtain the region to be predicted in T. Out Within each time step, the predicted query frequency for each spatial vector object at each time step.

[0168] Based on each spatial vector object in T Out The query frequency prediction results at each time step determine whether the area to be predicted is a popular area.

[0169] During implementation, the query popularity of the region is calculated at each time step. Where q j,t represents the query frequency of the i-th spatial vector object at time step t, and represents the number of spatial vector objects in the region to be predicted.

[0170] If T Out If the query popularity of a region exceeds a preset popularity threshold for S consecutive time steps within a given time step, the region is determined to be a popular region. In practice, the preheating popularity threshold can be set based on historical query data and experience. If the region to be predicted is a popular region, the local index within the HBase Region partition corresponding to the region to be predicted is loaded into memory, thereby improving the query efficiency of popular regions.

[0171] Based on the query frequency prediction results of each spatial vector object at each time step, the coordinates and attributes of popular spatial vector objects are loaded into memory before each time step.

[0172] For example, at time step s, if the query frequency of a spatial vector object exceeds a preset threshold, then that spatial vector object is considered a popular spatial vector object at time step s. Before time step s arrives, the coordinates and attributes of the popular spatial vector objects at that time step are loaded into memory. This allows relevant index grid data, spatial object data, etc., to be preloaded into the RegionServer memory or cache before off-peak periods or the predicted emergence of popular regions, thereby improving query performance and reducing latency.

[0173] Those skilled in the art will understand that all or part of the processes of the methods described in the above embodiments can be implemented by a computer program instructing related hardware, and the program can be stored in a computer-readable storage medium. The computer-readable storage medium may be a disk, optical disk, read-only memory, or random access memory, etc.

[0174] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for preloading spatial vector data, characterized in that, Includes the following steps: The spatial vector objects are encoded in multiple levels. Based on the multi-level encoding of the spatial vector objects, spatial pre-partitioning and local indexes of HBase Regions are constructed, and the spatial vector objects are stored in the HBase database. Calculate the multi-step feature data of spatial vector objects in the region to be predicted before the time to be predicted, input the multi-step feature data into the trained multi-step time series prediction network, and obtain the prediction result of the query frequency of each spatial vector object in the region to be predicted in the future multiple time steps; the feature data of each time step includes the meteorological data of that time step, the object type, geographical type, query frequency and average query interval of each spatial vector object in the region to be predicted. The prediction results of the query frequency of each spatial vector object in the region to be predicted in the future multiple time steps determine whether the region to be predicted is a popular region. If the region to be predicted is a popular region, the local index within the HBase Region partition corresponding to the region to be predicted is loaded into memory; based on the query frequency prediction results of each spatial vector object at each time step, the coordinates and attributes of the popular spatial vector objects at that time step are loaded into memory before each time step.

2. The method for preloading spatial vector data according to claim 1, characterized in that, Multi-level encoding of spatial vector objects includes: The Geohash code is calculated based on the coordinates of each spatial vector object and serves as the first-level code for the spatial vector object; each Geohash code prefix corresponds to a region. For each primary code, the S2 code is calculated based on the coordinates of each spatial vector object corresponding to that primary code, and is used as the secondary code for each spatial vector object corresponding to that primary code. For each secondary code, the Morton code is calculated based on the coordinates of each spatial vector object corresponding to that secondary code, and used as the tertiary code for that spatial vector object.

3. The method for preloading spatial vector data according to claim 2, characterized in that, The length of the first-level code is calculated using the following formula: in, This represents the total number of spatial vector objects. This represents the average number of bytes of storage occupied by each space vector object. This indicates the optimal performance capacity of the Region. Indicates rounding up. Indicates the length of characters in the first-level encoding.

4. The method for preloading spatial vector data according to claim 2, characterized in that, The length of the second-level code for each spatial vector object corresponding to the first-level code is calculated using the following formula: in, This indicates the total number of space vector objects corresponding to this level of encoding. Indicates the length of characters in the first-level encoding. Indicates the length of the second-level code. This represents the number of space vectors that each secondary grid is expected to store. This indicates the average number of secondary grids covered by the spatial vector object corresponding to this primary encoding.

5. The method for preloading spatial vector data according to claim 4, characterized in that, The length of the second-level code is corrected using the following formula: Indicates the length of the corrected second-level encoding. This represents the average of the standard deviations of the coordinates of the spatial vector object corresponding to this first-level encoding. This indicates the preset threshold. This indicates rounding down to the nearest integer.

6. The method for preloading spatial vector data according to claim 2, characterized in that, The length of the three-level code is calculated using the following formula: in, Indicates the number of first-level codes. This indicates the average number of secondary grid cells covered by a spatial vector object. Indicates the length of the second-level code. This represents the non-uniformity factor.

7. The method for preloading spatial vector data according to claim 2, characterized in that, Based on multi-level encoding of spatial vector objects, spatial pre-partitioning and local indexing of HBase Regions are constructed, and spatial vector objects are stored in the HBase database, including: Spatial pre-partitioning of HBase Regions is performed using the first-level encoding of spatial vector objects as partition keys; For each spatial pre-partition, a local index is built based on the secondary encoding within that partition; The multi-level encoding of the spatial vector object serves as the row key value of the spatial vector object, and the coordinates and attribute information of the spatial vector object serve as the column family, storing the spatial vector object in the HBase database.

8. The method for preloading spatial vector data according to claim 1, characterized in that, The multi-step temporal prediction network includes: The coding network, consisting of multiple encoders connected in sequence, is used to extract multi-level features from the input data based on an autocorrelation mechanism; The decoding network, consisting of multiple decoders connected in sequence, is used to decode the features extracted by the encoding network to obtain the prediction result.

9. The method for preloading spatial vector data according to claim 8, characterized in that, Each encoder includes two sequence decomposition modules for sequence decomposition, and feature extraction is performed using the following methods: in, Indicates the first The periodic term obtained by the first sequence decomposition module of the encoder. Indicates the first The periodic term obtained by the second sequence decomposition module of the encoder. Indicates the first The output data of the encoder Indicates the first The output data of the encoder Indicates feedforward layer, This indicates the calculation of the first autocorrelation. Indicates sequence decomposition. At 1 o'clock, Represents the input data of the coding network , At 1 o'clock, Indicates the first The output data of each encoder.

10. The method for preloading spatial vector data according to claim 8, characterized in that, Each decoder includes three sequence decomposition modules for sequence decomposition; Decoding is performed in the following manner: in, Indicates the first The periodic term obtained by the i-th sequence decomposition module of the decoder. Indicates the first The trend term obtained by the i-th sequence decomposition module of the decoder. This represents the output data of the Nth encoder. Indicates feedforward layer, This indicates the calculation of the first autocorrelation. This indicates the calculation of the second autocorrelation. Indicates sequence decomposition. Indicates the first Trend terms output by each decoder Indicates the first The periodic terms output by each decoder. Indicates the first The weights of the trend terms obtained by the i-th sequence decomposition module of the decoder; At 1 o'clock, Indicates the first The periodic terms output by each decoder. Indicates the first Trend terms output by each decoder; At 1 o'clock, Represents the initial periodic term. This represents the initial trend term.

Images