Query method and system based on BMTree dimension reduction index structure

Abstract

The invention provides a query method and system based on a BMTree dimensionality reduction index structure. The query method comprises the steps that the BMTree dimensionality reduction index structure is constructed based on double benchmarks; comprising the following steps: respectively constructing a global cost model and a local cost model based on a space filling curve SFC; constructing a hybrid cost model based on the global cost model and the local cost model; a space filling curve SFC adaptive to the target data set distribution and query mode is screened through a mixed cost model to serve as a forward reference, and a reverse reference is generated based on a reverse principle; respectively constructing a forward reference Monte Carlo search tree architecture and a reverse reference Monte Carlo search tree architecture based on the forward reference and the reverse reference; based on a forward reference Monte Carlo search tree architecture and a reverse reference Monte Carlo search tree architecture, a BMTree index structure is constructed by adopting a hierarchical iteration and reward mechanism preferential strategy. And obtaining a query request input by a user, and querying to obtain a corresponding query result according to the query parameters and the BMTree dimensionality reduction index structure.

Description

Technical Field

[0001] This invention relates to the field of multidimensional data indexing, specifically to a query method and system based on a BMTree dimensionality reduction index structure, and more specifically to a method and system for constructing a BMTree dimensionality reduction index structure based on dual benchmarks. Background Technology

[0002] Multi-column data sorting is a key optimization technique for improving database access efficiency, while index structures are a fundamental component of various data management applications. Compared to full database scans, indexes can narrow the search scope to at least a small amount of data, thus significantly improving data access speed. However, in the context of big data, the volume of data is growing explosively, and traditional index structures struggle to handle massive amounts of high-dimensional and complex data. Although they can maintain a certain level of query efficiency, they fail to fully utilize the data distribution characteristics and other relevant information in the database.

[0003] Space-filling curve (SFC) is a method for mapping multidimensional space to one-dimensional space. Besides its significant value in mathematics, SFC is widely used in computer science, geography, astronomy, and other disciplines. This technique converts multidimensional data into a one-dimensional sequence through space filling, facilitating computer storage and computation. Another important characteristic of SFC is its locality; the similarity between data is preserved on the curve—similar data are usually arranged adjacently in the SFC sequence, which is beneficial for querying and computational operations. For example, in the scenario of crack detection in boiler steel pipes based on millimeter-wave radar, SFC technology can be used to index millimeter-wave radar point cloud data. The core is to transform the proximity of three-dimensional space into the locality of a one-dimensional sequence through dimensionality reduction mapping, thereby significantly improving the efficiency of nearest neighbor search. In specific implementation, the three-dimensional coordinates of the radar point cloud are first normalized to a discrete grid space. Then, using SFC algorithms such as Hilbert curves or Morton curves (Z-order), the unique three-dimensional coordinates of each grid cell are encoded into a continuous one-dimensional key value. This key value is stored in the database as the index key of the point cloud data, ensuring that spatially adjacent points maintain a high probability of continuity in the index sequence. When range queries or K-nearest neighbor searches are required, the system only needs to perform local interval scans on the one-dimensional index key values, avoiding global traversal of the three-dimensional space, thus reducing the computational complexity from O(n) to close to O(log n).

[0004] In the field of multidimensional data indexing, various space-filling curves (SFCs) have been developed for dimensionality reduction. For example, the C-curve achieves dimensionality reduction by organizing data points according to dimensions, while the Z-curve and Hilbert curve are also widely used in index design. Although these SFCs perform well in many applications, they do not fully consider data distribution characteristics and query load. To address this issue, the QUILTS method has been proposed in the prior art, which considers both data distribution and query load when designing the SFC mapping function. However, existing SFCs, including QUILTS, all employ a single mapping scheme, which may not be suitable for all data spaces and query loads. To address this challenge, existing technologies have proposed a segmented SFC method based on a single benchmark, aiming to design personalized mapping functions for different data subspaces according to their data distribution and query load. However, typically no single pattern can dominate the entire data space, and similarly, optimizing an SFC as a benchmark should not rely solely on a single, determined SFC. Therefore, this invention proposes a dual-benchmark strategy, selecting two SFCs as benchmarks for constructing the BMTree, thereby improving the stability and performance of the BMTree structure in terms of optimization effectiveness. Summary of the Invention

[0005] To address the shortcomings of existing technologies, the purpose of this invention is to provide a query method and system based on the BMTree dimensionality reduction index structure.

[0006] A query method based on a BMTree dimensionality reduction index structure provided by the present invention includes: Step S1: Construct the BMTree dimensionality reduction index structure based on the dual benchmarks; Step S2: Obtain the query request input by the user, and perform the query to obtain the corresponding query results based on the query parameters and the BMTree dimensionality reduction index structure.

[0007] Preferably, step S1 includes: Step S1.1: Construct the global cost model and local cost model based on the space-filling curve SFC respectively; construct a hybrid cost model based on the global cost model and the local cost model; Step S1.2: Select the space filling curve SFC that matches the target dataset distribution and query pattern as a positive benchmark through the hybrid cost model, and generate a reverse benchmark based on the reverse principle; Step S1.3: Construct the forward benchmark Monte Carlo search tree architecture and the reverse benchmark Monte Carlo search tree architecture respectively based on the forward benchmark and the reverse benchmark; based on the forward benchmark Monte Carlo search tree architecture and the reverse benchmark Monte Carlo search tree architecture, construct the BMTree index structure using a hierarchical iteration and reward mechanism optimization strategy.

[0008] Preferably, step S1.1 includes: The SFC-based global cost model is used to measure the degree of global clustering in the query portion, including:

[0009] in, It is the first The length of each interval; The SFC-based local cost model is used to measure the degree of local clustering in the query portion, including:

[0010] The hybrid cost model includes: .

[0011] Preferably, step S1.2 includes: Step S1.2.1: Calculate the mixed cost of the space-filling curve SFC based on the mixed cost model, and select the SFC with the minimum mixed cost as the positive benchmark; Step S1.2.2: Perform a reversal operation on the forward SFC to obtain the opposite SFC, which serves as the reverse reference.

[0012] Preferably, step S1.3 includes: Step S1.3.1: Set the number of BMTree layers, and use the number of BMTree layers to represent the height of the BMTree structure; Step S1.3.2: Construct two independent Monte Carlo tree search architectures respectively; Step S1.3.3: BMTree adopts a hierarchical construction strategy, building layer by layer from the first layer down until the target depth is reached. In the construction of each layer, the forward and reverse benchmarks are injected into two independent Monte Carlo tree search architectures respectively. The Monte Carlo tree search architecture is used to simulate the construction of the spatial partitioning of the current layer based on the forward or reverse benchmark, and the number of data blocks traversed by the query is counted during the simulation. For each benchmark strategy, its corresponding reward value is calculated based on the number of data blocks traversed by the query. The reward values ​​of the two benchmark strategies are compared, and the benchmark with the lower reward value is selected as the construction mode of the current layer. After the optimal mode is selected for the current layer, the current mode information is recorded in the mode sequence, and the construction result of the current layer is used as the input state for the construction of the next layer. The entire construction process will continue to advance hierarchically until the preset target depth is reached.

[0013] Preferably, the simulation construction of the current layer spatial partitioning based on a Monte Carlo tree search architecture using a forward or reverse benchmark, and the counting of the number of data blocks traversed by the query during the simulation, includes: The selection phase uses the UCT algorithm to find the most promising nodes; The expansion phase partitions the space of the selected nodes based on the current baseline. During the simulation phase, multiple random simulations are conducted to count the number of data blocks traversed in each query, thereby evaluating the query performance under this benchmark based on spatial partitioning. The backhaul phase propagates the simulation results upwards to update the state information of each node.

[0014] Preferably, the step of calculating the corresponding reward value based on the number of data blocks traversed by the query; comparing the reward values ​​of the two benchmark strategies includes: obtaining the total reward value by summing the number of blocks traversed by all queries; wherein, the lower the reward value, the higher the query efficiency of the current benchmark.

[0015] A query system based on a BMTree dimensionality reduction index structure, provided by the present invention, includes: Module M1: Constructs a BMTree dimensionality reduction index structure based on dual benchmarks; Module M2: Obtains the user's query request and, based on the query parameters and the BMTree dimensionality reduction index structure, performs the query to obtain the corresponding query results.

[0016] Preferably, the module M1 includes: Module M1.1: Constructs a global cost model and a local cost model based on the space-filling curve SFC, respectively; constructs a hybrid cost model based on the global cost model and the local cost model; Module M1.2: Selects the space-filling curve (SFC) that matches the target dataset distribution and query pattern as a positive benchmark through a hybrid cost model, and generates a reverse benchmark based on the reverse principle; Module M1.3: Constructs a forward benchmark Monte Carlo search tree architecture and a reverse benchmark Monte Carlo search tree architecture based on the forward benchmark and the reverse benchmark respectively; based on the forward benchmark Monte Carlo search tree architecture and the reverse benchmark Monte Carlo search tree architecture, a hierarchical iteration and reward mechanism are used to select the best option to construct the BMTree index structure.

[0017] Preferably, step S1.1 includes: The SFC-based global cost model is used to measure the degree of global clustering in the query portion, including:

[0018] in, It is the first The length of each interval; The SFC-based local cost model is used to measure the degree of local clustering in the query portion, including:

[0019] The hybrid cost model includes: ; Step S1.2 includes: Step S1.2.1: Calculate the mixed cost of the space-filling curve SFC based on the mixed cost model, and select the SFC with the minimum mixed cost as the positive benchmark; Step S1.2.2: Perform a reversal operation on the forward SFC to obtain the opposite SFC, which serves as the reverse reference; Step S1.3 includes: Step S1.3.1: Set the number of BMTree layers, and use the number of BMTree layers to represent the height of the BMTree structure; Step S1.3.2: Construct two independent Monte Carlo tree search architectures respectively; Step S1.3.3: BMTree adopts a hierarchical construction strategy, building layer by layer from the first layer down until the target depth is reached. In the construction of each layer, the forward and reverse benchmarks are injected into two independent Monte Carlo tree search architectures respectively. The Monte Carlo tree search architecture is used to simulate the construction of the spatial partitioning of the current layer based on the forward or reverse benchmark, and the number of data blocks traversed by the query is counted during the simulation. For each benchmark strategy, its corresponding reward value is calculated based on the number of data blocks traversed by the query. The reward values ​​of the two benchmark strategies are compared, and the benchmark with the lower reward value is selected as the construction mode of the current layer. After the optimal mode is selected for the current layer, the current mode information is recorded in the mode sequence, and the construction result of the current layer is used as the input state for the construction of the next layer. The entire construction process will continue to advance hierarchically until the preset target depth is reached.

[0020] Compared with the prior art, the present invention has the following beneficial effects: 1. This invention proposes a dual-benchmark segmented SFC design method, which can better adapt to the diversity and complexity of data; 2. This invention selects a positive benchmark that matches the existing dataset and query load through a hybrid cost model, and constructs a BMTree structure through a combination strategy of positive and negative benchmarks; during the BMTree construction process, the optimal mode is selected through a reward mechanism to determine the construction strategy of each BMTree layer. 3. This invention proposes a dynamically adjusted space fill curve (SFC) strategy, which selects the most suitable baseline curve based on different regions of the dataset and their data distribution characteristics. This method can not only effectively improve the query efficiency of the index, but also reduce the space waste caused by uneven data distribution.

[0021] 4. In the implementation process, the segmented SFC design based on dual benchmarks selects a high-performance SFC as the forward benchmark through a hybrid cost model, and generates a corresponding reverse benchmark according to the reverse principle. Then, a BMTree structure is constructed using a combination strategy of forward and reverse benchmarks. When applying the Monte Carlo search tree algorithm, a hierarchical strategy is adopted; a pattern selection is performed at each BMTree layer to determine the construction pattern of the next layer. A reward mechanism is used to evaluate the combination strategy of forward and reverse benchmarks and select the optimal construction scheme. 5. The Monte Carlo Search Tree algorithm plays a crucial role in this process. Through multiple simulations and explorations of different SFC combinations, the Monte Carlo Search Tree can search for the optimal benchmark combination globally, thereby constructing a BMTree index structure that is more adaptable to the actual data distribution. Compared with the traditional static SFC design, this algorithm can adaptively adjust according to the data distribution, significantly improving the performance of the index structure under different query modes.

[0022] 6. This invention, based on a dual-benchmark segmented SFC design, further optimizes the space-filling effect of each layer of the BMTree by introducing a local optimization strategy. This local optimization not only improves query performance but also effectively reduces data structure and maintenance costs. Through multiple iterative optimizations, the final BMTree can better adapt to different data query needs, thereby improving the overall system efficiency and scalability.

[0023] 7. The present invention, based on a dual-benchmark segmented SFC design, effectively solves the problems of low query performance and uneven data distribution in traditional SFC design by flexibly combining various space filling curves and reinforcement learning algorithms. This method provides a new approach for large-scale data indexing and has strong application potential. Attached Figure Description

[0024] Other features, objects, and advantages of the present invention will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings: Figure 1 This is a diagram illustrating the mapping of range queries in BMTree.

[0025] Figure 2 This is a schematic diagram of the data distribution and query process.

[0026] Figure 3 This is a schematic diagram of the cost model.

[0027] Figure 4 A flowchart for building BMTree.

[0028] Figure 5 This is a diagram illustrating the performance comparison over time.

[0029] Figure 6 This is a performance comparison diagram in terms of I / O cost.

[0030] Figure 7 A diagram illustrating the percentage improvement in performance between different dual benchmarks and piecewise SFC based on Z-curves. Detailed Implementation

[0031] The present invention will now be described in detail with reference to specific embodiments. These embodiments will help those skilled in the art to further understand the present invention, but do not limit the invention in any way. It should be noted that those skilled in the art can make several changes and improvements without departing from the concept of the present invention. These all fall within the protection scope of the present invention.

[0032] Example 1 A query method based on a BMTree dimensionality reduction index structure provided by the present invention includes: Step S1: Construct the BMTree dimensionality reduction index structure based on the dual benchmarks; Specifically, step S1 includes: Step S1.1: Construct the global cost model and local cost model based on the space-filling curve SFC respectively; construct a hybrid cost model based on the global cost model and the local cost model; Specifically, step S1.1 includes: The SFC-based global cost model is used to measure the degree of global clustering in the query portion, including:

[0033] in, It is the first The length of each interval; The SFC-based local cost model is used to measure the degree of local clustering in the query portion, including:

[0034] The hybrid cost model includes: .

[0035] Step S1.2: Select the space filling curve SFC that matches the target dataset distribution and query pattern as a positive benchmark through the hybrid cost model, and generate a reverse benchmark based on the reverse principle; Specifically, step S1.2 includes: Step S1.2.1: Calculate the mixed cost of the space-filling curve SFC based on the mixed cost model, and select the SFC with the minimum mixed cost as the positive benchmark; Step S1.2.2: Perform a reversal operation on the forward SFC to obtain the opposite SFC, which serves as the reverse reference.

[0036] Step S1.3: Construct the forward benchmark Monte Carlo search tree architecture and the reverse benchmark Monte Carlo search tree architecture respectively based on the forward benchmark and the reverse benchmark; based on the forward benchmark Monte Carlo search tree architecture and the reverse benchmark Monte Carlo search tree architecture, construct the BMTree index structure using a hierarchical iteration and reward mechanism optimization strategy.

[0037] Specifically, step S1.3 includes: Step S1.3.1: Set the number of BMTree layers, and use the number of BMTree layers to represent the height of the BMTree structure; Step S1.3.2: Construct two independent Monte Carlo tree search architectures respectively; Step S1.3.3: BMTree adopts a hierarchical construction strategy, building layer by layer from the first layer down until the target depth is reached. In the construction of each layer, the forward and reverse benchmarks are injected into two independent Monte Carlo tree search architectures respectively. The Monte Carlo tree search architecture is used to simulate the construction of the spatial partitioning of the current layer based on the forward or reverse benchmark, and the number of data blocks traversed by the query is counted during the simulation. For each benchmark strategy, its corresponding reward value is calculated based on the number of data blocks traversed by the query. The reward values ​​of the two benchmark strategies are compared, and the benchmark with the lower reward value is selected as the construction mode of the current layer. After the optimal mode is selected for the current layer, the current mode information is recorded in the mode sequence, and the construction result of the current layer is used as the input state for the construction of the next layer. The entire construction process will continue to advance hierarchically until the preset target depth is reached.

[0038] The simulation construction of the current layer spatial partitioning using the Monte Carlo tree search architecture based on a forward or reverse datum, and the counting of the number of data blocks traversed by the query during the simulation, includes: The selection phase uses the UCT algorithm to find the most promising nodes; The expansion phase partitions the space of the selected nodes based on the current baseline. During the simulation phase, multiple random simulations are conducted to count the number of data blocks traversed in each query, thereby evaluating the query performance under this benchmark based on spatial partitioning. The backhaul phase propagates the simulation results upwards to update the state information of each node.

[0039] The process of calculating the reward value based on the number of data blocks traversed by the query and comparing the reward values ​​of the two benchmark strategies includes: obtaining the total reward value by summing the number of blocks traversed by all queries; wherein, the lower the reward value, the higher the query efficiency of the current benchmark.

[0040] Step S2: Obtain the query request input by the user, and perform the query to obtain the corresponding query results based on the query parameters and the BMTree dimensionality reduction index structure.

[0041] The present invention also provides a query system based on the BMTree dimensionality reduction index structure. The query system based on the BMTree dimensionality reduction index structure can be implemented by executing the process steps of the query method based on the BMTree dimensionality reduction index structure. That is, those skilled in the art can understand the query method based on the BMTree dimensionality reduction index structure as a preferred embodiment of the query system based on the BMTree dimensionality reduction index structure.

[0042] This invention aims to optimize the BMTree structure to improve query performance under different data distributions and query loads. Unlike traditional single-mapping SFC designs, this invention flexibly combines two SFCs as benchmarks, employs a combined positive and negative benchmark strategy, and incorporates a reinforcement learning algorithm (MCTS) to construct the optimized BMTree structure. This method better adapts to the diversity and complexity of data, thereby improving query efficiency and reducing space waste. In summary, this dual-benchmark-based index construction method provides a new approach for query optimization of large-scale datasets and has broad application potential. Experiments show that it improves query time by at least 24.34% and I / O cost by at least 12.73%.

[0043] Example 2 Example 2 is a preferred example of Example 1. A method for constructing a BMTree dimensionality reduction index structure based on dual benchmarks according to the present invention includes: Step 1: Construct a global cost model, a local cost model, and a hybrid cost model based on SFC; Step 2: Determine the forward and reverse benchmarks based on the hybrid cost model; Step 3: Construction of the basic BMTree with both positive and negative benchmarks, including: designing the Monte Carlo Tree Search (MCTS) architecture based on both positive and negative benchmarks; and the hierarchical construction strategy of the BMTree based on the dual benchmarks.

[0044] In this embodiment, the dataset is used This indicates that each data point have Each dimension is represented as For ease of understanding, this embodiment considers a two-dimensional data point. And it can be easily expanded. Each dimension. It can be converted into a bit string, as shown below: Each of them Is it 0 or 1? ,and It is the length of the bit string. The string depends on the dimension. X and The base number. For example, it can be converted to In previous studies on SFC-based multidimensional indexing, data point values ​​were typically mapped to fine-grained grid cells for discretization. SFC will... Mapping to a scalar value This is called the SFC value, obtained through a mapping function. An SFC value Can be used as data The key value is used to determine exist The order in which they appear.

[0045] Query workload aggregation This indicates that each query workload , represented as , middle Indicates the first Dimension 1 indicates that in the first... The minimum value of dimension, 2 represents the value at the th dimension. The maximum value of a dimension in two-dimensional space is expressed as... It indicates. Among them. For the minimum value data point, This represents the data point with the maximum value. For example... Figure 1 As shown, a query maximum value and minimum value After using BMTree (a type of dimensionality reduction index structure), the query range is obtained from the one-dimensional data and falls into page 2 and page 3 respectively. In BMTree, the green arrow indicates the minimum value and the blue arrow indicates the maximum value.

[0046] Based on the above description, given a database and a query workload By constructing a dimensionality reduction index structure That is, BMTree, which will take each data point Mapping to an SFC value When querying, the minimum and maximum values ​​are mapped using BMTree for the query.

[0047] In the dual-benchmark selection process, this embodiment proposes a strategy of combining positive and negative benchmarks. The core idea of ​​this strategy is to select the most suitable single SFC as the positive benchmark based on a hybrid cohesion cost model, and then generate a corresponding negative SFC according to the reverse principle, ultimately obtaining a pair of positive and negative benchmarks. Specifically, firstly, by analyzing the cohesion and cost-effectiveness of different single SFCs, an SFC that best matches the distribution and query pattern of the target dataset is selected as the positive benchmark. The selection of the positive benchmark considers not only the efficiency of space filling but also the need to reduce query time and computational complexity during the query process.

[0048] In selecting a positive benchmark, for an SFC-based query processing algorithm, query processing performance depends on the number of page visits. This number depends on the data distribution, as the coverage of each page varies depending on the data distribution. This embodiment uses... Figure 2 The model shown is used to analyze which features of the curve affect the number of page views.

[0049] First, this embodiment analyzes the distribution of sparse data, such as... Figure 2 As shown in a. Due to the limited number of page divisions, each page typically covers a large area. As the coverage of each page increases, the query range usually includes multiple consecutive pages, such as... Figure 2 Pages 2 and 3 in section a. When the intervals between query parts are short, fewer page visits can be expected. Therefore, in the case of sparse data distribution, the space-fill curve will perform better if the curve can globally aggregate the query parts within the query region.

[0050] This embodiment analyzes the situation of dense data distribution, such as Figure 2 b. In this case, due to the finer page division, each page has a smaller coverage area. Each page typically only covers a small portion of the query area, such as... Figure 2 Pages 5, 9, 11, and 12 in section b. Since page-based query processing algorithms typically only scan pages covering the query portion, larger intervals between query portions do not significantly affect the number of page visits. For example, even... Figure 2 Moving the query portion from page 5 in step b to page 1 will not change the number of pages required to be accessed. In other words, when the query portion exhibits local clustering, fewer page visits are needed. Therefore, in scenarios with dense data distribution, a space-fill curve that can locally cluster query portions within the query region is more suitable.

[0051] In summary, this embodiment should consider two clustering features of the space-filling curves on different data distributions to minimize the number of page visits. In the case of sparse data distribution, it is best for the curve to globally cluster the query portions mapped from the query region. In the case of dense data distribution, it is best for the curve to locally cluster these query portions.

[0052] Based on the above analysis, a cost model for measuring the dimensionality reduction effect of SFC can be obtained. The cost model is as follows: Figure 3 As shown. The horizontal axis represents the cost of global clustering properties based on the space-filling curve. The vertical axis represents the cost of local clustering properties based on the space-filling curve. The right and up directions indicate higher costs. For example, for space-filling curve 1, the cost of SFC-1 in the figure is lower than that of SFC-4, which is based on global clustering properties. Furthermore, the cost of SFC-1 is lower than that of SFC-2 and SFC-3, which are based on local clustering properties. This embodiment defines a global cost model to measure global clustering properties and a local cost model to measure local clustering properties.

[0053] The global cost model based on SFC includes: The global cost model measures the degree of global cohesion of query regions; in other words, global cohesion assesses the difference ratio between the minimum and maximum keys of the query region. Distance consists of the length of the query region and the length of the interval between query regions. Since the length of the query regions is equivalent to the area of ​​the query region, they are independent of the chosen space-fill curve. However, the length of the intervals depends on the chosen curve, so this study uses interval lengths to define global cohesion cost. Defined as the sum of the lengths of all intervals between the query parts. Let... It is the first The length of each interval is given by the following formula:

[0054] The SFC-based local cost model includes: Local cost models measure the degree to which query segments are locally clustered. Local clustering attributes have two aspects: the number of query segments and the variance of the interval length between query segments, such as... Figure 3 As shown in the figure. For example, SFC-1 has fewer query parts than SFC-2, while the variance of the interval length of SFC-2 is larger than that of SFC-3. To measure these two aspects, local cohesion is defined as the entropy of the ratio of the interval lengths between query parts, and the cost of local cohesion. Entropy is defined as the ratio of the interval lengths between query parts. Let... It is the first The length of each interval, note that if no interval exists. =0:

[0055] The SFC-based hybrid cost model includes: The above defines two cost metrics: a global cost model and a local cost model. These two measurement methods can be used in... Figure 3 The comparison of SFC-1 and SFC-4 is based on global cohesion, while the comparison of SFC-1 and either SFC-2 or SFC-3 is based on local cohesion. When comparing SFC-2 and SFC-4, a hybrid measurement of both global and local cohesion costs is required. Therefore, the total cohesion cost... Defined as and Multiplication. Let... For the first query between sections Intervals, like this:

[0056] Strictly speaking, cohesive costs should be prioritized based on the distribution of the data, considering either global or local factors. For example, global or local cost models should be applied separately if the data distribution is sparse or dense. A hybrid model is simple, but it is sufficient to capture the overall nature of the space-fill curve and the number of page visits.

[0057] Step 2 includes: After determining the positive baseline based on the aforementioned hybrid cost model, the next step is to generate a reverse SFC based on the reverse principle. The core idea of ​​the reverse principle is to obtain the corresponding space-filling curve by performing some form of inversion operation on the positive SFC. Specifically, if the positive baseline SFC is [ x, y, x, x, y Then, the reverse SFC can be obtained by rearranging the elements in the SFC according to a predetermined rule. y, x, y, y, x This reverse generation method introduces diversity into the original space-filling structure, thereby enhancing the adaptability and query performance of BMTree. The two SFCs represent different space-filling methods; the forward SFC may provide more efficient queries in densely populated areas, while the reverse SFC can provide better performance in sparsely populated areas or boundary queries. By combining forward and reverse SFCs, BMTree can achieve more balanced performance when handling complex query loads.

[0058] The generation of a reverse SFC is not merely a simple reversal of the forward benchmark; rather, it employs design strategies to ensure greater flexibility across different data regions and query scenarios. Specifically, a reverse SFC may use entirely different space-filling strategies in certain areas of the dataset compared to the forward SFC. This allows for more efficient retrieval under varying query loads. For instance, using a reverse SFC in query-intensive regions can avoid redundant query operations that might occur with the forward SFC, thereby reducing query response time.

[0059] Based on a dual-benchmark strategy combining positive and negative benchmarks, the final BMTree optimizes performance across multiple dimensions. During the query process, BMTree doesn't rely solely on a single SFC benchmark; instead, it switches between forward and reverse SFCs depending on the specific query requirements. Through this combined strategy, BMTree adaptively selects the optimal path for spatial search, thereby improving query efficiency, reducing query time, and optimizing storage space utilization.

[0060] By employing a combined positive and negative benchmark strategy, the dual-benchmark method proposed in this embodiment not only fully leverages the flexibility of the space-filling curve but also improves the adaptability and efficiency of the data structure under different query modes through the introduction of reverse SFC. Combined with the intelligent selection process of the hybrid cohesive cost model, the dual-benchmark strategy proposed in this embodiment provides a more efficient and dynamic solution for the construction of BMTree, offering new ideas and methods for query optimization of large-scale datasets.

[0061] Specifically, step 3 includes: This embodiment proposes a dual-benchmark Monte Carlo Tree Search (MCTS) architecture to optimize the construction process of BMTree. Traditional MCTS methods typically rely on a single strategy to guide tree construction and decision-making, while the dual-benchmark architecture introduces two different benchmarks for multi-strategy fusion, thereby better adapting to the characteristics of complex data distributions and high-dimensional spaces, and improving the flexibility and efficiency of BMTree construction.

[0062] Selection: The selection step is the core of MCTS, involving choosing the most promising node in the tree for further exploration. In a dual-benchmark architecture, the selection process involves evaluating the weights and influences of both benchmarks, and then, considering the query requirements and data distribution characteristics of the current node, selecting the optimal benchmark for further searching. During this process, MCTS uses the UCT algorithm based on the aforementioned formula to ensure both in-depth exploration and the utilization of already optimized benchmarks.

[0063] Node Expansion: In traditional MCTS, tree expansion is achieved by selecting the currently optimal action. In the dual-benchmark architecture, each expansion operation considers not only the current node's state but also two different benchmarks. Whenever a node needs expansion, MCTS evaluates the performance of both benchmarks on that node and selects the benchmark best suited to the current data distribution or query requirements for further spatial partitioning.

[0064] Simulation: The simulation phase estimates future rewards or performance by performing multiple random simulations at the current node. In dual-benchmark MCTS, the simulation attempts to simultaneously evaluate the effects of two benchmarks at the current node and adjusts the priority of the two benchmarks based on the simulation results. During this process, the algorithm uses Monte Carlo simulations to deduce the combined effects of different benchmarks, gradually optimizing the construction path of the BMTree.

[0065] Backpropagation: In the backpropagation step, the algorithm updates the nodes in the MCTS tree based on the simulation results. In the dual-benchmark architecture, each node updates its state based on the simulation results of both benchmarks, including how to adjust the weights of benchmark selection and how to optimize the spatial partitioning strategy. The backpropagation results not only update the performance metrics of the current node but also influence the decision-making strategies of parent and ancestor nodes, thereby improving the overall tree construction effect.

[0066] In the BMTree construction process of this embodiment, similar to the Segmented Space Filling Curve (SFC) method based on Z-curves, the BMTree construction also adopts a layered strategy. Specifically, a parameter `depth` is set to determine the number of layers of the BMTree to be constructed, where `depth` represents the height of the current BMTree structure. For example... Figure 4 As shown, in each iteration, a new BMTree structure is generated based on two different benchmarks.

[0067] In modern Monte Carlo Tree Search (MCTS) algorithms, the decision-making process typically relies on a selected Systemic Foundation (SFC) to evaluate the consequences of different choices. To improve decision accuracy and strategy diversity, this embodiment proposes establishing two independent Monte Carlo architectures and optimizing model performance through multi-strategy fusion. Each Monte Carlo architecture will operate independently and generate corresponding exploration paths based on different benchmarks, thereby providing diverse perspectives and decision paths.

[0068] Next, two different benchmarks are injected into the two Monte Carlo architectures respectively. In this step, two different benchmarks are injected into the two previously constructed Monte Carlo architectures. Injecting these two benchmarks into the two architectures separately ensures that each architecture can obtain policy performance based on its own criteria when evaluated independently. This process helps experiments obtain diverse information from different policies and increases the robustness of the model, avoiding the limitations of a single policy.

[0069] In each simulation and expansion of MCTS, the algorithm performs multiple random samplings and evaluates and selects the optimal policy based on the reward value for each path. For the dual-benchmark architecture, each choice is evaluated separately by the two benchmarks in each simulation, and the reward value for each path is calculated.

[0070] At the end of each iteration, the algorithm compares the reward values ​​calculated from the two benchmarks and selects the lower one as the decision mode for that level. The core idea of ​​this strategy is that by selecting the mode with the lower reward value, the algorithm is encouraged to be more cautious in its decision-making, avoiding over-reliance on local optima generated by a particular benchmark, thereby enhancing its global search capabilities.

[0071] Choosing a lower reward value as the pattern sequence for this layer is actually based on a conservative optimization principle. In complex decision problems, a single benchmark may be affected by uneven data or local patterns, leading to overfitting to certain limited regions. By selecting a benchmark with a lower reward, the algorithm can better explore the state space, reduce the risk of overfitting, and improve the generalization ability of the final model.

[0072] This strategy encourages MCTS to not only rely on the current best path, but also to maintain a balance across all paths, thereby providing a more diverse sequence of patterns for subsequent layer construction.

[0073] After completing the pattern selection for the current level, the algorithm proceeds to the construction process of the next level. When building the next level, the algorithm uses the pattern sequence selected in the previous level as a new benchmark or guiding principle to continue expanding the tree. The pattern sequence at each level adjusts the search strategy based on the results of the previous level, ensuring that each level's construction, guided by lower reward values, gradually approaches the global optimum.

[0074] In the new construction layer, the algorithm will again simulate and select based on two baseline strategies, receiving new reward feedback in each round of simulation. Based on this feedback, MCTS will re-evaluate the state of each node and select an optimal decision path. During this process, the low-reward pattern from the previous layer will serve as a new constraint, preventing the algorithm from getting stuck in local optima and guiding the search process to be more comprehensive and in-depth.

[0075] At the same time, moving to the next layer of construction also means that the pattern selection at each layer may change. As the search deepens, the model will adjust the pattern at each layer more finely to ensure that the construction of each layer can maximize the overall decision-making performance.

[0076] In summary, this layered BMTree construction method not only ensures a hierarchical spatial index structure, but also enables BMTree to adapt to complex data distribution characteristics through flexible selection of SFC and optimized space filling strategies, effectively improving the performance of large-scale data query and processing.

[0077] by For example, its benchmark is Given dataset and query workload First, the dataset is processed according to the sampling rate. from Randomly sample data points and sort them according to their SFC values. The sorted data points are then evenly divided into... Block, in which This represents the number of points in each block; next, the query workload is processed, querying for a given window. Depend on and This means that the SFC value of the minimum (maximum) point is first calculated using the following formula:

[0078] Then, and The falling block is represented as and Given and ,Will The calculation is as follows:

[0079] Finally, the reward is calculated according to the following formula. Intuitively, the lower the reward value achieved by the BMTree constructed by this strategy, the more it matches the selected objective.

[0080]

[0081] To prove the uniqueness of the mapping of the BMTree, this embodiment considers a two-step proof. First, it is proven that the BMTree maps to an input with only one output; then, it is proven that no two different inputs will have the same SFC value.

[0082] Specifically, given an input BMTree calculates by traversing the path from the root node to the leaf node. Based on the observations, each There is only one path, corresponding to one BMP, and one value; given two inputs. , Two conditions should be considered: i) If x， Sharing the same path means x， They share the same BMP. However, note that each BMP stored in the BMTree is itself injective, therefore, when x and They have the same value if and only if ;ii) if x and When they have different values, x and Having the same value path indicates x and x‘ It is associated with different bmps in T. Based on BMTree, we can know... x and The traversed paths share the first few nodes (at least the root node) until they branch at a specific node. Then one route goes to the left child, and another route goes to the right child. Let... This indicates the depth of the branch node. Then, Used for calculation , Share the same first Position, and exist Position and There are different bit values, one is 0 and the other is 1. Based on this, it was confirmed that... = This ensures uniqueness.

[0083] Given each dimension And a BMTree of x and satisfy This embodiment demonstrates ≥ This embodiment is based on x and We will discuss two cases based on the path: whether they share the same path on BMTree.

[0084] Specifically, (1) when x and When they share the same path, that is, when they are inT The same BMP is corresponding to the middle. Monotonicity comes from the shared mode. (2) When x and When dealing with different paths, the two inputs will share a portion of their path and enter different branches at the branch nodes. Assume the branch node is located at depth... This means the value and Having the same first Position, and they are in The bits are kept distinct. To represent monotonicity, i) first, it is stated that the value corresponding to the path to the right must be greater than the value corresponding to the path to the left. ii) Then, the value corresponding to the right path is displayed. Corresponding to the path on the left This is because in In position, it respectively from x and It tracks the same dimensions and the same bit indices. Given Satisfying all dimensions The conditions must be met. x The rightmost bit is 1, and... x‘ The leftmost bit is 0. Therefore, based on i) and ii), this embodiment yields... > The conclusion.

[0085] This embodiment's verification experiment was conducted on a synthetic dataset. For the synthetic dataset, a granularity of [size missing] was generated in a two-dimensional data space. × The data points follow a uniform distribution (denoted as UNI).

[0086] It follows efficient spatial indexing techniques to generate query workloads; it generates different types of window queries, each type of query starting from... A fixed region is selected, and a fixed aspect ratio is chosen from {4, 1, 1 / 4}; each workload contains multiple types of queries with different combinations of regions and ratios. Furthermore, this embodiment experiment generates skewed workloads (denoted as SKE) with different distributions by following efficient spatial indexing techniques.

[0087] To evaluate the performance of the proposed dual-benchmark segmented SFC compared to existing SFCs, this embodiment integrates the dual-benchmark segmented SFC, segmented SFC, and baseline SFC into a traditional index and a learned index structure. First, the dual-benchmark segmented SFC (and other SFCs) is integrated into the PostgreSQL database system, using a built-in B+ tree variant in PostgreSQL, with the SFC value as the key. Here, the PostgreSQL B+ tree is a disk-based index. It is chosen for experiments to evaluate the performance of the segmented SFC in different scenarios.

[0088] In this embodiment, the following SFC methods were selected as the baseline for the experiment: (1) z-curve; (2) QUILTS; (3) Segmented SFC based on z-curve.

[0089] For experiments conducted using PostgreSQL, this example uses I / O cost and query time recorded by the PostgreSQL system.

[0090] This example experiment uses PyTorch 1.9 and Python 3.8 to train BMTree. The experiment was conducted on an 80-core server, using an Intel(R) Xeon(R) Gold 6248 CPU@2.50GHz and 64.0GB RAM, without utilizing GPU resources for model training.

[0091] The experimental section is mainly divided into three parts. The first part verifies that the superior dual-benchmark piecewise SFC design is better than other SFC baselines. The second part compares the performance of the superior dual-benchmark design with the Z-curve-based piecewise SFC design. The third part compares the performance of the dual-benchmark piecewise SFC design using a hybrid cost model with the Z-curve-based piecewise SFC design.

[0092] First, we validated the superior dual-benchmark segmented SFC design by comparing its query time and I / O cost with other SFC baselines in a PostgreSQL database. The selected dataset was a uniform dataset, and the selected query workload was a skewed query workload. The results are as follows: Figure 5 and 6 As shown. In terms of query time, the dual-benchmark SFC optimizes time by at least 24% and at most 131%; in terms of I / O cost, the dual-benchmark SFC optimizes time by at least 12% and at most 197%.

[0093] Secondly, in the performance comparison between the optimally selected dual-benchmark design and the piecewise SFC design based on the Z-curve, a hybrid mode of Z-curve and C-curve was chosen as the dual-benchmark, denoted as A+B. The experiment used 0 and 1 to represent two dimensions; Z0 represents SFC starting at dimension 0, and Z0101 / 1010 represents SFC variants that cycle through this 0101 / 1010 pattern, thus obtaining multiple combined dual-benchmark modes. The experimental results are as follows... Figure 7 As shown, the dual-benchmark SFC outperforms the single-benchmark SFC based on the Z-curve in both query time and I / O cost, and is worst under a mixture of the C-curve and its corresponding Z-curve, but best under two different Z-curve variants of the BMP model. Query time is improved by a worst of 8% and a maximum of 16%.

[0094] Finally, in the performance comparison between the dual-benchmark segmented SFC design (D-SFC) using a hybrid cost model and the segmented SFC design based on the Z curve (Z-SFC), as shown in Table 1, D-SFC outperforms I / O cost in both query time and I / O cost, with a time optimization of 24.34% and an I / O cost optimization of 12.73%.

[0095] Table 1 Performance Comparison of D-SFC and Z-SFC

[0096] Those skilled in the art will understand that, in addition to implementing the system, apparatus, and their modules provided by this invention in purely computer-readable program code, the same program can be implemented in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers by logically programming the method steps. Therefore, the system, apparatus, and their modules provided by this invention can be considered a hardware component, and the modules included therein for implementing various programs can also be considered structures within the hardware component; alternatively, modules for implementing various functions can be considered both software programs implementing the method and structures within the hardware component.

[0097] Specific embodiments of the present invention have been described above. It should be understood that the present invention is not limited to the specific embodiments described above, and those skilled in the art can make various changes or modifications within the scope of the claims, which do not affect the essence of the present invention. Unless otherwise specified, the embodiments and features described in this application can be arbitrarily combined with each other.

Claims

1. A query method based on a BMTree dimensionality reduction index structure, characterized in that, include: Step S1: Construct the BMTree dimensionality reduction index structure based on the dual benchmarks; Step S2: Obtain the query request input by the user, and perform the query to obtain the corresponding query results based on the query parameters and the BMTree dimensionality reduction index structure.

2. The query method based on the BMTree dimensionality reduction index structure according to claim 1, characterized in that, Step S1 includes: Step S1.1: Construct the global cost model and local cost model based on the space-filling curve SFC respectively; construct a hybrid cost model based on the global cost model and the local cost model; Step S1.2: Select the space filling curve SFC that matches the target dataset distribution and query pattern as a positive benchmark through the hybrid cost model, and generate a reverse benchmark based on the reverse principle; Step S1.3: Construct the forward benchmark Monte Carlo search tree architecture and the reverse benchmark Monte Carlo search tree architecture respectively based on the forward benchmark and the reverse benchmark; based on the forward benchmark Monte Carlo search tree architecture and the reverse benchmark Monte Carlo search tree architecture, construct the BMTree index structure using a hierarchical iteration and reward mechanism optimization strategy.

3. The query method based on the BMTree dimensionality reduction index structure according to claim 2, characterized in that, Step S1.1 includes: The SFC-based global cost model is used to measure the degree of global clustering in the query portion, including: in, It is the first The length of each interval; The SFC-based local cost model is used to measure the degree of local clustering in the query portion, including: The hybrid cost model includes: 。 4. The query method based on the BMTree dimensionality reduction index structure according to claim 2, characterized in that, Step S1.2 includes: Step S1.2.1: Calculate the mixed cost of the space-filling curve SFC based on the mixed cost model, and select the SFC with the minimum mixed cost as the positive benchmark; Step S1.2.2: Perform a reversal operation on the forward SFC to obtain the opposite SFC, which serves as the reverse reference.

5. The query method based on the BMTree dimensionality reduction index structure according to claim 2, characterized in that, Step S1.3 includes: Step S1.3.1: Set the number of BMTree layers, and use the number of BMTree layers to represent the height of the BMTree structure; Step S1.3.2: Construct two independent Monte Carlo tree search architectures respectively; Step S1.3.3: BMTree adopts a hierarchical construction strategy, building layer by layer from the first layer down until the target depth is reached. In the construction of each layer, the forward and reverse benchmarks are injected into two independent Monte Carlo tree search architectures respectively. The Monte Carlo tree search architecture is used to simulate the construction of the spatial partitioning of the current layer based on the forward or reverse benchmark, and the number of data blocks traversed by the query is counted during the simulation. For each benchmark strategy, its corresponding reward value is calculated based on the number of data blocks traversed by the query. The reward values ​​of the two benchmark strategies are compared, and the benchmark with the lower reward value is selected as the construction mode of the current layer. After the optimal mode is selected for the current layer, the current mode information is recorded in the mode sequence, and the construction result of the current layer is used as the input state for the construction of the next layer. The entire construction process will continue to advance hierarchically until the preset target depth is reached.

6. The query method based on the BMTree dimensionality reduction index structure according to claim 5, characterized in that, The simulation construction of the current layer spatial partitioning using the Monte Carlo tree search architecture based on a forward or reverse datum, and the counting of the number of data blocks traversed by the query during the simulation, includes: The selection phase uses the UCT algorithm to find the most promising nodes; The expansion phase partitions the space of the selected nodes based on the current baseline. During the simulation phase, multiple random simulations are conducted to count the number of data blocks traversed in each query, thereby evaluating the query performance under this benchmark based on spatial partitioning. The backhaul phase propagates the simulation results upwards to update the state information of each node.

7. The query method based on the BMTree dimensionality reduction index structure according to claim 5, characterized in that, The reward value is calculated based on the number of data blocks spanned by the query. Comparing the reward values ​​of the two benchmark strategies includes: obtaining the total reward value by summing the block spans of all queries; wherein, the lower the reward value, the higher the query efficiency of the current benchmark.

8. A query system based on a BMTree dimensionality reduction index structure, characterized in that, include: Module M1: Constructs a BMTree dimensionality reduction index structure based on dual benchmarks; Module M2: Obtains the user's query request and, based on the query parameters and the BMTree dimensionality reduction index structure, performs the query to obtain the corresponding query results.

9. The query system based on the BMTree dimensionality reduction index structure according to claim 8, characterized in that, The module M1 includes: Module M1.1: Constructs a global cost model and a local cost model based on the space-filling curve SFC, respectively; constructs a hybrid cost model based on the global cost model and the local cost model; Module M1.2: Selects the space-filling curve (SFC) that matches the target dataset distribution and query pattern as a positive benchmark through a hybrid cost model, and generates a reverse benchmark based on the reverse principle; Module M1.3: Constructs a forward benchmark Monte Carlo search tree architecture and a reverse benchmark Monte Carlo search tree architecture based on the forward benchmark and the reverse benchmark respectively; based on the forward benchmark Monte Carlo search tree architecture and the reverse benchmark Monte Carlo search tree architecture, a hierarchical iteration and reward mechanism are used to select the best option to construct the BMTree index structure.

10. The query system based on the BMTree dimensionality reduction index structure according to claim 9, characterized in that, Step S1.1 includes: The SFC-based global cost model is used to measure the degree of global clustering in the query portion, including: in, It is the first The length of each interval; The SFC-based local cost model is used to measure the degree of local clustering in the query portion, including: The hybrid cost model includes: ； Step S1.2 includes: Step S1.2.1: Calculate the mixed cost of the space-filling curve SFC based on the mixed cost model, and select the SFC with the minimum mixed cost as the positive benchmark; Step S1.2.2: Perform a reversal operation on the forward SFC to obtain the opposite SFC, which serves as the reverse reference; Step S1.3 includes: Step S1.3.1: Set the number of BMTree layers, and use the number of BMTree layers to represent the height of the BMTree structure; Step S1.3.2: Construct two independent Monte Carlo tree search architectures respectively; Step S1.3.3: BMTree adopts a hierarchical construction strategy, building layer by layer from the first layer down until the target depth is reached. In the construction of each layer, the forward and reverse benchmarks are injected into two independent Monte Carlo tree search architectures respectively. The Monte Carlo tree search architecture is used to simulate the construction of the spatial partitioning of the current layer based on the forward or reverse benchmark, and the number of data blocks traversed by the query is counted during the simulation. For each benchmark strategy, its corresponding reward value is calculated based on the number of data blocks traversed by the query. The reward values ​​of the two benchmark strategies are compared, and the benchmark with the lower reward value is selected as the construction mode of the current layer. After the optimal mode is selected for the current layer, the current mode information is recorded in the mode sequence, and the construction result of the current layer is used as the input state for the construction of the next layer. The entire construction process will continue to advance hierarchically until the preset target depth is reached.

Images