Method for establishing multi-layer database index model, data indexing method and device

Abstract

This article relates to the field of big data and provides a method for establishing a multi-layer database index model, a data indexing method, and an apparatus, wherein the model establishment method includes: determining at least one filter bit based on the number of bits of data stored in the VBP / HBP; for each data block, establishing a multi-layer database index model for the data block in the following manner: forming a layer of prefix data from the binary data of each front filter bit in the data block; determining the first bitmap and second bitmap of each layer of prefix data based on the decimal numbers in each layer of prefix data and the number of decimal numbers in the prefix data of that layer; wherein the first bitmap is used to reflect the distribution of decimal numbers in the prefix data in binary form, and the second bitmap is used to reflect the distribution of the number of numbers less than the decimal number in the prefix data in binary form. The first bitmap and second bitmap of each layer of prefix data constitute the database index model of each layer. This embodiment can improve data query efficiency.

Description

Technical Field

[0001] This article relates to the field of big data, and in particular to a method for establishing a multi-layer database index model, a data indexing method and a device. Background Art

[0002] In modern analytical database systems, data is often stored in columnar form, and the performance of data queries is crucial to their execution. Numerous scanning and storage solutions exist to address these core operations, but many require specific workload attributes, such as specific data distribution and query selectivity, to deliver good performance.

[0003] Existing scanning methods are mainly divided into index scanning and sequential scanning. However, both scanning methods have inherent shortcomings.

[0004] Index scans, such as B+ tree indexes and Zone Map, are suitable for scenarios with low selectivity. Specifically, the B+ tree index builds a balanced binary tree through data, which has the problem of occupying a large amount of memory space. In addition, in analytical databases, the query selectivity is often relatively high, so B+ tree is not a good choice. Zone Map divides the data into blocks and stores some metadata for each data block, such as the maximum and minimum values of the elements in the block. When querying, many data blocks can be skipped directly by simply comparing this metadata information, but Zone Map is only suitable for scenarios with skewed data. When the data distribution cannot meet ideal conditions, the performance of Zone Map is not ideal. In summary, for index scans, when the selectivity is high, it may cause a large amount of random memory access.

[0005] For sequential scans, when the selectivity is low, a large amount of data that does not meet the conditions may be accessed.

[0006] Existing solutions that change the data storage layout mainly include VBP (Vertical bit-parallel) and HBP (Horizontal bit-parallel). Both of these data storage methods have the problem of reduced filtering efficiency due to individual data disturbances.

[0007] In the context of VBP, the original data binary bits are split and stored vertically in register words, which can use SIMD instructions to obtain higher parallelism. At the same time, the storage layout can obtain the ability of "early stop", thereby further improving the speed of data query. Among them, the "early stop" mechanism refers to the VBP storage layout. When the n-th bit of the predicate binary can fully determine the current query condition result, the binary bits after the n-th bit can be stopped from scanning, thereby improving the efficiency of the scan and eliminating the need to scan all binary bits. However, in most cases, due to the disturbance of individual data, the efficiency of "early stop" will drop sharply, resulting in the scanning of more "useless" data bits, seriously affecting the performance of data query. The original data is stored in VBP mode. When the data query x>v is performed, v is also stored in VBP mode. The query result finally generates a result-bitmap, such as Figure 1 , Figure 1 Where x is the original data stored in VBP mode, and the value of v is 109. When the query condition x>v is executed, the final result result-bitmap is obtained, see Figure 1 Right side, Figure 1 In the query, "?" indicates that it is not certain whether the condition is satisfied, "N" indicates not satisfied, and "Y" indicates satisfied. Due to the interference of the original data 108, all data bits need to be scanned to obtain the final query result.

[0008] HBP can store the binary bits of data horizontally according to the register word length. The storage format is as follows: Figure 2 As shown in the figure, it can also use the "early stop" mechanism like VBP, but it also has the problem of reduced filtering efficiency due to individual data disturbances, and the concurrency of data queries is not as high as VBP. Summary of the Invention

[0009] This paper addresses the "early stop" mechanism of existing VBP and HBP storage layouts, which is significantly affected by data noise. This can lead to inefficient data filtering, requiring the scanning of large amounts of "useless" data bits to obtain the final result, resulting in a sharp drop in performance. Existing indexes require a specific workload, but this workload is often difficult to accurately determine in specific application scenarios, resulting in low query efficiency.

[0010] In order to solve the above technical problems, this paper provides a method for establishing a multi-layer database index model, wherein the multi-layer database index model is used to determine the stop filtering position, including:

[0011] Determining at least one filter bit based on the number of bits of data stored in the VBP / HBP;

[0012] For each data block in the VBP / HBP, a multi-layer database index model for the data block is established in the following manner:

[0013] Combining the binary data of each front-filtered bit in the data block into a layer of prefix data;

[0014] Determine a first bitmap and a second bitmap of each layer of prefix data according to the decimal number in each layer of prefix data and the number of each decimal number in the prefix data of the layer;

[0015] The first bitmap is used to reflect the distribution of decimal numbers in the prefix data in binary form, and the second bitmap is used to reflect the distribution of numbers smaller than the decimal number in the prefix data in binary form.

[0016] The first bitmap and the second bitmap of the prefix data of each layer constitute the database index model of each layer.

[0017] As a further embodiment of the present invention, determining at least one filter bit according to the number of bits of data stored in the VBP / HBP includes:

[0018] Make sure 2 is met n ≤N, where N is the number of bits of data stored in the VBP / HBP;

[0019] Set the n value that meets the condition to the filter bit.

[0020] As a further embodiment of the present invention, determining the first bitmap and the second bitmap of each layer of prefix data according to the decimal number in each layer of prefix data and the number of decimal numbers in the prefix data of the layer includes:

[0021] Determine the first bitmap of each layer of prefix data according to the decimal number in each layer of prefix data;

[0022] The decimal numbers in each layer of prefix data and the number of decimal numbers in the prefix data of this layer are converted into data pairs in the following format:<prefix,count1> , prefix represents the decimal number in each layer of prefix data, count1 represents the number of prefixes in each layer of prefix data;

[0023] Sort the data pairs of each layer of prefix data in ascending order according to prefix, and obtain the first data pair queue of each layer of prefix data;

[0024] According to the first data pair queue of each layer of prefix data, a second data pair queue of each layer of prefix data is calculated, wherein each data pair in the second data pair queue has the following format:<prefix,count2> , where count2 represents the number less than prefix;

[0025] The second bitmap of each layer of prefix data is determined according to count2 in the second data pair queue of each layer of prefix data.

[0026] As a further embodiment of the present invention, the second data pair queue of each layer of prefix data is calculated based on the first data pair queue of each layer of prefix data, including:

[0027] Set the prefix in the first data pair queue of each layer of prefix data to the prefix in the second data pair queue of each layer of prefix data;

[0028] For each count1 in the queue, the first data of each layer of prefix data i Perform the following calculation to get the count2 of the second data pair queue of each layer prefix data i :

[0029]

[0030] Among them, count2 i Used to reflect the prefix data of this layer that is less than the decimal number prefix i The number of , i and j represent the i-th and j-th data.

[0031] As a further embodiment of the present invention, determining the first bitmap of each layer of prefix data according to the decimal number in each layer of prefix data includes:

[0032] For each layer of prefix data, create a n The first data vector of elements, where n is the filter bit corresponding to the prefix data;

[0033] Set the subscript of the first data vector from 0 to 2 in order from front to back n -1, the element with subscript x in the first data vector is used to reflect whether the decimal number of the prefix data contains the value x, with the first predetermined value indicating existence and the second predetermined value indicating non-existence;

[0034] The first data vector is filled according to the decimal number in the prefix data to obtain the first bitmap.

[0035] As a further embodiment of the present invention, determining the second bitmap of each layer of prefix data according to count2 in the second data pair queue of each layer of prefix data includes:

[0036] For each layer of prefix data, a second data vector is established according to the amount of stored data contained in the data block;

[0037] The subscripts of the second data vector are sequentially set from 0 to Z-1 in order from front to back, where Z is the number of stored data contained in the data block. The element with subscript x in the second data vector is used to reflect whether the value x exists in count2 of the second data pair queue of the prefix data of the layer, with a first predetermined value indicating existence and a second predetermined value indicating non-existence.

[0038] According to count2 in the second data pair queue of the prefix data, the second data vector is filled to obtain a second bitmap.

[0039] A second aspect of this document provides a data indexing method based on the multi-layer database indexing model described in any of the aforementioned embodiments, comprising:

[0040] receiving a query request, wherein the query request includes a comparison condition and a predicate;

[0041] For each data block in VBP / HBP, perform the following scan:

[0042] Sort the database index models of each layer of the data block in ascending order of the filter bit;

[0043] Obtaining a database index model corresponding to a filter bit from the sorting result;

[0044] Determine the decimal value Y of the first filtered bit of the predicate binary;

[0045] Calculate the amount of data in the data block that meets the comparison condition based on the decimal value Y and the database index model corresponding to the filter bit;

[0046] Determine whether the ratio of the amount of data that meets the comparison condition to the amount of data in the data block meets a preset threshold;

[0047] If so, determine that the filter bit is a stop filter bit, perform bit-by-bit filtering on the data before the stop filter bit in the data block, and perform overall comparison on the data after the stop filter bit in the data block;

[0048] If not, the database index model corresponding to the next filter bit is obtained from the sorting result, and the steps of determining the decimal value Y and subsequent steps are repeated.

[0049] As a further embodiment of the present invention, calculating the amount of data in the data block that meets the comparison condition based on the decimal value Y and the database index model corresponding to the filter bit includes: when the comparison condition is less than, determining the amount that meets the comparison condition using the following formula 1:

[0050] select C(rankP(Y)), formula 1;

[0051] Where C represents the second bitmap, P represents the first bitmap, rankP(Y) represents the number of bits in the subscript [0, Y] of the first bitmap that contains the first predetermined value, and select C(rankP(Y)) represents the subscript corresponding to the rankP(Y)th first predetermined value of the second bitmap, that is, the number that meets the comparison condition.

[0052] As a further embodiment of the present invention, calculating the amount of data in the data block that meets the comparison condition according to the decimal value Y and the database index model corresponding to the filter bit further includes:

[0053] When the comparison condition is greater than, use the following formula 2 to determine the number of times the comparison condition is met:

[0054] Z-select C(rankP(Y)+1), formula 2;

[0055] Wherein, Z represents the amount of data stored in VBP / HBP, select C(rankP(Y)+1) represents the subscript corresponding to the first predetermined value of rankP(Y)+1 of the second bitmap, that is, the number that meets the comparison condition of being less than or equal to.

[0056] As a further embodiment of the present invention, calculating the amount of data in the data block that meets the comparison condition according to the decimal value Y and the database index model corresponding to the filter bit further includes:

[0057] When the comparison condition is equal, use the following formula 3 to determine the number of times the comparison condition is met:

[0058] select C(rankP(Y)+1)-select C(rankP(Y)), formula three.

[0059] A third aspect of the present invention provides an apparatus for establishing a multi-layer database index model, wherein the multi-layer database index model is used to determine a stop filtering position, including:

[0060] a filter bit determination unit, configured to determine at least one filter bit according to the number of bits of data stored in the VBP / HBP;

[0061] The model building unit is used to build a multi-layer database index model for each data block in the VBP / HBP using the following modules:

[0062] A prefix data generating module, configured to form a layer of prefix data from the binary data of each front-filtered bit in the data block;

[0063] A bitmap generation module, configured to determine a first bitmap and a second bitmap of each layer of prefix data according to the decimal numbers in each layer of prefix data and the number of the decimal numbers in the prefix data of the layer;

[0064] The first bitmap is used to reflect the distribution of decimal numbers in the prefix data in binary form, and the second bitmap is used to reflect the distribution of the number of numbers smaller than the decimal number in the prefix data in binary form;

[0065] The model generation module is used to form a database index model for each layer using the first bitmap and the second bitmap of the prefix data of each layer.

[0066] A fourth aspect of the present invention further provides a data indexing device based on the multi-layer database indexing model described in any of the aforementioned embodiments, comprising:

[0067] A receiving unit, configured to receive a query request, wherein the query request includes a comparison condition and a predicate;

[0068] The scanning unit performs scanning on each data block in the VBP / HBP, wherein the scanning unit includes:

[0069] A sorting module is used to sort the database index models of each layer of the data block in order of the filter bit from small to large;

[0070] An acquisition module is used to obtain a database index model corresponding to a filter bit from the sorting result;

[0071] a determination module for determining a decimal value Y of a front-filtered bit of a predicate binary;

[0072] A calculation module, configured to calculate the amount of data in the data block that meets the comparison condition based on the decimal value Y and the database index model corresponding to the filter bit;

[0073] A loop control module is used to determine whether the ratio of the amount of data that meets the comparison condition to the amount of data in the data block meets a preset threshold;

[0074] If so, determine that the filter bit is a stop filter bit, perform bit-by-bit filtering on the data before the stop filter bit in the data block, and perform overall comparison on the data after the stop filter bit in the data block;

[0075] If not, the database index model corresponding to the next filter bit is obtained from the sorting result, and the steps of determining the decimal value Y and subsequent steps are repeated.

[0076] The fifth aspect of this document provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method described in any of the aforementioned embodiments when executing the computer program.

[0077] A sixth aspect of this document provides a computer storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor of a computer device, the computer program executes instructions of the method according to any of the aforementioned embodiments.

[0078] A sixth aspect of this document provides a computer program product, which includes a computer program, and when the computer program is executed by a processor, implements the method described in any of the above embodiments.

[0079] The multi-layer database index model, data indexing method, and device described herein are suitable for data search in VBP / HBP storage layouts. The multi-layer database index model for multiple data blocks constructed using the above method is able to be unaffected by specific workload limits (such as selectivity) during scanning. Based on the database index model corresponding to each filter bit and the decimal value Y of the predicate's front filter bit, the proportion of data meeting the query conditions can be calculated in advance. This data proportion provides strong judgment and data support for the next step of data query processing, and can further utilize technologies such as data prefetching to accelerate data filtering and improve query efficiency. Specifically, when the proportion meets a predetermined threshold, a stop filter bit is determined, and a bit-by-bit comparison is performed on the data before the stop filter bit in the data block. When the stop filter bit is reached, the bit-by-bit comparison stops, and a comprehensive comparison is performed on the data after the stop filter bit. This can improve the efficiency of data filtering in the VBP / HBP storage layout, while also accelerating data scanning and reducing the access to large amounts of data.

[0080] In order to make the above and other purposes, features and advantages of this article more obvious and easy to understand, the following specifically cites preferred embodiments and provides detailed descriptions in conjunction with the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0081] In order to more clearly illustrate the technical solutions in the embodiments of this article or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of this article. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0082] Figure 1 A schematic diagram of performing data search under a VBP structure in the prior art is shown;

[0083] Figure 2 A schematic diagram of performing data search under the HBP structure in the prior art is shown;

[0084] Figure 3 A schematic diagram showing the process of constructing a multi-layer database index model according to an embodiment of this invention;

[0085] Figure 4 A schematic diagram of a data block in the first embodiment of this invention is shown;

[0086] Figure 5 This embodiment is shown Figure 3 A schematic diagram of a multi-layer database index model for data blocks shown;

[0087] Figure 6 A flowchart showing a process for determining the first bitmap and the second bitmap of each layer of prefix data in an embodiment of this invention is shown;

[0088] Figure 7 A flowchart of the second data pair queue calculation process according to the embodiment of this invention is shown;

[0089] Figure 8 A flowchart of a first bitmap determination process according to an embodiment of the present invention is shown;

[0090] Figure 9 A flowchart of a second bitmap determination process according to an embodiment of the present invention is shown;

[0091] Figure 10 A flow chart of the data indexing method according to an embodiment of the present invention is shown;

[0092] Figure 11 The structure diagram of the device for establishing a multi-layer database index model according to the embodiment of this invention is shown;

[0093] Figure 12 The structure diagram of the data indexing device according to the embodiment of this invention is shown;

[0094] Figure 13 The diagram shows the structure of the computer device according to the embodiment of this article.

[0095] Description of the accompanying symbols:

[0096] 1110, filter bit determination unit;

[0097] 1120, model building unit;

[0098] 1121. Prefix data generation module;

[0099] 1122. Bitmap generation module;

[0100] 1123. Model generation module;

[0101] 1210, receiving unit;

[0102] 1220, scanning unit;

[0103] 1221, sorting module;

[0104] 1222. Get module;

[0105] 1223. Determine module;

[0106] 1224, calculation module;

[0107] 1225, loop control module;

[0108] 1302. Computer equipment;

[0109] 1304, processor;

[0110] 1306. Memory;

[0111] 1308, driving mechanism;

[0112] 1310, input / output module;

[0113] 1312. Input devices;

[0114] 1314. Output device;

[0115] 1316. Presentation equipment;

[0116] 1318. Graphical User Interface;

[0117] 1320, network interface;

[0118] 1322, communication link;

[0119] 1324. Communication bus. DETAILED DESCRIPTION

[0120] The following will be combined with the accompanying drawings to clearly and completely describe the technical solutions in the embodiments of this document. Obviously, the embodiments described are only part of the embodiments of this document, not all of the embodiments. Based on the embodiments of this document, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this document.

[0121] It should be noted that the terms "first," "second," and the like in the specification and claims herein and in the accompanying drawings are used to distinguish similar objects and are not necessarily used to describe a particular order or precedence. It should be understood that the terms used in this manner are interchangeable where appropriate so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having," as well as any variations thereof, are intended to cover non-exclusive inclusions. For example, a process, method, apparatus, product, or device comprising a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units that are not explicitly listed or that are inherent to these processes, methods, products, or devices.

[0122] This specification provides method operation steps as described in the embodiments or flowcharts, but more or fewer operation steps may be included based on routine or non-creative work. The order of steps listed in the embodiments is only one way of executing the steps among many orderings and does not represent the only execution order. When a system or device product is actually executed, the method can be executed in the order shown in the embodiments or the drawings or in parallel.

[0123] It should be noted that the multi-layer database index model, data index method and device in this article can be used in the financial field, and can also be used in any field other than the financial field. The application field of the multi-layer database index model, data index method and device in this article is not limited.

[0124] The "early stop" mechanism in existing VBP and HBP storage layouts is significantly affected by data interference, which can lead to low data filtering efficiency. This requires scanning a large number of "useless" data bits to obtain the final result, resulting in a sharp drop in performance. Existing indexes require a specific workload, but this workload is often difficult to accurately determine in specific application scenarios, resulting in low query efficiency.

[0125] To address these issues, this paper establishes a multi-layer database index model for each data block stored in the VBP / HBP. Each multi-layer database index model includes a first bitmap and a second bitmap corresponding to multiple filter bits. The first bitmap is used to reflect the distribution of decimal numbers in the data block's prefix data in binary form, while the second bitmap is used to reflect the distribution of numbers smaller than the decimal number in the data block's prefix data in binary form. The multi-layer database index model for each data block determines the stop filter bit. A bit-by-bit search is performed for the data before the stop filter bit, and a global search is performed for the data after the stop filter bit.

[0126] Specifically, such as Figure 3 As shown, a method for establishing a multi-layer database index model, wherein the multi-layer database index model is used to determine a stop filtering position, includes:

[0127] Step 310, determining at least one filter bit based on the number of bits of data stored in the VBP / HBP;

[0128] Step 320: For each data block in the VBP / HBP, a multi-layer database index model for the data block is established using the following method:

[0129] Step 321, the binary data of each front-filtered bit in the data block is combined into a layer of prefix data;

[0130] Step 322, determining a first bitmap and a second bitmap of each layer of prefix data according to the decimal numbers in each layer of prefix data and the number of each decimal number in the prefix data of that layer;

[0131] The first bitmap, referred to as Prefix-bitmap, is used to reflect the distribution of decimal numbers in the prefix data in binary form. The second bitmap, referred to as Count-bitmap, is used to reflect the distribution of the number of numbers smaller than the decimal number in the prefix data in binary form.

[0132] Step 323: construct a database index model for each layer using the first bitmap and the second bitmap of the prefix data for each layer.

[0133] The multi-layer database index model of multiple data blocks constructed by the above method can be free from the limits of specific workloads (such as the influence of high or low selectivity) during scanning, providing strong data support for the next step of data query processing and improving the efficiency of data filtering and query speed.

[0134] Specifically, in order to improve the efficiency of subsequent determination of the stop filter bit, the filter bit in step 310 can be increased in an exponential manner of 2. Specifically, first determine whether the filter bit satisfies 2 n ≤N, where N is the number of bits of data stored in the VBP / HBP, such as 8 bits, 16 bits, etc. Then, the n that meets the condition is set as the filter bit. Taking 8-bit data as an example, the filter bit can be set to 2 or 4.

[0135] When step 320 is implemented, the amount of data stored in the data block can be determined based on the maximum concurrent SIMD execution of the operating system. In this paper, a multi-layer database index model is generated for each data block.

[0136] like Figure 4 As shown, assuming that a data block includes 8 8-bit data, the corresponding filter bits include 2 and 4, Figure 4 The prefix data formed by the binary data of each leading filter bit in the data block shown are: the prefix data corresponding to filter bit 2 is: 00, 00, 00, 00, 00, 10, 01, 00; the prefix data corresponding to filter bit 4 is: 0001, 0010, 0010, 0001, 0000, 1000, 0100, 0011. Correspondingly, the decimal numbers of the prefix data corresponding to filter bit 2 are 0, 1, 3; and the decimal numbers of the prefix data corresponding to filter bit 2 are 0, 1, 2, 3, 4, 8. The number of decimal numbers in filter bit 2 is: 5, 1, 1, and the number of decimal numbers in filter bit 4 is: 1, 2, 2, 1, 1, 1. Figure 4 The multi-layer database index model determined by the data block shown is as follows Figure 5 shown.

[0137] The bitmap described in this article is a bit array consisting of only 0s and 1s, with each element representing either 0 or 1 and occupying only one bit. In implementation, 1 is set as the first predetermined bit and 0 is set as the second predetermined bit. If the i-th element in the bitmap is in the first predetermined bit, it indicates that the value i-1 exists in the distribution; if it is in the second predetermined bit, it indicates that the value i-1 does not exist.

[0138] Specifically, the subscripts of the elements in the first bitmap are arithmetic progressions starting from 0 and incrementing by 1. The elements in the first bitmap are used to indicate whether the decimal number in the prefix data has the subscript value. The subscripts of the elements in the second bitmap are also arithmetic progressions starting from 0 and incrementing by 1. The elements in the second bitmap are used to indicate whether the subscript of the element exists in a distribution of values less than the decimal number in the prefix data. The process of determining the first and second bitmaps will be described in subsequent embodiments and will not be described in detail here.

[0139] In one embodiment of this invention, Figure 6 As shown, the above step 322 determines the first bitmap and the second bitmap of each layer of prefix data according to the decimal numbers in each layer of prefix data and the number of each decimal number in the prefix data of the layer, including:

[0140] Step 601, determining the first bitmap of each layer of prefix data according to the decimal number in each layer of prefix data;

[0141] Step 602: The decimal numbers in each layer of prefix data and the number of decimal numbers in the prefix data of that layer are converted into data pairs in the following format:<prefix,count1> , prefix represents the decimal number in each layer of prefix data, count1 represents the number of prefixes in each layer of prefix data;

[0142] Step 603: sort the data pairs of each layer of prefix data in ascending order of prefix to obtain a first data pair queue of each layer of prefix data;

[0143] Step 604: Calculate a second data pair queue for each layer of prefix data based on the first data pair queue for each layer of prefix data, wherein each data pair in the second data pair queue has the following format:<prefix,count2> , where count2 represents the number less than prefix;

[0144] Step 605 : Determine the second bitmap of each layer of prefix data according to count2 in the second data pair queue of each layer of prefix data.

[0145] In one embodiment of this invention, Figure 7 As shown, the above step 604 calculates the second data pair queue of each layer of prefix data based on the first data pair queue of each layer of prefix data, including:

[0146] Step 701, setting the prefix in the first data pair queue of each layer of prefix data to the prefix in the second data pair queue of each layer of prefix data;

[0147] Step 702: For each count1 in the queue, the first data of each layer of prefix data is i Perform the following calculation to get the count2 of the second data pair queue of each layer prefix data i :

[0148]

[0149] Among them, count2 i Used to reflect the prefix data of this layer that is less than the decimal number prefix i The number of , i and j represent the i-th and j-th data.

[0150] In one embodiment of this invention, Figure 8 As shown, the above step 601 determines the first bitmap of each layer of prefix data according to the decimal number in each layer of prefix data, including:

[0151] Step 801: For each layer of prefix data, establish a n The first data vector of elements, where n is the filter bit corresponding to the prefix data;

[0152] Step 802: Set the subscripts of the first data vector from 0 to 2 in order from front to back. n -1, the element with subscript x in the first data vector is used to reflect whether the decimal number of the prefix data contains the value x, with the first predetermined value indicating existence and the second predetermined value indicating non-existence;

[0153] Step 803: Fill the first data vector according to the decimal number in the prefix data to obtain a first bitmap.

[0154] From step 801, we know that the first bitmap size is 2 n bit.

[0155] In step 802, the first predetermined value is 1 or 0, and the second predetermined value is 0 or 1. The specific values can be set as needed and are not limited herein. Generally, the first predetermined value is 1, and the second predetermined value is 0. The first data vector is populated according to the rules in step 802. For example, if the prefix data contains a decimal 1, the position at the subscript 1 in the first data vector, i.e., the second position, is set to 1 to indicate the presence of a decimal 1 in the prefix data.

[0156] In one embodiment of this invention, Figure 9As shown, the above step 605 determines the second bitmap of each layer of prefix data according to count2 in the second data pair queue of each layer of prefix data, including:

[0157] Step 901: For each layer of prefix data, a second data vector is created according to the amount of stored data contained in the data block;

[0158] Step 902: Set the subscripts of the second data vector from 0 to Z-1 in order from front to back, where Z is the number of stored data contained in the data block. The element with subscript x in the second data vector is used to reflect whether the value x exists in count2 of the second data pair queue of the prefix data layer. A first predetermined value indicates existence, and a second predetermined value indicates non-existence.

[0159] Step 903 : Fill the second data vector according to count2 in the second data pair queue of the prefix data to obtain a second bitmap.

[0160] In step 901, the number of stored data in the data block is the number of data in the data block. Figure 4 For example, Figure 4 The number of data stored in the data block shown is 8, that is, Z=8.

[0161] The first predetermined value and the second predetermined value in step 902 are set according to needs, and their specific values are not limited herein.

[0162] Step 903 is implemented according to the principle in step 902. For example, Figure 4 In the example, when the filter bit is 4, the prefix data includes 0000 (decimal: 0), 0001 (decimal: 1), 0010 (decimal: 2), 0011 (decimal: 3), 0100 (decimal: 4), and 1000 (decimal: 8). In descending decimal order, the counts are 1, 2, 2, 1, 1, 1. This is converted to key-value storage: <0, 1>, <1, 2>, <2, 2>, <3, 1>, <4, 1>, <8, 1>. The previous count is added based on the count: <0, 0>, <1, 1>, <2, 3>, <3, 5>, <4, 6>, <8, 7>. Based on this queue, the second bitmap corresponding to the first four bits can be obtained by following the principles of step 902.

[0163] In one embodiment of the present invention, based on determining the multi-layer database index model of each data block in the VBP / HBP, the data indexing process can be performed. Specifically, Figure 10 As shown, the data indexing method of the multi-layer database index model includes:

[0164] Step 1010: receiving a query request, wherein the query request includes a comparison condition and a predicate;

[0165] Step 1020: For each data block in the VBP / HBP, perform the following scan:

[0166] Step 1021, sorting the database index models of each layer of the data block in ascending order of the filter bit;

[0167] Step 1022: Obtain a database index model corresponding to a filter bit from the sorting result;

[0168] Step 1023, determine the decimal value Y of the front filter bit of the predicate binary;

[0169] Step 1024, calculating the amount of data in the data block that meets the comparison condition based on the decimal value Y and the database index model corresponding to the filter bit;

[0170] Step 1025, determining whether the ratio of the amount of data meeting the comparison condition to the amount of data in the data block meets a preset threshold;

[0171] If so, determine that the filter bit is a stop filter bit, perform bit-by-bit filtering on the data before the stop filter bit in the data block, and perform overall comparison on the data after the stop filter bit in the data block;

[0172] If not, the database index model corresponding to the next filter position is obtained from the sorting result, and step 1023 and subsequent steps are repeated.

[0173] Specifically, the comparison conditions in step 1010 include but are not limited to <, ≤, >, ≥, =, etc., denoted as op, and the predicate refers to the query predicate. For example, "age>20" refers to the value greater than 20 in the age column.

[0174] Furthermore, the above step 1024 calculates the amount of data in the data block that meets the comparison condition based on the decimal value Y and the database index model corresponding to the filter bit, including:

[0175] When the comparison condition is less than, use the following formula 1 to determine the number of times the comparison condition is met:

[0176] select C(rankP(Y)), formula 1;

[0177] Where C represents the second bitmap, P represents the first bitmap, rankP(Y) represents the number of bits in the subscript [0, Y] of the first bitmap that contains the first predetermined value, and select C(rankP(Y)) represents the subscript corresponding to the rankP(Y)th first predetermined value of the second bitmap, that is, the number that meets the comparison condition.

[0178] Furthermore, the above step 1024 calculates the amount of data in the data block that meets the comparison condition based on the decimal value Y and the database index model corresponding to the filter bit, and further includes:

[0179] When the comparison condition is greater than, use the following formula 2 to determine the number of times the comparison condition is met:

[0180] Z-select C(rankP(Y)+1), formula 2;

[0181] Wherein, Z represents the amount of stored data contained in the data block, select C(rankP(Y)+1) represents the subscript corresponding to the first predetermined value of rankP(Y)+1 of the second bitmap, that is, the number that meets the comparison condition of being less than or equal to.

[0182] Furthermore, the above step 1024 calculates the amount of data in the data block that meets the comparison condition based on the decimal value Y and the database index model corresponding to the filter bit, and further includes:

[0183] When the comparison condition is equal, use the following formula 3 to determine the number of times the comparison condition is met:

[0184] select C(rankP(Y)+1)-select C(rankP(Y)), formula three.

[0185] In specific implementation, step 1025 not only determines whether the proportion of the amount of data that meets the comparison conditions to the amount of data in the data block meets the preset threshold; it can also determine whether the proportion of the amount of data that does not meet the comparison conditions to the amount of data in the data block meets a certain threshold. If it does, the current filter bit is used as the stop filter bit.

[0186] The preset threshold in step 1025 can be specified manually or obtained by analyzing historical data, and this document does not limit its specific value.

[0187] During implementation, if the ratio calculated by a certain filter bit does not meet the preset threshold, and the ratio calculated by the next filter bit of the filter bit is 100%, it is determined that all data in the data block meets the requirement, and there is no need to compare the remaining data.

[0188] The data indexing method based on a multi-layer database index model described herein is suitable for data searches in VBP / HBP storage layouts. Based on the database index model corresponding to each filter bit and the decimal value Y of the predicate's front filter bit, the proportion of data meeting the query criteria can be calculated in advance. This data proportion provides strong judgment and data support for the next step of data query processing, and can further utilize technologies such as data prefetching to accelerate data filtering and improve query efficiency. Specifically, when the proportion meets a predetermined threshold, a stop filter bit is determined, and a bit-by-bit comparison is performed on the data before the stop filter bit in the data block. When the stop filter bit is reached, the bit-by-bit comparison stops, and a comprehensive comparison is performed on the data after the stop filter bit. This improves the efficiency of data filtering in the VBP / HBP storage layout, while also accelerating data scanning and reducing the access to large amounts of data.

[0189] The following Figure 4 and Figure 5 The following example illustrates the process of stopping the filter bit search. Specifically, Figure 4 The data block shown includes a multi-layer database index model corresponding to filter bits 2 and 4. The preset threshold is, for example, 60%. The query condition is X<51, where < is the comparison condition and 51 is the predicate. The corresponding binary value is 00110011. The first two digits are 00 (decimal value Y1 is 0), and the first four digits are 0011 (decimal value Y2 is 3).

[0190] The data indexing process based on the multi-layer database index model includes:

[0191] 1. For the first-level database index model (i.e., the multi-level database index model corresponding to filter bit 2), the decimal value is Y1, and the query condition is changed to search for data with X<0 in the prefix data corresponding to filter bit 2.

[0192] Combine Figure 5 The first and second bitmaps of the upper and middle layers can be calculated as follows:

[0193] rankP(0)=1, selectC(1)=0.

[0194] Then filter to the first two digits of the first layer, and the number of items that definitely meet the conditions is 0, which does not meet the preset threshold. At this time, obtain the second-layer database retrieval model (that is, the multi-layer database index model corresponding to filter bit 4).

[0195] 2. For the second-level database retrieval model, the decimal value is now Y2, and the query condition is changed to search for data with X<3 from the prefix data corresponding to filter bit 4.

[0196] Combine Figure 5 The first and second bitmaps of the middle and lower layers can be calculated:

[0197] rankP(3)=4, selectC(4)=5, then filtering to the first four digits of the second layer, the number of elements that definitely meet the conditions is 5. At this time, because the total number of elements is 8, the ratio is 5 / 8=62.5%, which is greater than the set threshold of 60%.

[0198] From this we can determine that the stop filter bit is 4, for Figure 4 In the data block shown, the data before the stop filter bit 4 is searched bit by bit for the number of bits that meet the conditions, and the data after the stop filter bit 4 is searched for all the number of bits that meet the conditions. There is no need to filter and compare the data after the stop filter bit 4 bit by bit, which can improve the query efficiency.

[0199] Based on the same inventive concept, this article also provides a device for establishing a multi-layer database index model and a data indexing device, as described in the following embodiments. Since the principles of solving problems by the device for establishing a multi-layer database index model and the data indexing device are similar to those of the method for establishing a multi-layer database index model and the data indexing method, the implementation of the device for establishing a multi-layer database index model and the data indexing device can refer to the method for establishing a multi-layer database index model and the data indexing method, and the repeated parts will not be repeated. Specifically, Figure 11 As shown,

[0200] The device for establishing a multi-layer database index model includes:

[0201] A filter bit determination unit 1110 is configured to determine at least one filter bit according to the number of bits of data stored in the VBP / HBP;

[0202] The model building unit 1120 is used to build a multi-layer database index model for each data block in the VBP / HBP using the following modules:

[0203] The prefix data generating module 1121 is used to form a layer of prefix data from the binary data of each front-filtered bit in the data block;

[0204] A bitmap generation module 1122 is configured to determine a first bitmap and a second bitmap of each layer of prefix data based on the decimal numbers in each layer of prefix data and the number of decimal numbers in the prefix data of that layer;

[0205] The first bitmap is used to reflect the distribution of decimal numbers in the prefix data in binary form, and the second bitmap is used to reflect the distribution of the number of numbers smaller than the decimal number in the prefix data in binary form;

[0206] The model generation module 1123 is used to construct a database index model for each layer using the first bitmap and the second bitmap of the prefix data for each layer.

[0207] like Figure 12As shown, the data indexing device includes:

[0208] The receiving unit 1210 is configured to receive a query request, wherein the query request includes a comparison condition and a predicate;

[0209] The scanning unit 1220 performs scanning on each data block in the VBP / HBP, wherein the scanning unit 1220 includes:

[0210] Sorting module 1221, used to sort the database index models of each layer of the data block in ascending order of the filter bit;

[0211] An acquisition module 1222 is configured to acquire a database index model corresponding to a filter bit from the sorting result;

[0212] A determination module 1223 is used to determine a decimal value Y of the front filter bit of the predicate binary;

[0213] A calculation module 1224 is configured to calculate the amount of data in the data block that meets the comparison condition based on the decimal value Y and the database index model corresponding to the filter bit;

[0214] The loop control module 1225 is used to determine whether the ratio of the amount of data that meets the comparison condition to the amount of data in the data block meets a preset threshold;

[0215] If so, determine that the filter bit is a stop filter bit, perform bit-by-bit filtering on the data before the stop filter bit in the data block, and perform overall comparison on the data after the stop filter bit in the data block;

[0216] If not, the database index model corresponding to the next filter bit is obtained from the sorting result, and the steps of determining the decimal value Y and subsequent steps are repeated.

[0217] The multi-layer database indexing model, data indexing method, and device provided in this article have the following advantages:

[0218] (1) Divide the data stored in the VBP / HBP into data blocks and construct a multi-layer database index model for each data block. This can improve the efficiency of data filtering based on the multi-layer database index model and further accelerate data access. For example, the stop filter bit is first determined based on the multi-layer database index model of the data block, and the data before the stop filter bit of the data block is searched bit by bit, and the data at the stop filter bit of the data block is searched as a whole.

[0219] (2) The multi-layer database index model is a bit-level lightweight index model that takes up little memory space and has low maintenance cost, and can provide a good balance between performance and memory space.

[0220] In one embodiment of the present invention, a computer device is also provided, such as Figure 13 As shown, computer device 1302 includes a processor 1304, a memory 1306, and a computer program stored on memory 1306 and executable on processor 1304. When processor 1304 executes the computer program, the method described in any of the aforementioned embodiments is implemented. Processor 1304 may be, for example, one or more central processing units (CPUs), each of which may implement one or more hardware threads. Memory 1306 is used to store any type of information, such as code, settings, data, and the like. For example, and without limitation, memory 1306 may include any one or more combinations of the following: any type of RAM, any type of ROM, a flash memory device, a hard disk, an optical disk, and the like. More generally, any memory may use any technology to store information. Furthermore, any memory may provide volatile or non-volatile retention of information. Furthermore, any memory may represent a fixed or removable component of computer device 1302. In one embodiment, when processor 1304 executes associated instructions stored in any memory or combination of memories, computer device 1302 may perform any operation of the associated instructions. The computer device 1302 also includes one or more drive mechanisms 1308 for interacting with any storage, such as a hard disk drive mechanism, an optical disk drive mechanism, and the like.

[0221] The computer device 1302 may also include an input / output module 1310 (I / O) for receiving various inputs (via input devices 1312) and for providing various outputs (via output devices 1314). A specific output mechanism may include a presentation device 1316 and an associated graphical user interface 1318 (GUI). In other embodiments, the input / output module 1310 (I / O), input devices 1312, and output devices 1314 may not be included, and the computer device 1302 may simply function as a computer device in a network. The computer device 1302 may also include one or more network interfaces 1320 for exchanging data with other devices via one or more communication links 1322. One or more communication buses 1324 couple the components described above together.

[0222] The communication link 1322 can be implemented in any manner, for example, through a local area network, a wide area network (e.g., the Internet), a point-to-point connection, etc., or any combination thereof. The communication link 1322 can include any combination of hardwired links, wireless links, routers, gateway functions, name servers, etc., governed by any protocol or combination of protocols.

[0223] Corresponding to Figure 3 、 Figures 6-10The embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon, which executes the steps of the above method when executed by a processor.

[0224] The embodiment of the present invention also provides a computer readable instruction, wherein when the processor executes the instruction, the program causes the processor to execute the following Figure 3 、 Figures 6-10 The method shown.

[0225] It should be understood that in the various embodiments of this document, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this document.

[0226] It should also be understood that in the embodiments herein, the term "and / or" merely describes an association between associated objects, indicating that three possible relationships exist. For example, "A and / or B" could represent: A alone, A and B simultaneously, or B alone. Furthermore, the character " / " in this document generally indicates an "or" relationship between the associated objects.

[0227] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the above description has generally described the composition and steps of each example according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this document.

[0228] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0229] In the several embodiments provided herein, it should be understood that the disclosed systems, devices, and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices, or units, or can be an electrical, mechanical, or other form of connection.

[0230] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the embodiments herein.

[0231] In addition, the functional units in the various embodiments herein may be integrated into a single processing unit, each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0232] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this article is essentially or the part that contributes to the existing technology, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in each embodiment of this article. The aforementioned storage medium includes: various media that can store program code, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0233] This article uses specific embodiments to illustrate the principles and implementation methods of this article. The description of the above embodiments is only used to help understand the methods and core ideas of this article. At the same time, for those skilled in the art, based on the ideas of this article, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation to this article.

Claims

1. A method for establishing a multi-layer database index model, characterized in that: The multi-layer database index model is used to determine the stop filtering position, including: Determining at least one filter bit based on the number of bits of data stored in the VBP / HBP; For each data block in the VBP / HBP, a multi-layer database index model for the data block is established in the following manner: Combining the binary data of each front-filtered bit in the data block into a layer of prefix data; Determine a first bitmap and a second bitmap of each layer of prefix data according to the decimal number in each layer of prefix data and the number of each decimal number in the prefix data of the layer; The first bitmap is used to reflect the distribution of decimal numbers in the prefix data in binary form, and the second bitmap is used to reflect the distribution of the number of numbers smaller than the decimal number in the prefix data in binary form; The first bitmap and the second bitmap of the prefix data of each layer constitute the database index model of each layer.

2. The method according to claim 1, wherein Determining at least one filter bit based on the number of bits of data stored in the VBP / HBP includes: Make sure 2 is met n ≤N, where N is the number of bits of data stored in the VBP / HBP; Set the n value that meets the condition to the filter bit.

3. The method according to claim 1, wherein Determining the first bitmap and the second bitmap of each layer of prefix data according to the decimal number in each layer of prefix data and the number of decimal numbers in the prefix data of the layer includes: Determine the first bitmap of each layer of prefix data according to the decimal number in each layer of prefix data; The decimal numbers in each layer of prefix data and the number of decimal numbers in the prefix data of this layer are transferred into the following data pairs:<prefix,count1> , prefix represents the decimal number in each layer of prefix data, count1 represents the number of prefixes in each layer of prefix data; Sort the data pairs of each layer of prefix data in ascending order according to prefix, and obtain the first data pair queue of each layer of prefix data; According to the first data pair queue of each layer of prefix data, a second data pair queue of each layer of prefix data is calculated, wherein each data pair in the second data pair queue has the following format:<prefix,count2> , where count2 represents the number less than prefix; The second bitmap of each layer of prefix data is determined according to count2 in the second data pair queue of each layer of prefix data.

4. The method according to claim 3, wherein The second data pair queue of each layer of prefix data is calculated based on the first data pair queue of each layer of prefix data, including: Set the prefix in the first data pair queue of each layer of prefix data to the prefix in the second data pair queue of each layer of prefix data; For each count1 in the queue, the first data of each layer of prefix data i Perform the following calculation to get the count2 of the second data pair queue of each layer prefix data i : Among them, count2 i Used to reflect the prefix data of this layer that is less than the decimal number prefix i The number of , i and j represent the i-th and j-th data.

5. The method according to claim 3, wherein According to the decimal number in each layer of prefix data, the first bit map of each layer of prefix data is determined to include: For each layer of prefix data, create a n The first data vector of elements, where n is the filter bit corresponding to the prefix data; Set the subscript of the first data vector from 0 to 2 in order from front to back n -1, the element with subscript x in the first data vector is used to reflect whether the decimal number of the prefix data contains the value x, with the first predetermined value indicating existence and the second predetermined value indicating non-existence; The first data vector is filled according to the decimal number in the prefix data to obtain the first bitmap.

6. The method according to claim 3, wherein Determining, based on count2 in the second data pair queue of each layer of prefix data, the second bitmap of each layer of prefix data includes: For each layer of prefix data, a second data vector is established according to the amount of stored data contained in the data block; The subscripts of the second data vector are sequentially set from front to back to 0 to Z-1, where Z is the amount of data stored in the data block. The element with subscript x in the second data vector is used to reflect whether the value x exists in count2 of the second data pair queue of the prefix data of the layer, with a first predetermined value indicating existence and a second predetermined value indicating non-existence. According to count2 in the second data pair queue of the prefix data, the second data vector is filled to obtain a second bitmap.

7. A data indexing method based on a multi-layer database indexing model established by the method according to any one of claims 1 to 6, characterized in that: include: receiving a query request, wherein the query request includes a comparison condition and a predicate; For each data block in VBP / HBP, perform the following scan: Sort the database index models of each layer of the data block in ascending order of the filter bit; Obtaining a database index model corresponding to a filter bit from the sorting result; Determine the decimal value Y of the first filtered bit of the predicate binary; Calculate the amount of data in the data block that meets the comparison condition based on the decimal value Y and the database index model corresponding to the filter bit; Determine whether the ratio of the amount of data that meets the comparison condition to the amount of data in the data block meets a preset threshold; If so, determine that the filter bit is a stop filter bit, perform bit-by-bit filtering on the data before the stop filter bit in the data block, and perform overall comparison on the data after the stop filter bit in the data block; If not, the database index model corresponding to the next filter bit is obtained from the sorting result, and the steps of determining the decimal value Y and subsequent steps are repeated.

8. The method according to claim 7, wherein Calculating the amount of data in the data block that meets the comparison condition based on the decimal value Y and the database index model corresponding to the filter bit includes: when the comparison condition is less than, determining the amount that meets the comparison condition using the following formula 1: select C(rankP(Y)), formula 1; Where C represents the second bitmap, P represents the first bitmap, rankP(Y) represents the number of bits in the subscript [0, Y] of the first bitmap that contains the first predetermined value, and select C(rankP(Y)) represents the subscript corresponding to the rankP(Y)th first predetermined value of the second bitmap, that is, the number that meets the comparison condition.

9. The method according to claim 8, wherein Calculating the amount of data in the data block that meets the comparison condition based on the decimal value Y and the database index model corresponding to the filter bit further includes: When the comparison condition is greater than, use the following formula 2 to determine the number of times the comparison condition is met: Z-select C(rankP(Y)+1), formula 2; Wherein, Z represents the amount of stored data contained in the data block, select C(rankP(Y)+1) represents the subscript corresponding to the first predetermined value of rankP(Y)+1 of the second bitmap, that is, the number that meets the comparison condition of being less than or equal to.

10. The method according to claim 9, wherein Calculating the amount of data in the data block that meets the comparison condition based on the decimal value Y and the database index model corresponding to the filter bit further includes: When the comparison condition is equal, use the following formula 3 to determine the number of times the comparison condition is met: select C(rankP(Y)+1)-select C(rankP(Y)), formula three.

11. A device for establishing a multi-layer database index model, characterized in that: The multi-layer database index model is used to determine the stop filtering position, including: a filter bit determination unit, configured to determine at least one filter bit according to the number of bits of data stored in the VBP / HBP; The model building unit is used to build a multi-layer database index model for each data block in the VBP / HBP using the following modules: A prefix data generating module, configured to form a layer of prefix data from the binary data of each front-filtered bit in the data block; A bitmap generation module, configured to determine a first bitmap and a second bitmap of each layer of prefix data according to the decimal numbers in each layer of prefix data and the number of the decimal numbers in the prefix data of the layer; The first bitmap is used to reflect the distribution of decimal numbers in the prefix data in binary form, and the second bitmap is used to reflect the distribution of the number of numbers smaller than the decimal number in the prefix data in binary form; The model generation module is used to form a database index model for each layer using the first bitmap and the second bitmap of the prefix data of each layer.

12. A data indexing device based on a multi-layer database indexing model established by the method according to any one of claims 1 to 6, characterized in that: include: A receiving unit, configured to receive a query request, wherein the query request includes a comparison condition and a predicate; The scanning unit performs scanning on each data block in the VBP / HBP, wherein the scanning unit includes: A sorting module is used to sort the database index models of each layer of the data block in order of filter bits from small to large; An acquisition module is used to obtain a database index model corresponding to a filter bit from the sorting result; a determination module for determining a decimal value Y of a front-filtered bit of a predicate binary; A calculation module, configured to calculate the amount of data in the data block that meets the comparison condition based on the decimal value Y and the database index model corresponding to the filter bit; A loop control module is used to determine whether the ratio of the amount of data that meets the comparison condition to the amount of data in the data block meets a preset threshold; If so, determine that the filter bit is a stop filter bit, perform bit-by-bit filtering on the data before the stop filter bit in the data block, and perform overall comparison on the data after the stop filter bit in the data block; If not, the database index model corresponding to the next filter bit is obtained from the sorting result, and the steps of determining the decimal value Y and subsequent steps are repeated.

13. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the method according to any one of claims 1 to 10 is implemented.

14. A computer storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor of a computer device, the computer program executes the instructions of the method according to any one of claims 1 to 10.

15. A computer program product, characterized in that The computer program product comprises a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 10 is implemented.

Images