An intelligent indexing method and device based on query perception

Abstract

The application belongs to the technical field of intelligent indexing, and discloses an intelligent indexing method and device based on query perception. The weight of each key in a query workload of a user is determined through a probability distribution model by acquiring the query workload. An index structure is constructed through a weighted minimum collision degree method by using the weight of each key. A target position of an inserted key is located through a query operation, and it is checked according to a preset condition whether the inserted key will trigger an adjustment operation of the index structure. If the adjustment operation is triggered, a new index structure is reconstructed, and the index performance is effectively improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of intelligent indexing, and in particular to an intelligent indexing method and device based on query perception. BACKGROUND

[0002] In the storage and use of data, databases are essential key links. Among them, there are various different index structures in the database to meet the needs of different access modes, for example, B-trees are generally used for range requests, hash maps have superior performance in single key lookup, and Bloom filters are usually used to check whether a record exists. At present, these indexes still use general data structures, without any assumptions about data distribution, and without taking advantage of the more common patterns that exist in real-world data. In the actual indexing process, it is often believed that all queries are equal, without considering that the different query frequencies in the actual index are not the same, resulting in low performance of the index. SUMMARY

[0003] To this end, embodiments of the present application provide an intelligent indexing method and device based on query perception, which effectively improves the overall performance of the index.

[0004] In a first aspect, the present application provides an intelligent indexing method based on query perception.

[0005] The present application is achieved by the following technical solutions:

[0006] An intelligent indexing method based on query perception, the method comprising:

[0007] Obtaining a query workload of a user, and determining the weight of each key in the query workload through a probability distribution model;

[0008] Using the weight of each key, an index structure is constructed through a weighted minimum collision degree method;

[0009] The target position of the inserted key is located through a query operation, and it is checked according to a preset condition whether the inserted key will trigger an adjustment operation of the index structure;

[0010] If the adjustment operation is triggered, a new index structure is reconstructed.

[0011] In a preferred example of the present application, the probability distribution model can be further configured to fit the workload distribution of the adjacent two windows in the query workload, and the Kullback-Leibler divergence is used to judge the deviation of the workload distribution of the adjacent two windows;

[0012] If the deviation is less than a preset deviation value, the offset variable is set to 0;

[0013] If the deviation is greater than or equal to the preset deviation value, the offset variable is increased by 1.

[0014] In a preferred example of the present application, it can be further configured that the steps of obtaining a user's query workload and determining the weight of each key in the query workload by using a probability distribution model include:

[0015] Given a query workload, a probability distribution model is used to fit the access ratio of each key in the query workload;

[0016] The weight of each key is calculated based on the access ratio of each key.

[0017] In a preferred example of the present application, it can be further configured that the step of constructing the index structure by using the weight of each key through a weighted minimum collision degree method includes:

[0018] Set the initial value of collision degree C to 1 and initialize x, y, U C :

[0019]

[0020] Starting from k0, determine whether the keys in the key set K satisfy F(k i +L(i))-F(k i )≥U, if satisfied, move to the next key, update L(i), and repeat the verification; if not satisfied, increase the collision degree C by 1, update x, y, U C The value of ki is retrieved and the verification loop starting from the current ki is returned until all keys are checked, and the model M of the index structure and the expected value of the collision degree C are obtained.

[0021] In a preferred example of the present application, it can be further configured that if any of the following conditions is met, the index structure adjustment operation is triggered:

[0022] If the number of inserted keys in the current segment index is greater than or equal to twice the number of keys in the last adjusted segment index, the index structure adjustment operation is triggered;

[0023] If the ratio between the number of conflicts and the number of inserted segments is greater than or equal to the preset ratio value, the index structure adjustment operation is triggered;

[0024] If the total weight of the inserted keys in the current segment index is greater than or equal to 2 times the total weight of the segment in the last adjustment, the index structure adjustment operation is triggered;

[0025] If the ratio of the total weight of the conflicting keys to the total weight of the inserted keys in the current segment index is greater than or equal to 0.1, the index structure adjustment operation is triggered.

[0026] In a preferred example of the present application, it can be further configured that the step of checking whether the insertion key triggers the index structure adjustment operation according to the preset condition includes:

[0027] If the insertion segment contains a key smaller than the first value or a key larger than the second value, the index structure adjustment operation is not triggered.

[0028] In a preferred example of the present application, it can be further configured that if the adjustment operation is triggered, the step of reconstructing a new index structure includes:

[0029] Calculate the weight of each key and use the weighted minimum collision degree to calculate the target position for placing the key in the index structure;

[0030] If a key is inserted at the target location, the relevant data will be inserted at the target location and the node type of the key will be changed to a data node;

[0031] If multiple keys are inserted at the target location, a new subtree is recursively constructed until no conflict occurs, and the node type of the key is changed to a pointer node, with the pointer pointing to the new segment.

[0032] In a preferred example of the present application, it can be further configured that the step of locating the target position of the insert key through the query operation includes:

[0033] Through range query, first locate the start key and end key of the preset range, scan the index structure in a monotonically increasing manner, and determine the target position for inserting the key.

[0034] In a second aspect, the present application provides an intelligent indexing device based on query perception.

[0035] This application is achieved through the following technical solutions:

[0036] An intelligent indexing device based on query perception, comprising:

[0037] A weight calculation module is used to obtain the user's query workload and determine the weight of each key in the query workload through a probability distribution model;

[0038] An index structure building module, configured to build an index structure by using the weights of the keys and a weighted minimum collision degree method;

[0039] An adjustment module is used to locate a target position of an insertion key through a query operation, and to check whether the insertion key triggers an adjustment operation of the index structure according to a preset condition;

[0040] The reconstruction module is used to rebuild a new index structure if an adjustment operation is triggered.

[0041] In a preferred example of the present application, it can be further configured that the device further includes:

[0042] A workload monitoring module, configured to use a probability distribution model to fit the workload distribution of two adjacent windows in the query workload, and use Kullback-Leibler divergence to determine the deviation of the workload distribution of the two adjacent windows;

[0043] If the deviation is less than the preset deviation value, the offset variable is set to 0;

[0044] If the deviation is greater than or equal to the preset deviation value, the offset variable is increased by 1.

[0045] In summary, compared with the prior art, the technical solution provided by the embodiment of the present application brings about at least the following beneficial effects: obtaining the user's query workload, determining the weight of each key in the query workload through a probability distribution model; using the weight of each key to construct an index structure through a weighted minimum collision degree method; locating the target position of the inserted key through a query operation, and checking whether the inserted key will trigger an adjustment operation of the index structure according to preset conditions; if the adjustment operation is triggered, rebuilding a new index structure. Different weights are assigned to query keys, and different priorities are indexed according to the priority of different keys, effectively improving index performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 A schematic diagram of an index structure provided for an exemplary embodiment of the present application;

[0047] Figure 2 A flowchart of an intelligent indexing method based on query perception provided by an exemplary embodiment of the present application;

[0048] Figure 3 A flowchart of a method for monitoring data set distribution deviation provided by an exemplary embodiment of the present application;

[0049] Figure 4 A schematic structural diagram of a query-aware intelligent indexing device provided as an exemplary embodiment of the present application. DETAILED DESCRIPTION

[0050] This specific embodiment is merely an explanation of the present application and is not a limitation of the present application. After reading this specification, those skilled in the art may make non-creative modifications to the present embodiment as needed, but as long as they are within the scope of the claims of the present application, they are protected by the patent law.

[0051] To make the purpose, technical solutions, and advantages of the embodiments of this application more clear, the technical solutions in the embodiments of this application will be clearly and completely described below in conjunction with the drawings in the embodiments of this application. Obviously, the described embodiments are part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0052] In addition, the term "and / or" in this application is simply a description of the association relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent three situations: A exists alone, A and B exist simultaneously, and B exists alone. In addition, the character " / " in this application, unless otherwise specified, generally indicates that the related objects are in an "or" relationship.

[0053] In this application, the terms "first", "second", etc. are used to distinguish identical or similar items with substantially the same effects and functions. It should be understood that there is no logical or temporal dependency between "first", "second", and "nth", nor is there any limitation on the quantity and execution order.

[0054] The embodiments of the present application are described in further detail below with reference to the accompanying drawings.

[0055] First, the index structure of WALDEN is introduced, such as Figure 1 As shown in Figure 2, the WALDEN structure is a model of a hierarchy of segments. WALDEN is a sorted index where, in each node, the items in the array are sorted by key, i.e., the keys on the left side of the array are smaller than the keys on the right side, whether directly in the node or in a subtree. This property is achieved by requiring all models in all nodes to be monotonically increasing.

[0056] A segment consists of an array of nodes N storing data or pointers to child nodes, a bit vector recording the node contents, and a model M predicting the node where a key resides. A key feature of WALDEN is that the model can predict the exact node for a key, eliminating the need for an exhaustive search around the prediction. A segment contains three types of nodes: null nodes, data nodes, and pointer nodes. Null nodes store nothing and represent empty space in memory. All nodes are initialized as null nodes but can be converted to data nodes or pointer nodes under certain operations. When data is written to a null node, it is converted to a data node. A data node stores a key and a corresponding payload (or a pointer to a payload). A pointer node points to the next segment. When multiple keys are inserted into the same data node, it becomes a pointer node and stores a pointer to the newly generated segment, which contains the previously conflicting keys. All three node types are 16 bytes long. A data entry consists of an 8-byte key value and an 8-byte payload or a pointer to a payload, while a node entry contains pointers to child nodes. A bit vector specifies the type of each data item using two bits. For the ith entry, the 2th bit indicates whether the entry is empty, and the (2i+1)th bit indicates the entry type, i.e., data or node. Because the bit vector is a lightweight structure, the node size can be limited by a predefined hyperparameter (e.g., 16MB). Unlike existing learned index structures, WALDEN does not distinguish between leaf nodes (DataNodes) and non-leaf nodes (InnerNodes). Instead, all nodes are treated equally, and the node type is used to guide the indexing operation to select different operations.

[0057] Set a key value as input, starting from the top down, and the model M outputs the node in the next layer, that is, the order from the root node to the leaf node of each layer, until it finally reaches the node that stores the key value. Model M is designed to be monotonically increasing, that is, given two keys ki and kj satisfying ki≤kj, then M(ki)≤M(kj). Given a key k and a list of nodes of length L, the model M can calculate the position of key k in the list according to the following formula:

[0058]

[0059] Where F represents a monotonically increasing kernel function, which represents a linear function F using only parameters A and b. Given a segment containing L nodes, the learned model M(key) and the set of keys K, the conflict degree T of the segment is M It can be expressed as:

[0060]

[0061] In one embodiment of the present application, a query-aware intelligent indexing method is provided, such as Figure 2As shown, the main steps are described as follows:

[0062] S10: Obtain the user's query workload and determine the weight of each key in the query workload using a probability distribution model. Specifically, given the query workload, the probability distribution model is used to fit the access ratio of each key in the query workload; the weight of each key is calculated based on the access ratio of each key.

[0063] It's important to note that in actual retrieval, not only does data (keys) follow a certain distribution, but requests for data (keys) may also have preferences and follow certain patterns. By collecting a user's historical query records, we can determine their query workload. Specifically, we define query workload Q as the user's request records to the database, which contain the query frequency ci for each key ki.

[0064] For the purpose of smoothing, this application does not directly use the query frequency ci of key ki, but instead fits the normalized query workload Q through the probability distribution model G. Given the query workload Q, the probability distribution model G is trained to fit the access ratio of the key: Where G can be any suitable probability distribution model. In some embodiments, the probability distribution model G can be a Gaussian mixture model (GMM). According to the learned probability distribution model G of the query workload Q, the weight of the key ki is defined as: i =1+λ·G(k i ), where λ is an adjustable scaling parameter and 1 is added for smoothing. This smoothing correction can effectively avoid the problem that the weights of some keys tend to 0 due to insufficient training samples.

[0065] S20: Using the weight of each key, an index structure is constructed using a weighted minimum collision degree method.

[0066] The key weight describes the priority of the key. Using this information for indexing can better serve the workload.

[0067] In some embodiments, a weighted minimum collision degree method (WMCD) is proposed to utilize key weights to construct an index structure. The WMCD method linearly traverses the keys ki to reduce the time complexity and obtains an approximate collision degree C and a corresponding model M.

[0068] Specifically, set the initial value of the collision degree C to 1 and initialize x, y, U C :

[0069]

[0070] Starting from k0, determine whether the keys in the key set K satisfy F(k i +L(i))-F(k i )≥U, if satisfied, move to the next key, update L(i), and repeat the verification; if not satisfied, increase the collision degree C by 1, update x, y, U C The value of ki is returned and the verification loop starts from the current ki until all keys in the range [0, N-1-y] are checked, and the model M of the index structure and the expected value of the collision degree C are obtained. Repeated escape starting from i=0 can save time and improve efficiency.

[0071] Among them, w i represents the weight of key ki, N represents the total number of keys, k i represents the i-th key value, k x Represents the xth key value, and L represents the length of the node list.

[0072] S30: Locate the target position of the inserted key through a query operation, and check whether the inserted key will trigger an adjustment operation of the index structure according to preset conditions.

[0073] It should be noted that the query operation starts from the root segment of the index, uses the model of the current segment and the search key to calculate the position in the node list and obtain the corresponding node. If the current node is a pointer node, follow the pointer recursively to locate the next end and node until it reaches a node of another type. If the current node type is a data node, it is necessary to check whether the key in the current node is consistent with the search key, rather than directly returning the current node, because different keys can be mapped to the same position. Once the data node is located, the node being searched can be confirmed without further search operations, because the index structure involved in this application will return the exact position of the key. Therefore, the query operation only consumes O(h) for the index tree height h. If the current node type is an empty node, that is, the key is not stored in the index structure, FLASE and the current empty node will be returned.

[0074] In some embodiments, the present application performs query operations through range queries to find a list of keys that meet a certain range. Specifically, given a range [a, b], the query operation is first used to locate the positions of the start key a and the end key b. Since the keys are stored in a monotonically increasing manner, a simple forward scan can be performed until the end key b is found. Since different data nodes are not linked, segments and pointer nodes, that is, index trees, can be traversed to scan nodes. At the same time, in order to further improve performance, bit vectors can be used to skip gaps, that is, empty nodes. In addition, the position of the end key b is first located, and then the index tree is traversed until the end key b is reached, in order to avoid unnecessary comparisons between the current key and the end key b.

[0075] The insert operation inserts a key at the node location predicted by the model to maintain index consistency. Insertion is also implemented based on query operations. Specifically, the query operation locates the exact node location for the insertion. If the node type is a null node, simply change the node type to a data node and write the insert key and payload. If the node type is a data node, it means a collision has occurred, and a new segment must be trained, containing the key from the data node and the insertion key. The node type is then changed to a pointer node and a pointer to the new segment is stored.

[0076] To determine whether a segment needs to be adjusted, the segment's statistics are needed to analyze its current and historical status. This statistical information includes the number of keys, the total key weight, the number of conflicting keys, and the total conflicting key weight. build_num represents the number of keys when the segment was created, key_num represents the number of keys in the current segment, build_weit represents the total key weight when the segment was created, key_weit represents the total key weight in the current segment, collision_num represents the number of conflicting keys, meaning two keys are mapped to the same location, and collision_weit is the total conflicting key weight.

[0077] During the insertion process, as more and more keys are inserted into the index structure, the index structure will become larger and deeper, and the index structure needs to be adjusted. However, to avoid the extra overhead caused by frequent adjustments, it is necessary to find appropriate times during the insertion process to adjust and rebuild the index structure.

[0078] In some embodiments, if the number of inserted keys of the current segment index is greater than or equal to 2 times the number of keys of the last adjusted segment index, the adjustment operation of the index structure is triggered, that is, s.key_num≥β·s.build_num, where β is set to 2 by default.

[0079] If the ratio of the number of conflicts to the number of inserted segments is greater than or equal to a preset ratio, the index structure adjustment operation is triggered, s.collision_num ≥ α (s.key_num - s.build_num). The preset ratio value α can be set to 0.1. This can constrain frequent conflicts and avoid the increase in index tree depth and performance degradation caused by new nodes generated due to conflicts.

[0080] If the total weight of the inserted keys in the current segment index is greater than or equal to twice the total weight of the segment in the last adjustment, the index structure adjustment operation is triggered; that is, s.key_weit ≥ β·s.build_weit, where β can be 2.

[0081] If the ratio of the total weight of the conflicting keys to the total weight of the inserted keys in the current segment index is greater than or equal to 0.1, the index structure adjustment operation is triggered, that is, s.collision_weit ≥ α·(s.key_weit - s.build_weit), where α can be 0.1.

[0082] In some embodiments, if the inserted segment contains a key smaller than the first value or a key larger than the second value, the adjustment operation of the index structure is not triggered. It should be noted that the size of the index will also affect the effect of the adjustment. If the index subtree contains very few keys, then inserting into it will frequently trigger the adjustment operation, which will result in unnecessary overhead. Similarly, the index subtree contains many keys. In extreme cases, when the root segment is adjusted, all keys will be retrained, and the cost may be unacceptable. In some embodiments, the first value can be 64, and the second value can be 1M (i.e., 1000000).

[0083] S40: If an adjustment operation is triggered, a new index structure is rebuilt.

[0084] In some embodiments, a weight is calculated for each key, and a target position for placing the key in the index structure is calculated using a weighted minimum collision degree;

[0085] If a key is inserted at the target location, the relevant data will be inserted at the target location and the node type of the key will be changed to a data node;

[0086] If multiple keys are inserted at the target location, a new subtree is recursively constructed until no conflict occurs, and the node type of the key is changed to a pointer node, with the pointer pointing to the new segment.

[0087] The adjustment process mainly replaces the divergent subtree structure with a newly constructed, compact, and optimal tree structure. The new subtree has the lowest collision degree, thereby effectively reducing the height of the tree. In some embodiments, the RebuildSubtree algorithm is used to rebuild the index structure. The RebuildSubtree algorithm first initializes segment s with L nodes, where the size of L is δ times the number of keys in the key set, which can ensure that there are enough empty nodes to absorb future insertions. At the same time, L has a predefined upper bound (1M) to limit the size of the tree.

[0088] In some embodiments, a probability distribution model is also used to fit the workload distribution of two adjacent windows in the query workload, and the Kullback-Leibler divergence is used to determine the deviation of the workload distribution of the two adjacent windows; if the deviation is less than the preset deviation value, the offset variable is set to 0; if the deviation is greater than or equal to the preset deviation value, the offset variable is increased by 1.

[0089] The index structure is built based on the recent query workload and the main keys are prioritized. However, as time goes by, the data workload may change. The shift of query workload will cause the performance of the originally built index structure to degrade because it deviates from the current workload. The shifted workload may bring many operations to keys that are rarely accessed in the workload of building the tree and are placed at a deeper level, resulting in long query overhead. You can monitor whether the workload has changed significantly from the old workload. Specifically, Figure 3 The black long box shown represents the query workload, and the dotted rectangle is a sliding window. Each sliding window has the same length, S = 6M, and the window is obtained by moving the step size τ, which is 10% of the workload, for example, 20M. The keys collected from the query workload in each window represent the workload distribution at that time, for example Figure 3 Window K and window K′ in the graph are fitted using a probability distribution model, and the Kullback-Leibler divergence is used to determine the deviation between window K and window K′ to calculate whether a shift has occurred. If the deviation kl between window K and window K′ is less than 1, it means that the new workload distribution is similar to the original workload distribution, and the condition for distribution shift does not occur. In this case, the shift variable (shift_mark) is set to 0; if the deviation kl between window K and window K′ is greater than or equal to 1, it means that the new workload distribution is different from the original workload distribution, and the condition for distribution shift occurs. In this case, the shift variable (shift_mark) is set to increase by 1. The current workload is updated to a weighted combination of G and G′. When changes continue, that is, when the workloads of the most recent consecutive windows are significantly different from the original window, the tree reconstruction procedure is triggered. This is modeled by marking whether the shift variable (shift_mark) is greater than a specific threshold η (for example, η=3), and then the tree reconstruction process is called. During reconstruction, the new workload distribution Gs is learned using the key Ks from the most recent window, and then rebuilt by calling the RebuildSubtree function, and finally resetting the shift_mark variable to 0.

[0090] In some embodiments, a delete operation can be performed by locating an index segment and changing the corresponding bitmap type to an empty node, which then updates the parameter value of the monitoring tree index state. An update operation can be performed by combining a lookup operation with a payload update, or by combining a delete operation with an insert operation, depending on whether the key value or the payload is being updated. Bulk load operations can also be performed.

[0091] This application is different from traditional methods that only focus on modifying the structure of the smart index itself. After introducing a query-aware method, we designed the corresponding index structure and algorithm to improve the overall performance of the smart index and at the same time cope with the situation where the data set distribution is offset.

[0092] This application also provides an intelligent indexing device based on query perception, such as Figure 4 As shown, the device includes: a weight calculation module 01, which is used to obtain the user's query workload and determine the weight of each key in the query workload through a probability distribution model;

[0093] Index structure construction module 02, used to construct the index structure by using the weight of each key through the weighted minimum collision degree method;

[0094] Adjustment module 03, used to locate the target position of the inserted key through query operation, and check whether the inserted key will trigger the adjustment operation of the index structure according to preset conditions;

[0095] The reconstruction module 04 is used to rebuild a new index structure if an adjustment operation is triggered.

[0096] In some embodiments, the apparatus further comprises: a workload monitoring module configured to use a probability distribution model to fit the workload distribution of two adjacent windows in the query workload, and to use Kullback-Leibler divergence to determine a deviation of the workload distribution of the two adjacent windows;

[0097] If the deviation is less than the preset deviation value, the offset variable is set to 0;

[0098] If the deviation is greater than or equal to the preset deviation value, the offset variable is increased by 1.

[0099] Those skilled in the art will appreciate that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink), DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0100] Those skilled in the art will clearly understand that for the sake of convenience and conciseness of description, the division of the above-mentioned functional units and modules is only used as an example. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the device described in this application is divided into different functional units or modules to complete all or part of the functions described above.

Claims

1. An intelligent indexing method based on query perception, characterized in that: The method comprises: Obtaining a user's query workload, and determining the weight of each key in the query workload using a probability distribution model; The weights of the keys are used to construct an index structure using a weighted minimum collision method, specifically including: Set the initial value of collision degree C to 1 and initialize x, y, U C : Starting from k0, determine whether the keys in the key set K satisfy F(k i +L(i))-F(k i )≥U, if it is satisfied, move to the next key, update L(i), and repeat the verification; if it is not satisfied, increase the collision degree C by 1 and update x, y, U C The value of ki is returned, and the verification loop starts from the current ki until all keys are checked, and the expected value of the index structure model M and the collision degree C is obtained; Among them, w i represents the weight of key ki, N represents the total number of keys, k i represents the i-th key value, k x Indicates the xth key value, L indicates the length of the node list; The target position of the insertion key is located by querying the operation, and whether the insertion key triggers the adjustment operation of the index structure is checked according to preset conditions; wherein the adjustment operation of the index structure is triggered if any of the following conditions is met: If the number of inserted keys in the current segment index is greater than or equal to 2 times the number of keys in the segment index during the last adjustment, the index structure adjustment operation is triggered; if the ratio between the number of conflicts and the number of inserted segments is greater than or equal to the preset ratio value, the index structure adjustment operation is triggered; if the total weight of the inserted keys in the current segment index is greater than or equal to 2 times the total weight of the segment during the last adjustment, the index structure adjustment operation is triggered; if the ratio of the total weight of the conflicting keys to the total weight of the inserted keys in the current segment index is greater than or equal to 0.1, the index structure adjustment operation is triggered; If an adjustment operation is triggered, a new index structure will be rebuilt, including: Calculate the weight of each key and use the weighted minimum collision degree to calculate the target position for placing the key in the index structure; if a key is inserted at the target position, insert the relevant data at the target position and change the key's node type to a data node; if multiple keys are inserted at the target position, recursively construct a new subtree until no conflict occurs, and change the key's node type to a pointer node, with the pointer pointing to the new segment.

2. The query-aware intelligent indexing method according to claim 1, characterized in that: Also includes: Using a probability distribution model to fit the workload distribution of two adjacent windows in the query workload, and using Kullback-Leibler divergence to determine the deviation of the workload distribution of the two adjacent windows; If the deviation is less than the preset deviation value, the offset variable is set to 0; If the deviation is greater than or equal to the preset deviation value, the offset variable is increased by 1.

3. The query-aware intelligent indexing method according to claim 1, characterized in that: The steps of obtaining a user's query workload and determining the weight of each key in the query workload using a probability distribution model include: Given a query workload, a probability distribution model is used to fit the access ratio of each key in the query workload; The weight of each key is calculated based on the access ratio of each key.

4. The query-aware intelligent indexing method according to claim 1, characterized in that: The step of checking whether the inserted key triggers the index structure adjustment operation according to the preset conditions includes: If the insertion segment contains a key smaller than the first value or a key larger than the second value, the index structure adjustment operation is not triggered.

5. The query-aware intelligent indexing method according to claim 1, characterized in that: The step of locating the target position of the insert key by querying the operation comprises: Through range query, first locate the start key and end key of the preset range, scan the index structure in a monotonically increasing manner, and determine the target position for inserting the key.

6. An intelligent indexing device based on query perception, characterized in that: The device is configured to execute the query-aware intelligent indexing method according to any one of claims 1 to 5, the device comprising: A weight calculation module is used to obtain the user's query workload and determine the weight of each key in the query workload through a probability distribution model; An index structure building module, configured to build an index structure by using the weights of the keys and a weighted minimum collision degree method; An adjustment module, configured to locate a target position of an insertion key through a query operation, and check whether the insertion key triggers an adjustment operation of the index structure according to a preset condition; The reconstruction module is used to rebuild a new index structure if an adjustment operation is triggered.

7. The intelligent indexing device based on query perception according to claim 6, characterized in that: The device further comprises: A workload monitoring module, configured to use a probability distribution model to fit the workload distribution of two adjacent windows in the query workload, and use Kullback-Leibler divergence to determine the deviation of the workload distribution of the two adjacent windows; If the deviation is less than the preset deviation value, the offset variable is set to 0; If the deviation is greater than or equal to the preset deviation value, the offset variable is increased by 1.

Images