Lock-free learning type indexing method and device and electronic equipment

Abstract

According to the lock-free learning type indexing method and device and the electronic equipment provided by the invention, the to-be-indexed data is subjected to grouping space division processing to obtain the plurality of leaf node data, and the model training processing is performed according to the leaf node data to obtain the first model; performing memory pre-allocation processing in the first model to obtain a storage space, and storing a corresponding lockless chain table in the storage space; determining a residual space corresponding to the storage space, performing recombination optimization processing on the leaf node data corresponding to the storage space with the residual space smaller than a preset space threshold to obtain an initial index structure, and connecting the initial index structure to a preset tree structure according to a lockless chain table to obtain a target index structure; and indexing the target element through the target index structure to obtain target position information of the target element in the to-be-indexed data. Index processing is carried out through the target index structure based on the preset tree structure, query and insertion updating under multiple threads can be supported, and efficient multi-thread loads are supported.

Description

Technical Field

[0001] The present disclosure relates to the field of data processing technology, and in particular to a lock-free learning indexing method and device, and an electronic device. Background Art

[0002] Learning indexing technology combines artificial intelligence technology with traditional databases, uses machine learning models to replace traditional index structures, and learns the distribution patterns of underlying data by analyzing the distribution characteristics of business data or the patterns of workloads. It establishes a mapping relationship between key values ​​and positions, making traditional database systems intelligent and becoming a research hotspot in the database field. The key idea of ​​learning indexing is to use a learning model to approximate the index, use the recorded key values ​​and their positions to train the model, and then use the trained model to predict the position of the specified key value. Learning indexing effectively reduces storage overhead and improves query efficiency by replacing tree traversal operations with mathematical calculations.

[0003] The learning index mainly focuses on optimizing query performance. In terms of insertion and update, there are two common methods, in-place and delta-buffer. Both in-place array insertion and delta-buffer writing are processed in single-threaded scenarios.

[0004] Therefore, how to solve the defect that the learning index only supports query and insertion and update under single thread and cannot support efficient multi-thread load is a problem that needs to be solved urgently. Summary of the invention

[0005] The present disclosure provides a lock-free learning indexing method, device, and electronic device, which is mainly intended to solve the problem that the learning index only supports query and insert update under a single thread and cannot support efficient multi-thread load.

[0006] According to a first aspect of the present disclosure, a lock-free learning indexing method is provided, comprising:

[0007] The data to be indexed is processed by grouping and spatial division to obtain a plurality of leaf node data, and model training is performed according to each of the leaf node data to obtain a first model corresponding to each of the leaf node data; wherein each of the leaf node data at least includes a plurality of elements to be indexed and position information corresponding to each of the elements to be indexed;

[0008] Perform memory pre-allocation processing in each of the first models to obtain storage space corresponding to each of the first models, and store a corresponding lock-free linked list in the storage space; wherein the lock-free linked list is a single-chain lock-free linked list formed according to the incremental modification of the corresponding leaf node data;

[0009] Determine the remaining space corresponding to each of the storage spaces, reorganize and optimize the leaf node data corresponding to the storage space whose remaining space is less than a preset space threshold to obtain an initial index structure, and connect the initial index structure to a preset tree structure according to the lock-free linked list to obtain a target index structure;

[0010] The target element is indexed by the target index structure to obtain target position information of the target element in the data to be indexed, wherein the target element is any element in the elements to be indexed.

[0011] Optionally, the grouping and spatial division of the data to be indexed to obtain a plurality of leaf node data includes:

[0012] Perform coordinate point construction processing according to each of the elements to be indexed in the data to be indexed and the position information corresponding to each of the elements to be indexed, to obtain the target data point corresponding to each of the elements to be indexed;

[0013] Determine a first error upper boundary and a first error lower boundary of a first data point according to a preset error, and determine an error range according to the first error upper boundary and the first error lower boundary to obtain a first error region corresponding to the first data point; wherein the first data point is any target data point;

[0014] Determine a second error upper boundary and a second error lower boundary of a second data point according to the preset error, and determine an error range according to the second error upper boundary and the second error lower boundary to obtain a second error region corresponding to the second data point; wherein the second data point is the next adjacent target data point of the first data point;

[0015] In the case where there is an overlap between the first error region and the second error region, dividing the to-be-indexed element corresponding to the first data point and the to-be-indexed element corresponding to the second data point into first leaf node data;

[0016] In the case where the first error region and the second error region do not overlap, the to-be-indexed element corresponding to the first data point is divided into first leaf node data, the to-be-indexed element corresponding to the second data point is divided into second leaf node data, and the second data point is used as the starting data point of the second leaf node data;

[0017] Repeat the above steps until all the data to be indexed are grouped to obtain the multiple leaf node data.

[0018] Optionally, performing model training processing according to each leaf node data to obtain a first model corresponding to each leaf node data includes:

[0019] Performing position prediction processing on the elements to be indexed by using a preset linear model to obtain predicted position information corresponding to each of the elements to be indexed;

[0020] Perform error analysis according to the position information and the predicted position information to obtain the prediction error corresponding to each of the elements to be indexed;

[0021] The preset linear model is optimized according to the prediction error, the element to be indexed and the position information to obtain the first model corresponding to each leaf node data.

[0022] Optionally, the step of reorganizing and optimizing the leaf node data corresponding to the storage space whose remaining space is less than a preset space threshold to obtain an initial index structure includes:

[0023] Processing the lock-free linked list corresponding to the target leaf node data to obtain a first data set and a second data set; wherein the target leaf node data is the leaf node data corresponding to the storage space whose remaining space is less than the preset space threshold;

[0024] Constructing a lock-free self-organizing linked list and a lock-free B+ tree in a target buffer corresponding to the target leaf node data according to the first data set; wherein the first data set includes existing data in the lock-free linked list after the insertion and / or deletion operation is performed, and the second data set includes data deleted from the lock-free linked list after the insertion and / or deletion operation is performed;

[0025] The target leaf node data is subjected to point query reorganization processing through the lock-free self-organizing linked list to obtain a first index structure, and the first index structure is subjected to range query reorganization processing through the lock-free B+ tree to obtain the initial index structure.

[0026] Optionally, performing point query reorganization processing on the target leaf node data through the lock-free self-organizing linked list to obtain a first index structure includes:

[0027] Performing window division processing in the lock-free self-organizing linked list to obtain a first window, and performing region division processing on the first window according to a preset ratio to obtain a first region and a second region;

[0028] storing the proximity data in the first area; wherein the proximity data is the most recently accessed data for accessing the target leaf node data;

[0029] Monitor the data storage records in the first window to obtain monitoring results, and when it is determined according to the monitoring results that there are additional records in the data storage records, perform numerical calculation processing on the frequency data, and store the processed frequency data in the second area; wherein the frequency data is the access frequency of accessing the target leaf node data;

[0030] The first index structure at least includes the first area and the second area.

[0031] Optionally, performing range query reorganization processing on the first index structure by using the lock-free B+ tree to obtain the initial index structure includes:

[0032] Initializing the inserted element to obtain a first insertion range, wherein performing range query reorganization processing on the first index result at least includes performing insertion sort processing on the first index structure, the inserted element is an element that performs insertion sort processing on the first index structure, and the first offset range is an offset search range corresponding to the inserted element;

[0033] Traversing the lock-free linked list, comparing the inserted element with the first element to be compared, and obtaining a comparison result; wherein the first element to be compared is the first element to be compared traversed from the lock-free linked list, and the element to be compared is an element of the insertion and / or deletion type in the lock-free linked list;

[0034] When it is determined according to the comparison result that the inserted element is identical to the first element to be compared, the first offset range is converged according to the first range to be compared to obtain a second offset range; wherein the first range to be compared is an offset search range corresponding to the first element to be compared;

[0035] When it is determined according to the comparison result that the inserted element is larger than the first element to be compared, and the range start value of the first offset range is smaller than the range start value of the first range to be compared, setting the range start value of the first offset range to the range start value of the first range to be compared, to obtain the second offset range;

[0036] When it is determined according to the comparison result that the inserted element is smaller than the first element to be compared, and the range end value of the first offset range is smaller than the range end value of the first range to be compared, setting the range end value of the first offset range to the range end value of the first range to be compared, to obtain the second offset range;

[0037] Until the comparison between the inserted element and all the elements to be compared is completed, the target offset range corresponding to the inserted element is obtained, and the inserted element is inserted into the target offset range to obtain the initial index structure.

[0038] Optionally, the indexing process is performed on the target element by using the target index structure to obtain the target position information of the target element in the data to be indexed, including:

[0039] Searching and processing the target element in the target index structure to determine leaf node data to be indexed; wherein the leaf node data to be indexed is the leaf node data containing the target element in the target index structure;

[0040] Traversing the to-be-indexed lock-free linked list to determine the element type of the target element; wherein the to-be-indexed lock-free linked list is the lock-free linked list contained in the to-be-indexed leaf node data;

[0041] In the case where it is determined that the target element is a non-unique element according to the element type, data analysis processing is performed on the lock-free linked list to be indexed to obtain a third data set and a fourth data set; wherein the third data set includes existing data of the lock-free linked list to be indexed after the insertion and / or deletion operations are performed, and the fourth data set includes data deleted from the lock-free linked list to be indexed after the insertion and / or deletion operations are performed;

[0042] Performing data analysis on the target element according to the third data set and the fourth data set to obtain a unique key value corresponding to the target element;

[0043] The unique key value is input into the first model to be indexed for position prediction processing to obtain the target position information, wherein the first model to be indexed is the first model corresponding to the leaf node data to be indexed.

[0044] Optionally, after traversing the lock-free linked list to be indexed to determine the element type of the target element, the method further includes:

[0045] When it is determined that the target element is a unique element according to the element type, the target element is input into the first model to be indexed for position prediction processing to obtain the target position information.

[0046] According to a second aspect of the present disclosure, a lock-free learning indexing device is provided, comprising:

[0047] A partitioning unit is used to perform grouping and spatial partitioning processing on the data to be indexed to obtain a plurality of leaf node data;

[0048] A training unit, used to perform model training processing according to each leaf node data to obtain a first model corresponding to each leaf node data; wherein each leaf node data includes at least a plurality of elements to be indexed and position information corresponding to each element to be indexed;

[0049] An allocation unit, configured to perform memory pre-allocation processing in each of the first models, obtain a storage space corresponding to each of the first models, and store a corresponding lock-free linked list in the storage space; wherein the lock-free linked list is a single-chain lock-free linked list formed according to the incremental modification of the corresponding leaf node data;

[0050] an optimization unit, configured to determine the remaining space corresponding to each of the storage spaces, reorganize and optimize the leaf node data corresponding to the storage spaces whose remaining space is less than a preset space threshold, obtain an initial index structure, and connect the initial index structure to a preset tree structure according to the lock-free linked list to obtain a target index structure;

[0051] An indexing unit is used to perform indexing processing on a target element through the target index structure to obtain target position information of the target element in the data to be indexed, wherein the target element is any element among the elements to be indexed.

[0052] Optionally, the division unit includes:

[0053] A construction module, used to perform coordinate point construction processing according to each of the elements to be indexed in the data to be indexed and the position information corresponding to each of the elements to be indexed, to obtain a target data point corresponding to each of the elements to be indexed;

[0054] a determination module, configured to determine a first error upper boundary and a first error lower boundary of a first data point according to a preset error, and determine an error range according to the first error upper boundary and the first error lower boundary to obtain a first error region corresponding to the first data point; wherein the first data point is any target data point;

[0055] The determination module is further configured to determine a second error upper boundary and a second error lower boundary of a second data point according to the preset error, and determine an error range according to the second error upper boundary and the second error lower boundary to obtain a second error region corresponding to the second data point; wherein the second data point is a next adjacent target data point of the first data point;

[0056] a partitioning module, configured to partition the to-be-indexed element corresponding to the first data point and the to-be-indexed element corresponding to the second data point into first leaf node data when there is an overlapped area between the first error area and the second error area;

[0057] The division module is further used for, when there is no overlap between the first error region and the second error region, dividing the to-be-indexed element corresponding to the first data point into first leaf node data, dividing the to-be-indexed element corresponding to the second data point into second leaf node data, and using the second data point as the starting data point of the second leaf node data;

[0058] Repeat the above steps until all the data to be indexed are grouped to obtain the multiple leaf node data.

[0059] Optionally, the training unit includes:

[0060] A prediction module, used to perform position prediction processing on the elements to be indexed by using a preset linear model to obtain predicted position information corresponding to each of the elements to be indexed;

[0061] An analysis module, used to perform error analysis based on the position information and the predicted position information to obtain the prediction error corresponding to each of the elements to be indexed;

[0062] An optimization module is used to perform model optimization processing on the preset linear model according to the prediction error, the element to be indexed and the position information to obtain the first model corresponding to each leaf node data.

[0063] Optionally, the optimization unit includes:

[0064] A processing module, used for processing the lock-free linked list corresponding to the target leaf node data to obtain a first data set and a second data set; wherein the target leaf node data is the leaf node data corresponding to the storage space whose remaining space is less than the preset space threshold;

[0065] A construction module, configured to construct a lock-free self-organizing linked list and a lock-free B+ tree in a target buffer corresponding to the target leaf node data according to the first data set; wherein the first data set includes existing data of the lock-free linked list after the insertion and / or deletion operation is performed, and the second data set includes data deleted from the lock-free linked list after the insertion and / or deletion operation is performed;

[0066] The reorganization module is used to perform point query reorganization processing on the target leaf node data through the lock-free self-organizing linked list to obtain a first index structure, and perform range query reorganization processing on the first index structure through the lock-free B+ tree to obtain the initial index structure.

[0067] Optionally, the recombination module is also used for:

[0068] Performing window division processing in the lock-free self-organizing linked list to obtain a first window, and performing region division processing on the first window according to a preset ratio to obtain a first region and a second region;

[0069] storing the proximity data in the first area; wherein the proximity data is the most recently accessed data for accessing the target leaf node data;

[0070] Monitor the data storage records in the first window to obtain monitoring results, and when it is determined according to the monitoring results that there are additional records in the data storage records, perform numerical calculation processing on the frequency data, and store the processed frequency data in the second area; wherein the frequency data is the access frequency of accessing the target leaf node data;

[0071] The first index structure at least includes the first area and the second area.

[0072] Optionally, the recombination module is also used for:

[0073] Initializing the inserted element to obtain a first insertion range, wherein performing range query reorganization processing on the first index result at least includes performing insertion sort processing on the first index structure, the inserted element is an element that performs insertion sort processing on the first index structure, and the first offset range is an offset search range corresponding to the inserted element;

[0074] Traversing the lock-free linked list, comparing the inserted element with the first element to be compared, and obtaining a comparison result; wherein the first element to be compared is the first element to be compared traversed from the lock-free linked list, and the element to be compared is an element of the insertion and / or deletion type in the lock-free linked list;

[0075] When it is determined according to the comparison result that the inserted element is identical to the first element to be compared, the first offset range is converged according to the first range to be compared to obtain a second offset range; wherein the first range to be compared is an offset search range corresponding to the first element to be compared;

[0076] When it is determined according to the comparison result that the inserted element is larger than the first element to be compared, and the range start value of the first offset range is smaller than the range start value of the first range to be compared, setting the range start value of the first offset range to the range start value of the first range to be compared, to obtain the second offset range;

[0077] When it is determined according to the comparison result that the inserted element is smaller than the first element to be compared, and the range end value of the first offset range is smaller than the range end value of the first range to be compared, setting the range end value of the first offset range to the range end value of the first range to be compared, to obtain the second offset range;

[0078] Until the comparison between the inserted element and all the elements to be compared is completed, the target offset range corresponding to the inserted element is obtained, and the inserted element is inserted into the target offset range to obtain the initial index structure.

[0079] Optionally, the index unit includes:

[0080] A search module, used to search the target element in the target index structure to determine the leaf node data to be indexed; wherein the leaf node data to be indexed is the leaf node data containing the target element in the target index structure;

[0081] A traversal module, used for traversing the lock-free linked list to be indexed to determine the element type of the target element; wherein the lock-free linked list to be indexed is the lock-free linked list contained in the leaf node data to be indexed;

[0082] An analysis module is used for, when it is determined that the target element is a non-unique element according to the element type, performing data analysis on the to-be-indexed lock-free linked list to obtain a third data set and a fourth data set; wherein the third data set includes existing data of the to-be-indexed lock-free linked list after the insertion and / or deletion operations are performed, and the fourth data set includes data deleted from the to-be-indexed lock-free linked list after the insertion and / or deletion operations are performed;

[0083] The analysis module is further used to perform data analysis on the target element according to the third data set and the fourth data set to obtain a unique key value corresponding to the target element;

[0084] A prediction module is used to input the unique key value into the first model to be indexed for position prediction processing to obtain the target position information, wherein the first model to be indexed is the first model corresponding to the leaf node data to be indexed.

[0085] Optionally, the indexing unit is further used to, when it is determined that the target element is a unique element according to the element type, input the target element into the first model to be indexed for position prediction processing to obtain the target position information.

[0086] According to a third aspect of the present disclosure, there is provided an electronic device, including:

[0087] at least one processor; and

[0088] a memory communicatively connected to the at least one processor; wherein,

[0089] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method described in the first aspect.

[0090] According to a fourth aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable the computer to execute the method described in the first aspect.

[0091] According to a fifth aspect of the present disclosure, a computer program product is provided, comprising a computer program, wherein when the computer program is executed by a processor, the computer program implements the method as described in the first aspect above.

[0092] The lock-free learning indexing method, device and electronic device provided by the present disclosure divide the data to be indexed into groups and spaces to obtain multiple leaf node data, and perform model training processing according to each leaf node data to obtain a first model corresponding to each leaf node data; wherein each leaf node data at least includes multiple elements to be indexed and position information corresponding to each element to be indexed; memory pre-allocation processing is performed in each first model to obtain a storage space corresponding to each first model, and a corresponding lock-free linked list is stored in the storage space; wherein the lock-free linked list is a single-link lock-free linked list composed of incremental modifications of the corresponding leaf node data; the remaining space corresponding to each storage space is determined, and the leaf node data corresponding to the storage space whose remaining space is less than a preset space threshold is reorganized and optimized to obtain an initial index structure, and the initial index structure is connected to a preset tree structure according to the lock-free linked list to obtain a target index structure; the target element is indexed by the target index structure to obtain the target position information of the target element in the data to be indexed, wherein the target element is any element among the elements to be indexed. Compared with the related art, the embodiment of the present disclosure divides the data to be indexed into different underlying data groups, namely leaf node data, each group corresponds to a linear model prediction, namely the first model, and adds a lock-free linked list inside the group as a multi-threaded lock-free structure. Finally, a target index structure based on a preset tree structure is constructed, and index processing is performed through the target index structure. It can perform multi-threaded high-concurrency access, support multi-threaded queries and insertion updates, and support efficient multi-threaded loads.

[0093] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present application, nor is it intended to limit the scope of the present application. Other features of the present application will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0094] The accompanying drawings are used to better understand the present solution and do not constitute a limitation of the present disclosure.

[0095] Figure 1 A flowchart of a lock-free learning indexing method provided by an embodiment of the present disclosure;

[0096] Figure 2 A schematic diagram of a first model corresponding to leaf node data provided by an embodiment of the present disclosure;

[0097] Figure 3 A property diagram of a lock-free linked list provided in an embodiment of the present disclosure;

[0098] Figure 4 A schematic diagram of a memory pre-allocation process provided by an embodiment of the present disclosure;

[0099] Figure 5 A schematic diagram of a target index structure provided by an embodiment of the present disclosure;

[0100] Figure 6 A schematic diagram of a process of grouping space division provided by an embodiment of the present disclosure;

[0101] Figure 7 A schematic diagram of the principle of grouping space division provided by an embodiment of the present disclosure;

[0102] Figure 8 A schematic diagram of an error of a first model provided by an embodiment of the present disclosure;

[0103] Fig. 9 A schematic diagram of a structure of adaptive data reorganization optimization provided by an embodiment of the present disclosure;

[0104] Fig.10 A schematic diagram of a point query reorganization process provided by an embodiment of the present disclosure;

[0105] Fig.11 A schematic diagram of the structure of a lock-free self-organizing linked list provided by an embodiment of the present disclosure;

[0106] Fig.12 A schematic diagram of the structure of another lock-free self-organizing linked list provided in an embodiment of the present disclosure;

[0107] Fig.13 A structural diagram of a lock-free B+ tree structure provided by an embodiment of the present disclosure;

[0108] Fig.14 A schematic diagram of the principle of element insertion provided by an embodiment of the present disclosure;

[0109] Fig.15 A schematic diagram of the principle of non-unique index processing provided by an embodiment of the present disclosure;

[0110] Fig.16 A structural diagram of a lock-free learning indexing device provided by an embodiment of the present disclosure;

[0111] Fig.17 A schematic diagram of the structure of another lock-free learning indexing device provided by an embodiment of the present disclosure;

[0112] Fig.18 A schematic block diagram of an electronic device provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0113] The following is a description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0114] The lock-free learning indexing method and device, and electronic device of the embodiments of the present disclosure are described below with reference to the accompanying drawings.

[0115] Figure 1 A flowchart of a lock-free learning indexing method provided in an embodiment of the present disclosure.

[0116] like Figure 1 As shown, the method comprises the following steps:

[0117] Step 101, group the data to be indexed and perform spatial division processing to obtain multiple leaf node data, and perform model training processing based on each leaf node data to obtain the first model corresponding to each leaf node data; wherein each leaf node data includes at least multiple elements to be indexed and position information corresponding to each element to be indexed.

[0118] The lock-free learning indexing method described in the embodiment of the present disclosure requires constructing a target index structure based on a tree structure to index the elements that need to be indexed (target elements), wherein the target index structure is divided into two parts, namely the upper tree structure and the lower leaf node grouping (the multiple leaf node data).

[0119] At the same time, each data to be indexed corresponds to a target index structure, that is, when the data to be indexed needs to be indexed, a corresponding target index structure needs to be established according to the data to be indexed, and then the target element is indexed in the data to be indexed based on the target index structure.

[0120] In the embodiment of the present disclosure, it is necessary to determine the lower leaf node groupings of the target index structure by dividing the data to be indexed, that is, each leaf node data represents each lower leaf node grouping of the target index structure. The data to be indexed is the entire data set that needs to be indexed, and the type includes but is not limited to: key value sequence, etc. Specifically, the embodiment of the present disclosure does not limit the type and content of the data to be indexed.

[0121] In order to facilitate the understanding of the lock-learning indexing method described in the present disclosure, the data to be indexed in the subsequent embodiments is described by taking a key value sequence as an example.

[0122] Among them, when performing the group space division processing, it is necessary to determine the error area corresponding to each element to be indexed in the data to be indexed, and determine the grouping of each element to be indexed in the data to be indexed (that is, the leaf node data where each element to be indexed is located) according to the error area. The error area is an area determined according to the preset error corresponding to each element to be indexed. The preset error of each element to be indexed is the maximum acceptable error between the actual position and the predicted position of this element to be indexed set by the user. Specifically, regarding the division process of the group space, it can be adopted but not limited to the following method: if the data to be indexed is a key value sequence k, each element to be indexed is each key value, the preset error is δ, and the data point is each key value and the corresponding actual position in the plane coordinate system. When the next data point pnext arrives, based on pstart (starting data point), pnext and δ, two inequalities are derived to determine the two boundaries of the first error area. Then read the next data point to form two new boundaries, and intersect with the first error area to obtain a new error area. As data points arrive continuously, the error area gradually shrinks and eventually becomes empty at a certain data point. This data point is used as the starting data point of the next grouping, and the previous data point of this data point is used as the split point of the current segment. Repeat the above process until the entire key value sequence is grouped.

[0123] When performing model training, each leaf node data must correspond to a model. The first model is a trained linear model. The training method of the first model includes but is not limited to: least square error (minimizing the second error norm), etc. Specifically, the present disclosure embodiment does not limit the training method of the first model.

[0124] After completing the model training, each leaf node data can be represented as a five-tuple (keys, slope, intercept, pre_group, next_group), where keys is the key value array for storing data, that is, an array composed of the elements to be indexed in the leaf node data, slope represents the model parameter slope of the first model, intercept represents the intercept of the first model, pre_group represents the address pointing to the previous leaf node data, and pre_group represents the address pointing to the next leaf node data.

[0125] Meanwhile, the first model can be expressed by, but not limited to, formula (1):

[0126] fs(k)=k×slope+intercept formula (1)

[0127] Among them, fs(k) is the predicted position of the key value (target element), and k is the key value.

[0128] Regarding the first model, the embodiment of the present disclosure also provides a schematic diagram of the first model corresponding to the leaf node data, such as Figure 2 As shown, it represents the first model corresponding to a group of leaf node data with key values ​​of [27, 46].

[0129] Step 102, perform memory pre-allocation processing in each of the first models to obtain a storage space corresponding to each of the first models, and store a corresponding lock-free linked list in the storage space; wherein the lock-free linked list is a single-link lock-free linked list composed of incremental modifications of the corresponding leaf node data.

[0130] In the embodiment of the present disclosure, the lock-free linked list includes at least the modification records made to the leaf node data, the node status, the latest attributes of the node, etc. Specifically, the embodiment of the present disclosure does not limit the content of the lock-free linked list.

[0131] Regarding the lock-free linked list, the embodiment of the present disclosure provides a property diagram of a lock-free linked list, such as Figure 3 As shown, Size: records the size of the leaf node data, that is, the key value (the element to be indexed), such as Figure 3 As shown in the figure, after executing Δdelete[K1, V1] (deleting the key value [K1, V1]), the size of a LeafNode (leaf node data) becomes 6; after executing Δinsert[K2, V2] (inserting the key value [K2, V2]), the size becomes 7.

[0132] Depth: records the distance between the current operation step and LeafNode in the lock-free linked list, such as Figure 3 As shown, the Depth of the Δdelete[K1, V1] operation is 2, and the Depth of the Δinsert[K2, V2] operation is 1.

[0133] Offset: The position of the key value to be operated in the leaf node data (such as Figure 3 In the Δdelete[K1, V1] operation, K1 is the first LeafNode, so Offset is 0. In the Δinsert[K2, V2] operation, K2 is the second LeafNode, so Offset is 1.

[0134] Furthermore, when performing memory pre-allocation processing, it is necessary to pre-allocate space inside the first model corresponding to each leaf node data to store the lock-free linked list. Specifically, regarding the process of memory pre-allocation processing, the embodiment of the present disclosure provides a schematic diagram of memory pre-allocation processing, such as Figure 4 As shown, the first model is stored in the high address end of the pre-allocated MemoryChunk (storage space), and the lock-free linked list is stored from the high address end to the low address end ( Figure 4 By designing the connection direction of the lock-free linked list opposite to the memory growth direction, an efficient traversal method can be provided. Reading incremental records in order from new to old is consistent with the linear access method from low to high. This gives full play to the hardware prefetch function of the CPU and improves the performance of multi-threaded lock-free linked list reading and writing.

[0135] Step 103, determine the remaining space corresponding to each of the storage spaces, reorganize and optimize the leaf node data corresponding to the storage space whose remaining space is less than a preset space threshold, obtain an initial index structure, and connect the initial index structure to the preset tree structure according to the lock-free linked list to obtain the target index structure.

[0136] In the embodiment of the present disclosure, the preset space threshold is a threshold set by user, such as 0%, 2%, etc. Specifically, the embodiment of the present disclosure does not limit the preset space threshold.

[0137] Regarding the reorganization and optimization processing of leaf node data, the following methods can be used but are not limited to: when the storage space of a leaf node data is full (that is, the remaining space is 0%), the reorganization optimization of this leaf node data is triggered (that is, as data is continuously inserted, the amount of data stored in the lock-free linked list corresponding to each group (leaf node data) increases. In order to prevent the lock-free linked list from being too long and increasing the workload of merging and reorganization, the reorganization optimization will be triggered when the number of lock-free linked lists exceeds the upper limit).

[0138] The target index structure is divided into two parts, namely, an upper tree structure and a lower leaf node group (the plurality of leaf node data). The preset tree structure is a tree structure with a custom setting, such as an R tree, etc. Specifically, regarding the target index structure, the present disclosure embodiment provides a structural schematic diagram of a target index structure, such as Figure 5 As shown, there are two parts: (1) R-tree is the upper tree structure (preset tree structure), and multiple groups are designed as leaf node data. The leaf node data of the bottom group is indexed by R-tree. R-tree saves the group node start_key. The address of each group is Group[i]. The upper tree structure stores<start_key,Group[i]> key / value pair. (2) The lower leaf node group (leaf node data) contains a key value array, which is connected to the tree structure of the first part through a lock-free structure delta-node (i.e., the lock-free linked list). The delta-node stores incremental data records. Each insert / delete operation (insert / delete operation) generates a new delta-node connected to the old delta-node. A linear model (first model line model) is trained based on the data in each group (leaf node data), and the key (target element) to be queried is input. The output is the model predicted data location (target location information). At the same time, when the index data is grouped and divided into space, it can be segmented into multiple groups, i.e., multiple leaf node data, based on a greedy algorithm, so that the grouped data is minimized within the error range of the linear model, thereby reducing the index memory space.

[0139] Step 104 , indexing the target element through the target index structure to obtain target position information of the target element in the data to be indexed, wherein the target element is any element in the elements to be indexed.

[0140] In an embodiment of the present disclosure, when performing index processing of a target element, the following methods may be used but are not limited to: searching for the group to which the target element belongs (leaf node data) in a preset tree structure; traversing the delta-node (unlinked list) corresponding to the group to which the target element belongs to find the target element; using the linear model (first model) corresponding to the group to which the target element belongs to obtain the predicted position, thereby obtaining the target position information.

[0141] The search in the preset tree structure means that the address of each group (leaf node data) is stored in the R-tree (with start_key as the key and Group[i] as the value). The preset tree structure is first searched to find the group to which the target element belongs. The R-tree (preset tree structure) is traversed from the root to the leaf by using a standard tree traversal algorithm. It terminates when the leaf node data containing the target element is reached. Since the R-tree is used to locate the confirmed group rather than a single point, the running time of searching the group to which the target element belongs is O(logb(p)), where b is the fanout constant representing the preset tree structure, and p is the number of groups created (i.e., the number of leaf node data).

[0142] The lock-free learning indexing method provided by the present disclosure performs grouping space division processing on the data to be indexed to obtain multiple leaf node data, and performs model training processing according to each of the leaf node data to obtain a first model corresponding to each of the leaf node data; wherein each of the leaf node data at least includes multiple elements to be indexed and position information corresponding to each of the elements to be indexed; performs memory pre-allocation processing in each of the first models to obtain a storage space corresponding to each of the first models, and stores a corresponding lock-free linked list in the storage space; wherein the lock-free linked list is a single-link lock-free linked list composed of incremental modifications of the corresponding leaf node data; determines the remaining space corresponding to each of the storage spaces, reorganizes and optimizes the leaf node data corresponding to the storage space whose remaining space is less than a preset space threshold to obtain an initial index structure, and connects the initial index structure to a preset tree structure according to the lock-free linked list to obtain a target index structure; performs index processing on the target element through the target index structure to obtain the target position information of the target element in the data to be indexed, wherein the target element is any element among the elements to be indexed. Compared with the related art, the embodiment of the present disclosure divides the data to be indexed into different underlying data groups, namely leaf node data, each group corresponds to a linear model prediction, namely the first model, and adds a lock-free linked list inside the group as a multi-threaded lock-free structure. Finally, a target index structure based on a preset tree structure is constructed, and index processing is performed through the target index structure. It can perform multi-threaded high-concurrency access, support multi-threaded queries and insertion updates, and support efficient multi-threaded loads.

[0143] In one possible implementation of the embodiment of the present disclosure, regarding the grouping space division processing of the index data, the embodiment of the present disclosure provides a flow chart of the grouping space division, such as Figure 6 As shown, including:

[0144] Step 601 : performing coordinate point construction processing according to each of the elements to be indexed in the data to be indexed and the position information corresponding to each of the elements to be indexed, to obtain the target data point corresponding to each of the elements to be indexed.

[0145] In the embodiment of the present disclosure, when constructing coordinate points, the element to be indexed is used as the horizontal coordinate, and the position information corresponding to the element to be indexed is constructed as the vertical coordinate. Specifically, for ease of understanding, the embodiment of the present disclosure provides a schematic diagram of the principle of grouping space division, such as Figure 7 As shown, p0, ..., p5 are the target data points corresponding to the five elements to be indexed in the data to be indexed.

[0146] Step 602, determining a first error upper boundary and a first error lower boundary of a first data point according to a preset error, and determining an error range according to the first error upper boundary and the first error lower boundary to obtain a first error region corresponding to the first data point; wherein the first data point is any target data point.

[0147] In the embodiment of the present disclosure, the preset error is the maximum acceptable error between the actual position and the predicted position of the element to be indexed, which is set by the user. Regarding the determination of the first error area, the following method may be used but is not limited to it: assuming that the preset error is δ, p0 (x0, y0) is the starting data point of the key value sequence (data to be indexed), and the starting data point is the starting point of the first model. When reading the second data point p1 (x1, y1), the ordinate of x1 on the key value sequence must be between points p1┬ and p1┴, which are the upper and lower boundary points of p1 and are obtained by y1±δ. Therefore, |p1┬-p1|=|p1-p1┴|=δ, any line between the upper line u1 and the lower line l1 meets the error requirement of p1, and the area between the upper line u1 and the lower line l1 is the feasible space of the data point p1. At this time, if the first data point is represented by p1, the first error upper boundary is u1, the first error lower boundary is l1, and the first error area is the area between the upper line u1 and the lower line l1.

[0148] Step 603, determine the second error upper boundary and the second error lower boundary of the second data point according to the preset error, and determine the error range according to the second error upper boundary and the second error lower boundary to obtain the second error area corresponding to the second data point; wherein the second data point is the next adjacent target data point of the first data point.

[0149] In the embodiment of the present disclosure, when a new data point p2 (x2, y2) is read, any line between the upper line u2 and the lower line l2 meets the error requirement of p2, and the area between the upper line u2 and the lower line l2 is the feasible space of the data point p2. At this time, if the second data point is represented by p2, the second error upper boundary is u2, the second error lower boundary is l2, and the second error area is the area between the upper line u2 and the lower line l2.

[0150] Step 604: When there is an overlap between the first error region and the second error region, the to-be-indexed element corresponding to the first data point and the to-be-indexed element corresponding to the second data point are divided into first leaf node data.

[0151] In the disclosed embodiment, when a new data point is read, the error region is updated incrementally, for example: Figure 7 In the example, p1 is taken as the first data point and p2 is taken as the second data point. When reading the data point p2, two upper and lower boundary lines u2 and l2 are obtained. The area between u2 and l2 is the second error region of p2. In the region generated by the intersection of the second error region and the first error region, any straight line can meet the error requirements of p1 and p2. Therefore, it can be determined that p1 and p2 belong to the same group, that is, the same leaf node data.

[0152] Step 605, when there is no overlapping area between the first error area and the second error area, the to-be-indexed elements corresponding to the first data point are divided into first leaf node data, and the to-be-indexed elements corresponding to the second data point are divided into second leaf node data, and the second data point is used as the starting data point of the second leaf node data.

[0153] In the disclosed embodiment, when a new data point is read, the error region is updated incrementally, for example: Figure 7 In the example, p4 is taken as the first data point and p5 is taken as the second data point. When reading the data point p4, two upper and lower boundary lines u4 and l4 are obtained. The area between u5 and l5 is the second error region of p5. The second error region does not intersect with the first error region, and any straight line cannot meet the error requirements of p4 and p5. Therefore, it can be determined that p4 and p5 do not belong to the same group, that is, they do not belong to the same leaf node data.

[0154] Step 606, repeat the above steps until all the data to be indexed are grouped to obtain the multiple leaf node data.

[0155] In the disclosed embodiment, the incremental update process of the error region is repeated until the intersection region of the error region becomes empty. When the intersection region of the error region becomes empty, it means that any subsequent data point (including the current data point) cannot be approximated by a straight line within the error range. Therefore, the previous data point will be the endpoint of the current segment and the new starting data point of the next grouping segment.

[0156] Specifically, Figure 7 As shown in the right figure in , the overall process of the embodiment of the present disclosure includes but is not limited to: after p1 and p2, the next two data points p3 and p4 are read in sequence. At this time, the common intersection area between p5, p5, p5, and p5 is [u3, l2]. After reading p5, the common intersection area between p5 and p1, p2, p3, and p4 becomes empty, because the upper limit line u5 is lower than the lower limit line l2. Therefore, data point p4 is the end point of the grouping, and the line connecting p0 and p4 is the approximate result of the data points between p0 and p4. Use p5 as the starting data point of the next group, and repeat this process until all the data to be indexed are grouped.

[0157] In one implementable method of the embodiment of the present disclosure, when performing model training processing based on the leaf node data, it can also be implemented in but not limited to the following methods: performing position prediction processing on the elements to be indexed through a preset linear model to obtain predicted position information corresponding to each of the elements to be indexed; performing error analysis based on the position information and the predicted position information to obtain a prediction error corresponding to each of the elements to be indexed; performing model optimization processing on the preset linear model based on the prediction error, the elements to be indexed and the position information to obtain the first model corresponding to each of the leaf node data.

[0158] In the embodiment of the present disclosure, it is necessary to pre-calculate the prediction error, and then optimize the model according to the prediction error through a preset optimization algorithm to obtain the first model. Regarding the prediction error, the embodiment of the present disclosure provides an error schematic diagram of the first model, such as Figure 8 As shown, if the line offset of the (x2, y2) linear model is greater than the error, the model from (x1, y1) to (x3, y3) is invalid. Therefore, it is necessary to optimize the model according to the prediction error. The optimization algorithm is a custom algorithm, such as the second-order method, etc. Specifically, the embodiments of the present disclosure are not limited.

[0159] In an implementable manner of the embodiment of the present disclosure, when performing reorganization optimization processing, it can also be implemented in but not limited to the following manner: the lock-free linked list corresponding to the target leaf node data is processed to obtain a first data set and a second data set; wherein the target leaf node data is the leaf node data corresponding to the storage space whose remaining space is less than the preset space threshold; according to the first data set, a lock-free self-organizing linked list and a lock-free B+ tree are constructed in the target buffer corresponding to the target leaf node data; wherein, the first data set includes existing data of the lock-free linked list after the insertion and / or deletion operations are performed, and the second data set includes data deleted from the lock-free linked list after the insertion and / or deletion operations are performed; the target leaf node data is subjected to point query reorganization processing through the lock-free self-organizing linked list to obtain a first index structure, and the first index structure is subjected to range query reorganization processing through the lock-free B+ tree to obtain the initial index structure.

[0160] For ease of understanding, the present disclosure provides a schematic diagram of a structure of adaptive data reorganization optimization, such as Fig. 9 As shown in the figure, the target leaf node data has a corresponding buffer, namely the target buffer, which is divided into a lock-free self-organizing linked list and a lock-free b+tree+delta-node (lock-free B+ tree). The lock-free self-organizing linked list and the lock-free B+ tree process point queries and range queries respectively. First, the background thread processes the delta-node (lock-free linked list) corresponding to the target leaf node data and outputs two data sets L exist (first dataset) and L deleted (Second data set), add the incremental records of insertion and deletion to L exist and L deleted To confirm that the inserted and deleted values ​​are not overwritten by the newly added values, use L exist Store delta-node existing data, using L exist The data is constructed into a lock-free self-organizing linked list and a lock-free b+tree. The query returns the result (L buffer U result)-L deleted , where L buffer The data is returned to the buffer, and result is the grouped return result.

[0161] In an implementable manner of the embodiment of the present disclosure, when performing point query reorganization, the embodiment of the present disclosure provides a flowchart of point query reorganization, such as Fig.10 As shown, including:

[0162] Step 1001, performing window division processing in the lock-free self-organizing linked list to obtain a first window, and performing region division processing on the first window according to a preset ratio to obtain a first region and a second region.

[0163] In the embodiment of the present disclosure, the preset ratio is a custom-set ratio, for example, 1% of the total linked list length, 20% of the total linked list length, etc. Specifically, the embodiment of the present disclosure does not limit the preset ratio.

[0164] For ease of understanding, the present disclosure provides a schematic diagram of the structure of a lock-free self-organizing linked list. Fig.11 As shown, it is necessary to combine the least frequently used strategy (LFU) and the least recently used strategy (LRU) to ensure the freshness of the data in the first window, and to achieve adaptation between LFU and LRU. The first window is divided at the front end of the lock-free self-organizing linked list to save frequency data and recency data at the same time, and the window is divided into an A1 area (first area) and an A2 area (second area). 1% of the total linked list length is used as the A1 area to store the recency data that has been recently accessed, and the following 20% ​​of the total linked list length is used as the A2 area to store the frequency data that has been accessed more frequently, that is, the recency data is stored in the first area, and the frequency data is stored in the second area.

[0165] Step 1002: store proximity data in the first area; wherein the proximity data is the most recently accessed data for accessing the target leaf node data.

[0166] In the disclosed embodiment, 1% of the total linked list length is used as the A1 area to store the recency data most recently accessed, that is, the recency data can be directly stored in the first area.

[0167] Step 1003, monitor the data storage records in the first window to obtain monitoring results. When it is determined according to the monitoring results that there are additional records in the data storage records, perform numerical calculation processing on the frequency data, and store the processed frequency data in the second area; wherein the frequency data is the access frequency of accessing the target leaf node data; wherein the first index structure includes at least the first area and the second area.

[0168] In the disclosed embodiment, in order to avoid the problem of infinite growth of the frequency count and changes in the data access mode, each time a record is added to the first window, the frequency values ​​of all records in the first window are divided by 2. The division by 2 can be efficiently implemented in hardware using a shift register, which can play a role in count attenuation.

[0169] Furthermore, through the point query reorganization of the lock-free self-organizing linked list, L exist Data reorganization and merging are performed, and the lock-free self-organizing linked list is applied to concurrent scenarios, which can fully utilize the advantages of multi-core processors and supplement the scenario of data deletion. Specifically, regarding the lock-free self-organizing linked list, the embodiment of the present disclosure also provides a structural schematic diagram of another lock-free self-organizing linked list, such as Fig.12 As shown, the lock-free self-organizing linked list contains L exist And a bitmap, which is used to save L exist Does the corresponding key value (to be indexed element) exist? The length is L exist_max_key -L exist_min_key When multiple threads access a lock-free self-organizing linked list, if one thread moves a node to the head of the table, other threads that are still searching for the node will not be able to find the node, and will return the result that the node does not exist. In order to solve the data inconsistency problem of multiple concurrent accesses to the lock-free self-organizing linked list, if the thread does not find the data when searching for data, but finds that the bitmap exists, it will return to the head of the table and search again. When a node is moved, the node pointer is modified through the compare-and-swap (CAS) atomic operation, so that multiple concurrent accesses and modifications to the linked list structure can be achieved without locking.

[0170] In an implementable manner of the embodiment of the present disclosure, when performing range query reorganization, it can also be implemented in but not limited to the following manner: initializing the inserted element to obtain a first insertion range, wherein the range query reorganization processing of the first index result at least includes inserting and sorting the first index structure, the inserted element is the element that performs insertion and sorting on the first index structure, and the first offset range is the offset search range corresponding to the inserted element; traversing the lock-free linked list, comparing the inserted element with the first element to be compared, and obtaining a comparison result; wherein the first element to be compared is the first element to be compared traversed from the lock-free linked list, and the element to be compared is an element of the insertion and / or deletion type in the lock-free linked list; when it is determined according to the comparison result that the inserted element is the same as the first element to be compared, converging the first offset range according to the first range to be compared to obtain a second offset range. range; wherein, the first range to be compared is the offset search range corresponding to the first element to be compared; when it is determined according to the comparison result that the inserted element is larger than the first element to be compared, and the range starting value of the first offset range is smaller than the range starting value of the first range to be compared, the range starting value of the first offset range is set to the range starting value of the first range to be compared, and the second offset range is obtained; when it is determined according to the comparison result that the inserted element is smaller than the first element to be compared, and the range end value of the first offset range is smaller than the range end value of the first range to be compared, the range end value of the first offset range is set to the range end value of the first range to be compared, and the second offset range is obtained; until the comparison of the inserted element with all the elements to be compared is completed, the target offset range corresponding to the inserted element is obtained, and the inserted element is inserted into the target offset range to obtain the initial index structure.

[0171] In the embodiment of the present disclosure, an ordered lock-free B+ tree structure is used to process range queries. Since the lock granularity of the lock-free B+ tree structure is coarse, select adds a read lock, and insert / delete adds a write lock. Therefore, it is necessary to add a delta-node (lock-free linked list) between the leaf node and the tree node of the lock-free B+ tree structure to record the incremental operation, such as Fig.13 As shown, Fig.13 A structural diagram of a lock-free B+ tree structure provided in an embodiment of the present disclosure.

[0172] Among them, the unsorted data of the lock-free self-organizing linked list is sorted into an ordered data lock-free B+ tree structure. The newly added data modification is placed in the delta-node (lock-free linked list), avoiding the locking of the lock-free B+ tree structure. When querying, the delta-node is traversed first. During the range query recombination process, it includes: key-value data sorting and model retraining. To reduce the time complexity of sorting, the Offset attribute of the delta-node is used for insertion sorting to find the insertion position of the inserted element in the leaf node data. When the thread traverses the delta-node to initialize the binary search range of the key (inserted element) as [start, end] (the first insertion range), during the traversal, every time an insert / delete type key value key' (element to be compared) is scanned, then key is compared with key'. If they are equal, the offset search range is converged to [Offset1, Offset2] (the first offset range). If Offset1>start and key>key', then start is set to Offset1. Otherwise, if Offset2<end and key<key', then end is set to Offset2. Specifically, regarding the execution process of the inserted element, the embodiments of the present disclosure provide a schematic diagram of the principle of element insertion, as Fig.14 shown, insert the key value with key = 7 at the position with an offset of 3 in the array until the traversal of the delta-node ends. The running time is O(logn), compared with other sorting algorithms O(n) and O(nlogn), which optimizes the partial time complexity of grouped data recombination.

[0173] In one implementable manner of the disclosed embodiment, when indexing a target element, the following manner may also be used but is not limited to: searching the target element in the target index structure to determine the leaf node data to be indexed; wherein the leaf node data to be indexed is the leaf node data containing the target element in the target index structure; traversing the lock-free linked list to be indexed to determine the element type of the target element; wherein the lock-free linked list to be indexed is the lock-free linked list contained in the leaf node data to be indexed; in the case where it is determined that the target element is a non-unique element according to the element type, traversing the lock-free linked list to be indexed Performing data analysis and processing to obtain a third data set and a fourth data set; wherein the third data set includes existing data in the unlinked list to be indexed after insertion and / or deletion operations are performed, and the fourth data set includes data deleted from the unlinked list to be indexed after insertion and / or deletion operations are performed; performing data analysis and processing on the target element according to the third data set and the fourth data set to obtain a unique key value corresponding to the target element; inputting the unique key value into the first model to be indexed for position prediction processing to obtain the target position information, wherein the first model to be indexed is the first model corresponding to the leaf node data to be indexed.

[0174] Furthermore, the following method may be used but is not limited to: when the target element is determined to be a unique element according to the element type, the target element is input into the first model to be indexed for position prediction processing to obtain the target position information.

[0175] In the disclosed embodiment, the delta-node (unlinked list) corresponding to the group to which the target element belongs is traversed to find the target element; the linear model (first model) corresponding to the group to which the target element belongs is used to obtain the predicted position to obtain the target position information.

[0176] When traversing the delta-node corresponding to the group to which the target element belongs, searching from top to bottom in sequence, if it is determined that the target element is a non-unique element, the non-unique index is performed through the target index structure. Regarding the unique index, the embodiment of the present disclosure provides a schematic diagram of the principle of non-unique index processing, such as Fig.15 As shown, when it is necessary to traverse the delta-node, two data sets L are output through data processing. exist and L deleted , L exist Save the existing key value, L deleted Save the deleted key value. It should be noted that the first data set and the third data set are of the same type and can be accessed through L exist Indicates that the second data set and the fourth data set are of the same type and can be used by Ldeleted Specifically, the algorithm for processing non-unique indexes can be represented by, but not limited to, the following algorithm:

[0177] if node=delete[K1, Vi]&&if node not in L deleted

[0178] L deleted .add(node), L exist .add(noting)

[0179] if node=insert[K1, Vi]&&if node in L deleted

[0180] L deleted .add(noting), L exist .add(noting)

[0181] if node=delete[K1,Vj]&&if node not in L deleted =>

[0182] L deleted .add(node), L exist .add(noting)

[0183] if node=insert[K1, Vk]&&if node not in L deleted

[0184] L deleted .add(noting), L exist .add(node)

[0185] get result = Line_model (K1)

[0186] return L exist U(result-L deleted )

[0187] The non-unique index ensures the integrity of the result set when querying using non-unique elements.

[0188] If the target element is determined to be a unique element, the position of the target element in the group is found. When creating a group, the error between the position of the element to be indexed stored by the FSW algorithm and the position of the target element determined by the linear model does not exceed a constant. In order to calculate the approximate position of the target element in the leaf node to be indexed, the target element is subtracted from the first element to be indexed that appears in the leaf node to be indexed, and then the difference is multiplied by the first model slope Groupslope to be indexed. Specifically, it can be expressed by formula (2) (3):

[0189] pred_pos=(k-start_key)×Group.slope formula (2)

[0190] true_pos∈[pred_pos-err,pred_pos+err] formula (3)

[0191] Among them, after the position of the interpolated element, the actual position of key_pos can be guaranteed to be within the error threshold, the time complexity of group search Key is limited, and the running time of locating the key is O(log2(error)), where error is a constant.

[0192] In summary, the embodiments of the present disclosure can achieve the following technical effects:

[0193] 1. The disclosed embodiment divides the data to be indexed into different underlying data groups, namely leaf node data, each group corresponds to a linear model prediction, namely the first model, and adds a lock-free linked list inside the group as a multi-threaded lock-free structure, and finally constructs a target index structure based on a preset tree structure. The target index structure is used for index processing, and multi-threaded high-concurrency access can be performed, and query, insertion and update under multi-threading can be supported, and efficient multi-threaded load can be supported.

[0194] 2. The disclosed embodiment is implemented by a lock-free structure through a learning index, supports multi-threaded high-concurrency access, and compared with locks, reduces the lock waiting time and improves high-concurrency performance.

[0195] 3. The disclosed embodiment uses memory pre-allocation to store data from high address end to low address end, which is consistent with the linear access method from low to high. It gives full play to the memory hardware prefetch function (hardware memory prefetch) of modern CPUs and improves the performance of multi-threaded delta-node linked list reading and writing. New data positioning only requires binary search to traverse the delta-node linked list, avoiding traversal of the entire key value array for writing.

[0196] 4. The disclosed embodiment supports multi-threading without blocking through a lock-free self-organizing linked list, efficiently processes cache data, and improves the efficiency of hot data access based on LFU & LRU load perception.

[0197] 5. The embodiment of the present disclosure ensures the integrity of the result set when using non-unique element queries through non-unique indexes.

[0198] Corresponding to the above-mentioned lock-free learning indexing method, the present invention also proposes a lock-free learning indexing device. Since the device embodiment of the present invention corresponds to the above-mentioned method embodiment, the details not disclosed in the device embodiment can be referred to the above-mentioned method embodiment, and will not be repeated in the present invention.

[0199] Fig.16 A structural diagram of a lock-free learning indexing device provided by an embodiment of the present disclosure is shown in FIG. Fig.16 As shown, including:

[0200] A partitioning unit 161 is used to perform grouping and spatial partitioning processing on the data to be indexed to obtain a plurality of leaf node data;

[0201] The training unit 162 is used to perform model training processing according to each leaf node data to obtain a first model corresponding to each leaf node data; wherein each leaf node data at least includes a plurality of elements to be indexed and position information corresponding to each element to be indexed;

[0202] The allocation unit 163 is used to perform memory pre-allocation processing in each of the first models to obtain a storage space corresponding to each of the first models, and store a corresponding lock-free linked list in the storage space; wherein the lock-free linked list is a single-link lock-free linked list formed according to the incremental modification of the corresponding leaf node data;

[0203] The optimization unit 164 is used to determine the remaining space corresponding to each of the storage spaces, reorganize and optimize the leaf node data corresponding to the storage space whose remaining space is less than a preset space threshold to obtain an initial index structure, and connect the initial index structure to a preset tree structure according to the lock-free linked list to obtain a target index structure;

[0204] The indexing unit 165 is used to perform indexing processing on the target element through the target index structure to obtain target position information of the target element in the data to be indexed, wherein the target element is any element in the elements to be indexed.

[0205] The lock-free learning indexing device provided by the present disclosure performs grouping space division processing on the data to be indexed to obtain multiple leaf node data, and performs model training processing according to each of the leaf node data to obtain a first model corresponding to each of the leaf node data; wherein each of the leaf node data at least includes multiple elements to be indexed and position information corresponding to each of the elements to be indexed; performs memory pre-allocation processing in each of the first models to obtain a storage space corresponding to each of the first models, and stores a corresponding lock-free linked list in the storage space; wherein the lock-free linked list is a single-link lock-free linked list composed of incremental modifications of the corresponding leaf node data; determines the remaining space corresponding to each of the storage spaces, reorganizes and optimizes the leaf node data corresponding to the storage space whose remaining space is less than a preset space threshold to obtain an initial index structure, and connects the initial index structure to a preset tree structure according to the lock-free linked list to obtain a target index structure; performs index processing on the target element through the target index structure to obtain the target position information of the target element in the data to be indexed, wherein the target element is any element among the elements to be indexed. Compared with the related art, the embodiment of the present disclosure divides the data to be indexed into different underlying data groups, namely leaf node data, each group corresponds to a linear model prediction, namely the first model, and adds a lock-free linked list inside the group as a multi-threaded lock-free structure. Finally, a target index structure based on a preset tree structure is constructed, and index processing is performed through the target index structure. It can perform multi-threaded high-concurrency access, support multi-threaded queries and insertion updates, and support efficient multi-threaded loads.

[0206] Furthermore, in a possible implementation of the embodiment of the present disclosure, as Fig.17 As shown, the division unit 161 includes:

[0207] A construction module 1611 is used to perform coordinate point construction processing according to each of the elements to be indexed in the data to be indexed and the position information corresponding to each of the elements to be indexed, so as to obtain a target data point corresponding to each of the elements to be indexed;

[0208] A determination module 1612 is used to determine a first error upper boundary and a first error lower boundary of a first data point according to a preset error, and determine an error range according to the first error upper boundary and the first error lower boundary to obtain a first error region corresponding to the first data point; wherein the first data point is any target data point;

[0209] The determination module 1612 is further configured to determine a second error upper boundary and a second error lower boundary of a second data point according to the preset error, and determine an error range according to the second error upper boundary and the second error lower boundary to obtain a second error region corresponding to the second data point; wherein the second data point is a next adjacent target data point of the first data point;

[0210] A division module 1613 is used to divide the to-be-indexed element corresponding to the first data point and the to-be-indexed element corresponding to the second data point into first leaf node data when there is an overlapped area between the first error area and the second error area;

[0211] The division module 1613 is further used for, when there is no overlap between the first error region and the second error region, dividing the to-be-indexed element corresponding to the first data point into first leaf node data, dividing the to-be-indexed element corresponding to the second data point into second leaf node data, and using the second data point as the starting data point of the second leaf node data;

[0212] Repeat the above steps until all the data to be indexed are grouped to obtain the multiple leaf node data.

[0213] Furthermore, in a possible implementation of the embodiment of the present disclosure, as Fig.17 As shown, the training unit 162 includes:

[0214] The prediction module 1621 is used to perform position prediction processing on the elements to be indexed by using a preset linear model to obtain the predicted position information corresponding to each of the elements to be indexed;

[0215] An analysis module 1622 is used to perform error analysis according to the position information and the predicted position information to obtain a prediction error corresponding to each of the elements to be indexed;

[0216] The optimization module 1623 is used to perform model optimization processing on the preset linear model according to the prediction error, the element to be indexed and the position information to obtain the first model corresponding to each leaf node data.

[0217] Furthermore, in a possible implementation of the embodiment of the present disclosure, as Fig.17 As shown, the optimization unit 164 includes:

[0218] The processing module 1641 is used to process the lock-free linked list corresponding to the target leaf node data to obtain a first data set and a second data set; wherein the target leaf node data is the leaf node data corresponding to the storage space whose remaining space is less than the preset space threshold;

[0219] A construction module 1642 is used to construct a lock-free self-organizing linked list and a lock-free B+ tree in a target buffer corresponding to the target leaf node data according to the first data set; wherein the first data set includes the existing data of the lock-free linked list after the insertion and / or deletion operation is performed, and the second data set includes the data deleted from the lock-free linked list after the insertion and / or deletion operation is performed;

[0220] The reorganization module 1643 is used to perform point query reorganization processing on the target leaf node data through the lock-free self-organizing linked list to obtain a first index structure, and perform range query reorganization processing on the first index structure through the lock-free B+ tree to obtain the initial index structure.

[0221] Furthermore, in a possible implementation of the embodiment of the present disclosure, as Fig.17 As shown, the recombinant module 1643 is also used for:

[0222] Performing window division processing in the lock-free self-organizing linked list to obtain a first window, and performing region division processing on the first window according to a preset ratio to obtain a first region and a second region;

[0223] storing the proximity data in the first area; wherein the proximity data is the most recently accessed data for accessing the target leaf node data;

[0224] Monitor the data storage records in the first window to obtain monitoring results, and when it is determined according to the monitoring results that there are additional records in the data storage records, perform numerical calculation processing on the frequency data, and store the processed frequency data in the second area; wherein the frequency data is the access frequency of accessing the target leaf node data;

[0225] The first index structure at least includes the first area and the second area.

[0226] Furthermore, in a possible implementation of the embodiment of the present disclosure, as Fig.17 As shown, the recombinant module 1643 is also used for:

[0227] Initializing the inserted element to obtain a first insertion range, wherein performing range query reorganization processing on the first index result at least includes performing insertion sort processing on the first index structure, the inserted element is an element that performs insertion sort processing on the first index structure, and the first offset range is an offset search range corresponding to the inserted element;

[0228] Traversing the lock-free linked list, comparing the inserted element with the first element to be compared, and obtaining a comparison result; wherein the first element to be compared is the first element to be compared traversed from the lock-free linked list, and the element to be compared is an element of the insertion and / or deletion type in the lock-free linked list;

[0229] When it is determined according to the comparison result that the inserted element is identical to the first element to be compared, the first offset range is converged according to the first range to be compared to obtain a second offset range; wherein the first range to be compared is an offset search range corresponding to the first element to be compared;

[0230] When it is determined according to the comparison result that the inserted element is larger than the first element to be compared, and the range start value of the first offset range is smaller than the range start value of the first range to be compared, setting the range start value of the first offset range to the range start value of the first range to be compared, to obtain the second offset range;

[0231] When it is determined according to the comparison result that the inserted element is smaller than the first element to be compared, and the range end value of the first offset range is smaller than the range end value of the first range to be compared, setting the range end value of the first offset range to the range end value of the first range to be compared, to obtain the second offset range;

[0232] Until the comparison between the inserted element and all the elements to be compared is completed, the target offset range corresponding to the inserted element is obtained, and the inserted element is inserted into the target offset range to obtain the initial index structure.

[0233] Furthermore, in a possible implementation of the embodiment of the present disclosure, as Fig.17 As shown, the index unit 165 includes:

[0234] A search module 1651 is used to search the target element in the target index structure to determine the leaf node data to be indexed; wherein the leaf node data to be indexed is the leaf node data containing the target element in the target index structure;

[0235] A traversal module 1652 is used to traverse the un-locked linked list to be indexed to determine the element type of the target element; wherein the un-locked linked list to be indexed is the un-locked linked list contained in the leaf node data to be indexed;

[0236] The analysis module 1653 is used for, when it is determined that the target element is a non-unique element according to the element type, performing data analysis processing on the to-be-indexed lock-free linked list to obtain a third data set and a fourth data set; wherein the third data set includes existing data of the to-be-indexed lock-free linked list after the insertion and / or deletion operation is performed, and the fourth data set includes data deleted from the to-be-indexed lock-free linked list after the insertion and / or deletion operation is performed;

[0237] The analysis module 1653 is further used to perform data analysis on the target element according to the third data set and the fourth data set to obtain a unique key value corresponding to the target element;

[0238] The prediction module 1654 is used to input the unique key value into the first model to be indexed for position prediction processing to obtain the target position information, wherein the first model to be indexed is the first model corresponding to the leaf node data to be indexed.

[0239] Furthermore, in a possible implementation of the embodiment of the present disclosure, as Fig.17 As shown, the indexing unit 165 is also used to, when it is determined that the target element is a unique element according to the element type, input the target element into the first model to be indexed for position prediction processing to obtain the target position information.

[0240] It should be noted that the above explanation of the method embodiment is also applicable to the device of the embodiment of the present disclosure, and the principle is the same, which is no longer limited in the embodiment of the present disclosure.

[0241] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium and a computer program product.

[0242] Fig.18 A schematic block diagram of an example electronic device 1800 that can be used to implement an embodiment of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.

[0243] like Fig.18 As shown, the device 1800 includes a computing unit 1801, which can perform various appropriate actions and processes according to a computer program stored in a ROM (Read-Only Memory) 1802 or a computer program loaded from a storage unit 1808 to a RAM (Random Access Memory) 1803. In the RAM 1803, various programs and data required for the operation of the device 1800 can also be stored. The computing unit 1801, the ROM 1802, and the RAM 1803 are connected to each other via a bus 1804. An I / O (Input / Output) interface 1805 is also connected to the bus 1804.

[0244] A number of components in the device 1800 are connected to the I / O interface 1805, including: an input unit 1806, such as a keyboard, a mouse, etc.; an output unit 1807, such as various types of displays, speakers, etc.; a storage unit 1808, such as a disk, an optical disk, etc.; and a communication unit 1809, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 1809 allows the device 1800 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0245] The computing unit 1801 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 1801 include, but are not limited to, a CPU (Central Processing Unit), a GPU (Graphic Processing Units), various dedicated AI (Artificial Intelligence) computing chips, various computing units running machine learning model algorithms, a DSP (Digital Signal Processor), and any appropriate processor, controller, microcontroller, etc. The computing unit 1801 performs the various methods and processes described above, such as a lock-free learning indexing method. For example, in some embodiments, the lock-free learning indexing method may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 1808. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 1800 via the ROM 1802 and / or the communication unit 1809. When the computer program is loaded into the RAM 1803 and executed by the computing unit 1801, one or more steps of the method described above may be performed. Alternatively, in other embodiments, the computing unit 1801 may be configured to execute the aforementioned lock-free learning indexing method in any other appropriate manner (for example, by means of firmware).

[0246] Various embodiments of the systems and techniques described above herein may be implemented in digital electronic circuit systems, integrated circuit systems, FPGAs (Field Programmable Gate Arrays), ASICs (Application-Specific Integrated Circuits), ASSPs (Application Specific Standard Products), SOCs (System On Chips), CPLDs (Complex Programmable Logic Devices), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include: being implemented in one or more computer programs that may be executed and / or interpreted on a programmable system including at least one programmable processor that may be a dedicated or general-purpose programmable processor that may receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0247] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0248] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or device, or any suitable combination of the foregoing. More specific examples of machine-readable storage media may include electrical connections based on one or more lines, portable computer disks, hard disks, RAM, ROM, EPROM (Electrically Programmable Read-Only-Memory) or flash memory, optical fiber, CD-ROM (Compact Disc Read-Only Memory), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0249] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (Cathode-Ray Tube) or LCD (Liquid Crystal Display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0250] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: LAN (Local Area Network), WAN (Wide Area Network), the Internet, and blockchain networks.

[0251] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship between the client and the server is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services ("Virtual Private Server", or "VPS" for short). The server may also be a server of a distributed system, or a server combined with a blockchain.

[0252] It should be noted that artificial intelligence is a discipline that studies how computers can simulate certain human thought processes and intelligent behaviors (such as learning, reasoning, thinking, planning, etc.), and includes both hardware-level and software-level technologies. Artificial intelligence hardware technologies generally include technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, and big data processing; artificial intelligence software technologies mainly include computer vision technology, speech recognition technology, natural language processing technology, as well as machine learning / deep learning, big data processing technology, knowledge graph technology, and other major directions.

[0253] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.

[0254] The above specific implementations do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. A lock-free learning indexing method, characterized in that: include: The data to be indexed are processed by grouping and spatial division to obtain a plurality of leaf node data, and model training is performed according to each of the leaf node data to obtain a first model corresponding to each of the leaf node data; wherein each of the leaf node data at least includes a plurality of elements to be indexed and position information corresponding to each of the elements to be indexed; Perform memory pre-allocation processing in each of the first models to obtain storage space corresponding to each of the first models, and store a corresponding lock-free linked list in the storage space; wherein the lock-free linked list is a single-chain lock-free linked list formed according to the incremental modification of the corresponding leaf node data; Determine the remaining space corresponding to each of the storage spaces, reorganize and optimize the leaf node data corresponding to the storage space whose remaining space is less than a preset space threshold to obtain an initial index structure, and connect the initial index structure to a preset tree structure according to the lock-free linked list to obtain a target index structure; The target element is indexed by the target index structure to obtain target position information of the target element in the data to be indexed, wherein the target element is any element in the elements to be indexed.

2. The method according to claim 1, characterized in that: The grouping and spatial division of the data to be indexed to obtain a plurality of leaf node data includes: Perform coordinate point construction processing according to each of the elements to be indexed in the data to be indexed and the position information corresponding to each of the elements to be indexed, to obtain the target data point corresponding to each of the elements to be indexed; Determine a first error upper boundary and a first error lower boundary of a first data point according to a preset error, and determine an error range according to the first error upper boundary and the first error lower boundary to obtain a first error region corresponding to the first data point; wherein the first data point is any target data point; Determine a second error upper boundary and a second error lower boundary of a second data point according to the preset error, and determine an error range according to the second error upper boundary and the second error lower boundary to obtain a second error region corresponding to the second data point; wherein the second data point is the next adjacent target data point of the first data point; In the case where there is an overlap between the first error region and the second error region, dividing the to-be-indexed element corresponding to the first data point and the to-be-indexed element corresponding to the second data point into first leaf node data; In the case where the first error region and the second error region do not overlap, the to-be-indexed element corresponding to the first data point is divided into first leaf node data, the to-be-indexed element corresponding to the second data point is divided into second leaf node data, and the second data point is used as the starting data point of the second leaf node data; Repeat the above steps until all the data to be indexed are grouped to obtain the multiple leaf node data.

3. The method according to claim 1, characterized in that The performing of model training processing according to each leaf node data to obtain a first model corresponding to each leaf node data comprises: Performing position prediction processing on the elements to be indexed by using a preset linear model to obtain predicted position information corresponding to each of the elements to be indexed; Perform error analysis according to the position information and the predicted position information to obtain the prediction error corresponding to each of the elements to be indexed; The preset linear model is optimized according to the prediction error, the element to be indexed and the position information to obtain the first model corresponding to each leaf node data.

4. The method according to claim 1, characterized in that: The step of reorganizing and optimizing the leaf node data corresponding to the storage space whose remaining space is less than the preset space threshold to obtain the initial index structure includes: Processing the lock-free linked list corresponding to the target leaf node data to obtain a first data set and a second data set; wherein the target leaf node data is the leaf node data corresponding to the storage space whose remaining space is less than the preset space threshold; Constructing a lock-free self-organizing linked list and a lock-free B+ tree in a target buffer corresponding to the target leaf node data according to the first data set; wherein the first data set includes existing data in the lock-free linked list after the insertion and / or deletion operation is performed, and the second data set includes data deleted from the lock-free linked list after the insertion and / or deletion operation is performed; The target leaf node data is subjected to point query reorganization processing through the lock-free self-organizing linked list to obtain a first index structure, and the first index structure is subjected to range query reorganization processing through the lock-free B+ tree to obtain the initial index structure.

5. The method according to claim 4, characterized in that The point query reorganization processing of the target leaf node data by the lock-free self-organizing linked list to obtain the first index structure includes: Performing window division processing in the lock-free self-organizing linked list to obtain a first window, and performing region division processing on the first window according to a preset ratio to obtain a first region and a second region; storing the proximity data in the first area; wherein the proximity data is the most recently accessed data for accessing the target leaf node data; Monitor the data storage records in the first window to obtain monitoring results, and when it is determined according to the monitoring results that there are additional records in the data storage records, perform numerical calculation processing on the frequency data, and store the processed frequency data in the second area; wherein the frequency data is the access frequency of accessing the target leaf node data; The first index structure at least includes the first area and the second area.

6. The method according to claim 4, characterized in that The performing range query reorganization processing on the first index structure by the lock-free B+ tree to obtain the initial index structure comprises: Initializing the inserted element to obtain a first insertion range, wherein performing range query reorganization processing on the first index result at least includes performing insertion sort processing on the first index structure, the inserted element is an element that performs insertion sort processing on the first index structure, and the first offset range is an offset search range corresponding to the inserted element; Traversing the lock-free linked list, comparing the inserted element with the first element to be compared, and obtaining a comparison result; wherein the first element to be compared is the first element to be compared traversed from the lock-free linked list, and the element to be compared is an element of the insertion and / or deletion type in the lock-free linked list; When it is determined according to the comparison result that the inserted element is identical to the first element to be compared, the first offset range is converged according to the first range to be compared to obtain a second offset range; wherein the first range to be compared is an offset search range corresponding to the first element to be compared; When it is determined according to the comparison result that the inserted element is larger than the first element to be compared, and the range start value of the first offset range is smaller than the range start value of the first range to be compared, setting the range start value of the first offset range to the range start value of the first range to be compared, to obtain the second offset range; When it is determined according to the comparison result that the inserted element is smaller than the first element to be compared, and the range end value of the first offset range is smaller than the range end value of the first range to be compared, setting the range end value of the first offset range to the range end value of the first range to be compared, to obtain the second offset range; Until the comparison between the inserted element and all the elements to be compared is completed, the target offset range corresponding to the inserted element is obtained, and the inserted element is inserted into the target offset range to obtain the initial index structure.

7. The method according to claim 1, characterized in that The indexing process of the target element by using the target index structure to obtain the target position information of the target element in the data to be indexed includes: Searching and processing the target element in the target index structure to determine leaf node data to be indexed; wherein the leaf node data to be indexed is the leaf node data containing the target element in the target index structure; Traversing the to-be-indexed lock-free linked list to determine the element type of the target element; wherein the to-be-indexed lock-free linked list is the lock-free linked list contained in the to-be-indexed leaf node data; In the case where it is determined that the target element is a non-unique element according to the element type, data analysis processing is performed on the lock-free linked list to be indexed to obtain a third data set and a fourth data set; wherein the third data set includes existing data of the lock-free linked list to be indexed after the insertion and / or deletion operations are performed, and the fourth data set includes data deleted from the lock-free linked list to be indexed after the insertion and / or deletion operations are performed; Performing data analysis on the target element according to the third data set and the fourth data set to obtain a unique key value corresponding to the target element; The unique key value is input into the first model to be indexed for position prediction processing to obtain the target position information, wherein the first model to be indexed is the first model corresponding to the leaf node data to be indexed.

8. The method according to claim 7, characterized in that After traversing the lock-free linked list to be indexed to determine the element type of the target element, the method further includes: When it is determined that the target element is a unique element according to the element type, the target element is input into the first model to be indexed for position prediction processing to obtain the target position information.

9. A lock-free learning indexing device, characterized in that: include: A partitioning unit is used to perform grouping and spatial partitioning processing on the data to be indexed to obtain a plurality of leaf node data; A training unit, used to perform model training processing according to each leaf node data to obtain a first model corresponding to each leaf node data; wherein each leaf node data includes at least a plurality of elements to be indexed and position information corresponding to each element to be indexed; An allocation unit, configured to perform memory pre-allocation processing in each of the first models, obtain a storage space corresponding to each of the first models, and store a corresponding lock-free linked list in the storage space; wherein the lock-free linked list is a single-chain lock-free linked list formed according to the incremental modification of the corresponding leaf node data; An optimization unit, configured to determine the remaining space corresponding to each of the storage spaces, and to reorganize and optimize the leaf node data corresponding to the storage spaces whose remaining space is less than a preset space threshold, to obtain an initial index structure; An indexing unit is used to perform indexing processing on a target element through the target index structure to obtain target position information of the target element in the data to be indexed, wherein the target element is any element among the elements to be indexed.

10. An electronic device, characterized in that: include: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 8.

Images