Data access and index establishment method, device, storage medium and program

Abstract

Embodiments of the present disclosure provide a data access and index establishment method, a device, a storage medium and a program. The method comprises: on the basis of index layer numbers of a plurality of data nodes arranged in sequence, generating a skip list comprising a plurality of index linked lists. One index linked list comprises a plurality of pieces of index information arranged according to the sequence of corresponding data nodes, and the index information comprises index feature values and a position identifier of a successor data node. The index feature value corresponding to a predecessor data node comprises the length of a common prefix character string of the predecessor data node and the successor data node. During querying of target data, a first index feature value corresponding to a first data node for comparison with the target data and a second index feature value of the target data relative to the first data node are determined, and the target data is queried on the basis of a comparison result between the second index feature value and the first index feature value. On the basis of the design of the skip list structure, the data query overhead can be reduced, and the query efficiency is improved.

Description

[0001] Data Access and Indexing Method, Device, Storage Medium and Program This disclosure claims priority to Chinese Patent Application No. 202410525979.8, filed with the China Patent Office on April 26, 2024, entitled "Data Access and Indexing Method, Device, Storage Medium and Program", the entire contents of which are incorporated herein by reference. Technical Field This disclosure relates to the field of computer technology, and more particularly to a data access and indexing method, device, storage medium and program. Background Art In computer systems, in order to improve data access performance, various indexing algorithms and data structures have been designed to achieve more efficient data access based on indexes, such as hash tables, binary trees, B-trees, B+ trees, skip lists, etc. Although hash tables have excellent performance and can complete data access (such as adding data, deleting data, modifying data, querying data) in O(1) time, they cannot support ordered data access, which limits the application scenarios. Various tree structures were designed, resulting in an exponential reduction in data access overhead and improved system performance. However, tree-based data structures generally face the problem of not supporting multi-threaded concurrency, failing to fully utilize system performance. Therefore, skip lists were designed as a new data structure, achieving similar performance to tree-based data structures and easily supporting multi-threaded concurrency. However, in practical use, it was found that cache misses become a significant bottleneck when multi-threaded concurrent operations are performed on skip lists, resulting in still relatively high data access performance overhead. This invention discloses a data access and index creation method, device, storage medium, and program to reduce the performance overhead of data access. In a first aspect, embodiments of this disclosure provide an index building method, the method comprising: determining a plurality of data nodes corresponding to a plurality of strings in a sequential arrangement; determining the index level corresponding to each of the plurality of data nodes; generating a skip list containing a plurality of index linked lists based on the index level corresponding to each of the plurality of data nodes, wherein any index linked list includes a plurality of index information arranged in a sequential order according to the corresponding data nodes, the index information including an index feature value and a position identifier of the next data node; wherein, for two adjacent data nodes corresponding to any index linked list, the index feature value corresponding to the preceding data node includes the length of the common prefix string of the two data nodes.Secondly, embodiments of this disclosure provide an index building apparatus, the apparatus comprising: a determining module, configured to determine a plurality of data nodes corresponding to a plurality of strings arranged in sequence; and a generating module, configured to determine the index level corresponding to each of the plurality of data nodes, and generate a skip list containing a plurality of index linked lists based on the index level corresponding to each of the plurality of data nodes, wherein any index linked list includes a plurality of index information arranged in order of the corresponding data nodes, the index information including an index feature value and a next data node position identifier; wherein, for any two adjacent data nodes corresponding to any index linked list, the index feature value corresponding to the preceding data node includes the length of the common prefix string of the two data nodes. Thirdly, embodiments of this disclosure provide a data access method, the method comprising: receiving a query request for target data; obtaining a skip list containing multiple index linked lists, the skip list including multiple data nodes corresponding to multiple strings arranged in sequence, wherein any index linked list includes multiple index information arranged in order of the corresponding data nodes, the index information including an index feature value and a next data node position identifier; wherein, for two adjacent data nodes corresponding to any index linked list, the index feature value corresponding to the preceding data node includes the length of the common prefix string of the two data nodes; determining from the skip list a first index feature value corresponding to a first data node to be compared with the target data, wherein the first index feature value is the index feature value corresponding to the first data node in the first index linked list; determining a second index feature value of the target data relative to the first data node; and querying the target data according to the comparison result of the second index feature value and the first index feature value.Fourthly, embodiments of this disclosure provide a data access device, the device comprising: a receiving module, configured to receive a query request for target data; an obtaining module, configured to obtain a skip list containing multiple index linked lists, the skip list including multiple data nodes corresponding to multiple strings arranged in sequence, wherein any index linked list includes multiple index information arranged in order of the corresponding data nodes, the index information including an index feature value and a next data node position identifier; wherein, for two adjacent data nodes corresponding to any index linked list, the index feature value corresponding to the preceding data node includes the length of the common prefix string of the two data nodes; a query module, configured to determine from the skip list a first index feature value corresponding to a first data node to be compared with the target data, determine a second index feature value of the target data relative to the first data node, and perform a query on the target data based on the comparison result of the second index feature value and the first index feature value; wherein, the first index feature value is the index feature value corresponding to the first data node in the first index linked list. Fifthly, embodiments of this disclosure provide an electronic device, including: a memory, a processor, and a communication interface; wherein, the memory stores executable code, and when the executable code is executed by the processor, the processor can at least implement the indexing method as described in the first aspect or the data access method as described in the third aspect. Sixthly, embodiments of this disclosure provide a non-transitory machine-readable storage medium storing executable code, and when the executable code is executed by a processor of an electronic device, the processor can at least implement the indexing method as described in the first aspect or the data access method as described in the third aspect. Seventhly, embodiments of this disclosure provide a computer program product containing a computer program, and when executed by a processor of an electronic device, the processor can at least implement the indexing method as described in the first aspect or the data access method as described in the third aspect. In the indexing and data access methods provided by embodiments of this disclosure, by introducing the concept of index feature values ​​of data nodes that are more friendly to cache access overhead into the skip list, the bandwidth and overhead of the original data node comparison operation can be reduced by comparing the index feature values ​​between data nodes.Specifically, for multiple data nodes corresponding to multiple strings arranged sequentially, a skip list containing multiple index linked lists can be generated based on the determined index level of each data node. Each index linked list includes multiple index information arranged sequentially according to the order of the corresponding data nodes. The index information contains an index feature value and an identifier for the next data node's position. For any two adjacent data nodes corresponding to any index linked list, the index feature value of the preceding data node includes the length of the common prefix string of the following data node relative to the preceding data node. In other words, the index feature value corresponding to a data node in a certain level of the index linked list can be the length of the common prefix string of the next data node pointed to by that data node in that level of the index linked list relative to the current data node. Based on the skip list structure described above, when querying target data, the first index feature value (the first index feature value is the index feature value corresponding to the first data node to be compared with the target data in the current first index linked list) is determined from the skip list, as well as the second index feature value of the target data relative to the first data node. The target data is then queried based on the comparison result between the second and first index feature values. In fact, based on the comparison result, the index feature value of the data node to be compared with the target data next is determined, thus iterating continuously until the target data query is completed. In the above scheme, completing the target data query through the comparison of the defined index feature values ​​achieves higher query efficiency and lower performance overhead compared to directly comparing the target data sequentially with each data node in the ordinary data linked list. Brief Description of the Drawings: To more clearly illustrate the technical solutions in the embodiments of this disclosure, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are some embodiments of this disclosure. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.Figure 1 is a flowchart of an index creation method provided by an embodiment of this disclosure; Figure 2 is a schematic diagram of a common data linked list provided by an embodiment of this disclosure; Figure 3a is a schematic diagram of a skip list provided by an embodiment of this disclosure; Figure 3b is a schematic diagram of a skip list based on index feature values ​​provided by an embodiment of this disclosure; Figure 4 is a flowchart of a data access method provided by an embodiment of this disclosure; Figure 5 is a structural schematic diagram of an index creation device provided by an embodiment of this disclosure; Figure 6 is a structural schematic diagram of a data access device provided by an embodiment of this disclosure; Figure 7 is a structural schematic diagram of an electronic device provided by an embodiment of this disclosure. Specifically, to make the objectives, technical solutions, and advantages of the embodiments of this disclosure clearer, the technical solutions in the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this disclosure, not all embodiments. Based on the embodiments in this disclosure, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this disclosure. It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in the embodiments of this disclosure are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use, and processing of related data must comply with the relevant laws, regulations, and standards of the relevant countries and regions, and corresponding operation entry points are provided for users to choose to authorize or refuse. The following describes some embodiments of this disclosure in detail with reference to the accompanying drawings. Where there is no conflict between the embodiments, the following embodiments and features in the embodiments can be combined with each other. In addition, the timing of the steps in the following method embodiments is only an example and not a strict limitation. First, some concepts involved in the embodiments of this disclosure will be briefly explained. Index: A separate storage structure for sorting one or more keywords in data.

[0002] B-Tree: A commonly used self-balancing tree data structure for indexing. It maintains data order, providing logarithmic time complexity performance for insertion, deletion, and search operations. B-trees are suitable for storage systems that read and write relatively large blocks of data, such as databases and file systems. Skip List: A data structure that can be considered an enhanced linked list. It improves search efficiency by adding multi-level indexes to the original linked list. Its average time complexity for search, insertion, and deletion operations can reach O(log n). oIn summary, skip lists maintain the simplicity of linked lists while significantly improving performance. A database is a repository for organizing, storing, and managing data according to data structures; it is a large collection of organized, shareable, and centrally managed data stored permanently in a computer.

[0003] KV storage: Key-Value storage, or KV storage for short, organizes, indexes, and stores data in the form of key-value pairs. KV storage is suitable for data storage that does not involve too many data relationships, and can effectively reduce the number of disk reads and writes, offering better read and write performance than databases. A cache miss refers to a situation where the required data cannot be found in the cache, requiring access to memory to retrieve the data. The following details the data access and index creation scheme provided in this disclosure. This scheme can be executed by an electronic device that stores data in some storage method (such as a database), which can be a terminal device or a server. Figure 1 is a flowchart of an index creation method provided in this disclosure. As shown in Figure 1, the method may include the following steps:

[0004] 101. Determine the sequential arrangement of multiple data nodes corresponding to multiple strings.

[0005] 102. Determine the index level corresponding to each of the multiple data nodes.

[0006] 103. Based on the index levels corresponding to multiple data nodes, a skip list containing multiple index linked lists is generated. Each index linked list includes multiple index information arranged sequentially according to the order of the corresponding data nodes. The index information includes index feature values ​​and the position identifier of the next data node. For any two adjacent data nodes corresponding to any index linked list, the index feature value of the preceding data node includes the length of the common prefix string of the two data nodes. Different application scenarios require different data to be stored. For example, in e-commerce scenarios, the data to be stored may include order information (including order number, product information, transaction amount, etc.). Another example is the data stored in a user information storage system, which may include various dimensions of attribute information for different users, such as the grades of different students in various subjects, or the relevant information of registered users of a certain application. In this embodiment, the data to be stored is stored in an ordered manner in the form of multi-level linked lists (i.e., skip lists). It should be noted that the data sorting in the skip list is not based on its write time, but on the result of data size comparison. In this embodiment, smaller data is listed first and larger data last, but reverse sorting still applies. The data can be a string, which can be converted into some codeword (such as ASCII code) for size comparison. As can be seen from the structure of the skip list, the bottom layer of the skip list can include several ordered data nodes, and each data node can have a pointer to the next data node to indicate the order of the data nodes. Above the data nodes, there can be multiple layers of index linked lists, which are used to index the data nodes. Each index linked list contains index information corresponding to the strings that are different data nodes, for searching for the corresponding data node. Therefore, in this embodiment, as data is written, new data nodes can be continuously inserted into the skip list, thereby continuously updating the skip list. It should be noted that each data node is actually a string, which can be the same as or different from the corresponding data written. For example, in a key-value (KV) storage method, where K = user name and V = various user attributes (such as gender and transaction amount), the two actual data entries might be: Zhang Moumou, gender = male, transaction amount = 1000 yuan; Li Moumou, gender = female, transaction amount = 5000 yuan. In the above example, the strings corresponding to the two data nodes could be: Zhang Moumou, male; Li Moumou, female, or they could be: Zhang Moumou; Li Moumou, or they could be: Zhang Moumou, male, 1000 yuan; Li Moumou, female, 5000 yuan.

[0007] As illustrated by the examples above, the string used as a data node can be the same as the data being written, or it can be just a partial keyword from the data being written. In practical applications, the data to be stored can be in the form of text, numbers, or other types. For ease of description, the data format of the data nodes in this embodiment is uniformly described as a string. As mentioned above, when data begins to be written to a storage system, a corresponding skip list can be created. As data is continuously written and updated, the data nodes in the skip list are continuously added and updated. At the same time, in order to achieve efficient data access, the corresponding index linked lists in the skip list are also created and updated. During the process of generating the skip list, whenever a data node is inserted, the index level and insertion position corresponding to the data node need to be determined first. Then, a skip list containing multiple index linked lists and these data nodes is generated based on the index level corresponding to each inserted data node. Specifically, the index level corresponding to each data node can be obtained based on a set random function. Simply put, for a certain data node, a random value will be generated within a set range, and the index level corresponding to the data node is determined based on this random value. Assuming a data node has an index level of 2, this means there will be two levels of index information on that data node, located in two separate index linked lists. The method for determining the index level of a data node can be found in existing related technologies and will not be elaborated here. In this embodiment, the main improvement in skip list generation lies in the design and storage of the index information corresponding to each data node. To facilitate understanding of the skip list structure in this embodiment, examples are provided in conjunction with Figures 2 and 3a. Assuming the data written to the storage system consists of the following strings: ABCDXYZ. ABDXXZZ. ABDXYOXY. ABEOXZMN, initially, an empty head node a is created, containing an empty string (represented as ni l). When the first string ABCDXYZ is written to the storage system, the first data node b is created, with the string corresponding to the first string ABCDXYZ, and data node a points to data node b. Next, assuming the string written is ABDXXZZ, since the string ABDXXZZ is greater than ABCDXYZ, the data node c corresponding to the string ABDXXZZ is inserted after the data node b, and the data node b points to the data node c.Similarly, assuming the string to be written is ABDXYOXY, since ABDXYOXY is greater than ABDXXZZ, the data node d corresponding to the string ABDXYOXY is inserted after data node c, and data node c points to data node do. Assuming the string to be written is ABEOXZMN, since ABEOXZMN is greater than ABDXYOXY, the data node e corresponding to the string ABEOXZMN is inserted after data node d, and data node d points to data node e. oThis results in a typical linked list as shown in Figure 2. Understandably, in the example above, assuming the strings written after data node b are ABDXYOXY and ABDXXZZ, then since ABDXYOXY is greater than the string ABCDXYZ corresponding to data node b, ABDXYOXY will be placed after data node b, let's call it data node c. However, when the string ABDXXZZ is written, since ABDXXZZ is smaller than the string ABDXYOXY corresponding to data node c' and larger than the string ABDXYOXY corresponding to data node b, ABDXXZZ will be inserted between data node b and data node c', let's call it data node d. At this point, the order of the data nodes will be: data node a, data node b, data node d, data node c. Similar to the construction process of a regular linked list as shown in Figure 2, assuming that the index levels of each data node (a, b, c, d, e) are randomly generated as 3, 1, 2, 1, and 3 respectively, then a skip list structure as shown in Figure 3a will be formed. This structure includes three index linked lists arranged from top to bottom and multiple data nodes arranged sequentially as shown in the figure (which can be considered as the 0th level linked list). Since the index level of each data node is greater than or equal to 1, meaning it has at least one index level, the first-level index information of each data node forms the first-level index linked list shown in Figure 3a. The first-level index linked list includes the index information corresponding to each data node in the first-level index linked list shown in the figure, represented as: index information a1, index information b1, index information c1, index information d1, and index information el. Furthermore, based on the order of the corresponding data nodes, the pointing relationship of these index information is shown in Figure 3a: For example, in the first-level index linked list, the next data node after data node a is data node b. Since the index levels of data nodes a, c, and e are all greater than or equal to 2, meaning that these three data nodes all have a second-level index, the second-level index information of these three data nodes forms the second-level index linked list shown in Figure 3a.The second-level index linked list contains the index information corresponding to these three data nodes in this level of the index linked list, represented as: index information a2, index information c2, and index information e2. Furthermore, based on the order of the corresponding data nodes, the pointing relationship of these index information is shown in Figure 3a. For example, in the second-level index linked list, the next data node after data node a is data node c. Similarly, since the index level of data nodes a and e is both equal to 3, meaning both data nodes have a third-level index, the third-level index information of these two data nodes forms the third-level index linked list illustrated in Figure 3a. The third-level index linked list contains the index information corresponding to these two data nodes in this level of the index linked list, represented as: index information a3 and index information e3. Furthermore, based on the sorting of the corresponding data nodes, the pointing relationship of these index information is shown in Figure 3a: For example, in the third-level index linked list, the next data node of data node a is data node 6o. As shown in Figure 3a, any index linked list includes multiple index information arranged sequentially according to the sorting of the corresponding data nodes. The same data node may correspond to different index information in different index linked lists. In an optional embodiment, when generating a skip list in memory, the index information corresponding to the same data node can be stored together with the data node in the same memory space. The design of the index information in the embodiments of this disclosure will be described below. In summary, the index information includes an index feature value and a next data node position identifier (such as the subscript position of the data node stored in the memory pool). Taking two adjacent data nodes corresponding to any index linked list as an example, the index feature value corresponding to the previous data node of these two data nodes includes the length of the common prefix string of the two data nodes. And the "next data node" is the next data node corresponding to this data node in a certain level of the index linked list. Additionally, in an optional embodiment, the index feature value corresponding to the preceding data node further includes: a subsequent string of a set length that exists after the common prefix string in the following data node, such as a length of 3 characters (i.e., 3 bytes). Therefore, the index feature value is the feature value of the following data node relative to the preceding data node. As described above, the bottom layer of the skip list consists of multiple data nodes arranged sequentially, and above this are multiple index linked lists, each containing index information corresponding to different data nodes.The purpose of this index information is to enable faster retrieval of the required data nodes. Simply put, based on the index information corresponding to a specific data node in an indexed linked list, the size relationship between the desired data node and the current data node can be clearly determined, thus deciding the search direction: whether to search after or before the current data node. If the search is after the current data node, then the index information corresponding to the next data node needs to be read again for comparison. The index information corresponding to a data node can include index feature values ​​and the location information of the next data node it points to. If the index feature values ​​for each data node only consider the characteristics of the data (i.e., the string) stored in that data node—for example, even directly using the string as the index feature value to fill the index information—then, with different index information scattered throughout memory, frequent memory accesses will be involved during the top-down, layer-by-layer retrieval of the indexed linked list, increasing the number of reads and comparisons and resulting in significant overhead. Therefore, in this embodiment of the disclosure, for any two adjacent data nodes corresponding to any index linked list, the index feature value corresponding to the preceding data node includes the length of the common prefix string of the following data node relative to the preceding data node, thus introducing the feature of the following data node into the index feature value. That is to say, the index feature value in this embodiment of the disclosure is an index feature that considers the similarity between data nodes, and this similarity is manifested in the length of the common prefix of the two strings. Thus, when traversing to the index feature value of a certain data node, the positional relationship of the data node to be searched relative to the current data node and the next data node it points to in the current index linked list can be determined by this single index feature value, thereby helping to reduce the frequent access to memory during the data query process, achieving higher query efficiency and lower performance overhead. Taking the skip list illustrated in Figure 3a and the above-mentioned setting of a length of 3 characters as an example, the sorting of the data nodes corresponding to the first-level index linked list is: data node a, data node b, data node c, data node d, data node e. Then the process of determining the index information of each index in the first-level index linked list is as follows:

[0008] 1. The next data node after data node a is data node b. Since data node a is empty, the common prefix string between data node b and data node a is empty. Therefore, the length of the common prefix string is 0, that is, there is no common prefix string. Thus, the three characters after the common prefix string of data node b are ABC. Therefore, the index feature value of data node a in the first-level index linked list is (0, ABC). The position identifier of the next data node is the position identifier of data node b.

[0009] 2. The next data node after data node b is data node c. The common prefix string of data node b and data node c is AB, so the length of the common prefix string is 2. The three characters after the common prefix string of data node c are DXX. Therefore, the index feature value of data node b in the first-level index linked list is (2, DXX). The position identifier of the next data node is the position identifier of data node c.

[0010] 3. The next data node after data node c is data node d. The common prefix string of data node c and data node d is ABDX, so the length of the common prefix string is 4. The three characters after the common prefix string of data node d are YOX. Therefore, the index feature value of data node c in the first-level index linked list is (4, YOX). The position identifier of the next data node is the position identifier of data node d.

[0011] 4. The next data node after data node d is data node e. The common prefix string of data node d and data node e is AB, so the length of the common prefix string is 2. The three characters after the common prefix string of data node e are EOX. Therefore, the index feature value of data node c in the first-level index linked list is (2, EOX). The position identifier of the next data node is the position identifier of data node e.

[0012] 5. The next data node after data node e is empty, meaning data node e is the tail node. Since there are no subsequent data nodes after data node e, the index characteristic value of data node e in the first-level index linked list is (0, ni l), and the position identifier of the next data node is empty. Similarly, the order of the data nodes in the second-level index linked list is: data node a, data node c, data node e. The process of determining the index information in the second-level index linked list is as follows:

[0013] 1. The next data node after data node a is data node c. The common prefix string of data node c and data node a is empty, so the length of the common prefix string is 0, that is, there is no common prefix string. Therefore, the 3 characters after the common prefix string of data node c are ABD. So the index feature value of data node a in the second-level index linked list is (0, ABD). The position identifier of the next data node is the position identifier of data node c.

[0014] 2. The next data node after data node c is data node e. The common prefix string of data node c and data node e is AB, so the length of the common prefix string is 2. The three characters after the common prefix string of data node e are EOX. Therefore, the index feature value of data node c in the second-level index linked list is (2, EOX). The position identifier of the next data node is the position identifier of data node e.

[0015] 3. The next data node after data node e is empty, meaning data node e is the tail node. Since there are no subsequent data nodes after data node e, the index characteristic value of data node e in the second-level index linked list is (0, ni l), and the position identifier of the next data node is empty. Similarly, the order of data nodes in the third-level index linked list is: data node a, data node e. The process of determining the index information in the third-level index linked list is as follows:

[0016] 1. The next data node after data node a is data node e. The common prefix string of data node e and data node a is empty, so the length of the common prefix string is 0, that is, there is no common prefix string. Therefore, the 3 characters after the common prefix string of data node e are ABE. So the index feature value of data node a in the third-level index linked list is (0, ABE). The position identifier of the next data node is the position identifier of data node e.

[0017] 2. The next data node of data node e is empty, meaning data node e is the tail node. Since there are no subsequent data nodes after data node e, the index characteristic value of data node e in the third-level index linked list is (0, nil), and the position identifier of the next data node is empty. Based on the above generation process, the skip list shown in Figure 3b can be generated. However, the field containing the next data node position identifier in each index information is not shown in the skip list shown in Figure 3b. Furthermore, when a request to insert a new data node is received, the insertion position of the new data node and the index level of the new data node can be determined based on the comparison results between the new data node and multiple existing data nodes in the current skip list. The skip list is then updated based on this insertion position and the index level of the new data node, that is, the connection relationship between the new data node at the corresponding index level and the corresponding level of the skip list data nodes is updated. For example, continuing from the previous example, suppose a new data node f is added, with the string "ABDZYXX". Using a regular skip list lookup, the insertion position can be determined to be between data nodes d and e. Furthermore, assuming the index level of the determined data node f is 1, this means the first level of the indexed linked list needs to be updated. The update process is as follows:

[0018] 1. In the first-level indexed list, the next data node of data node d is data node f. The common prefix string of data node f and data node d is ABD, so the length of the common prefix string is 3. The three characters after the common prefix string of data node f are ZYX. Therefore, the index feature value of data node d in the first-level indexed list is (3, ZYX). The position identifier of the next data node is the position identifier of data node f.

[0019] 2. In the first-level indexed list, the next data node of data node f is data node e. The common prefix string of data node f and data node e is AB, so the length of the common prefix string is 2. The three characters after the common prefix string of data node e are EOX. Therefore, the index feature value of data node f in the first-level indexed list is (2, EOX). The position identifier of the next data node is the position identifier of data node e.

[0020] 3. In the first-level indexed linked list, during the insertion of data node f, data node e remains unchanged. The above process completes the creation and updating of the skip list in this embodiment. In production practice, the number of skip list elements can often be controlled. For example, if the maximum capacity of the skip list is expected to be one billion elements (1G), then only 30 bits are needed as the position identifier for each data node in the skip list. Thus, the remaining 34 bits of a 64-bit integer can be used to represent the index feature value, i.e., 64 bits are used to represent an index information. In the example above, 10 bits can be used to encode the length of the common prefix string, and the remaining 24 bits are used to encode the 3 characters (3 bytes = 24 bits) of the next data node after the common prefix string. Of course, if the length of the common prefix string might be longer or shorter, the number of bits used in each part can be adjusted as needed. Furthermore, the number of bits allocated to each part can be flexibly adjusted for different types of data. For example, for numeric data, only a maximum of 6 bits are needed to represent the length of the common bit portion. Based on the definition of the index feature value mentioned above, during the skip list creation process, the index feature value of each data node's subsequent data nodes relative to this data node is generated and recorded in the memory space corresponding to this data node. Then, during the skip list access process, this skip list structure design helps reduce the number of memory accesses and the number of cache misses. In other words, the same data node and its corresponding index information are stored in the same memory space, which reduces the number of memory accesses when accessing the skip list. In other words, when it is necessary to load certain data nodes and their corresponding index information from memory into the cache, the same data node and its corresponding index information have a higher probability of being loaded into the same cache line. Therefore, when querying target data based on the skip list, there is a higher probability that the index feature values ​​can be compared within the same cache line, thereby reducing the overhead of frequent accesses to different cache lines. The data access process based on this skip list is described below. Figure 4 is a flowchart of a data access method provided by an embodiment of this disclosure. As shown in Figure 4, the method includes the following steps:

[0021] 401. Receive a query request for the target data.

[0022] 402. Obtain a skip list containing multiple indexed linked lists. The skip list includes multiple data nodes corresponding to multiple strings, arranged in sequence. This skip list is established based on the scheme provided in the foregoing embodiments. Each indexed linked list includes multiple index information arranged sequentially according to the order of the corresponding data nodes. The index information includes index feature values ​​and the position identifier of the next data node. For any two adjacent data nodes corresponding to any indexed linked list, the index feature value corresponding to the preceding data node includes the length of the common prefix string of the two data nodes.

[0023] 403. Determine the first index feature value corresponding to the first data node that needs to be compared with the target data from the jump list, and determine the second index feature value of the target data relative to the first data node, wherein the first index feature value is the index feature value corresponding to the first data node in the first index linked list.

[0024] 404. Based on the comparison result between the second index feature value and the first index feature value, the target data is queried. In the traditional data query process, if only a regular data linked list (such as the linked list shown in Figure 2) is used, then it is necessary to compare the target data to be queried with each data node in the regular data linked list one by one to determine whether the regular data linked list contains the data node corresponding to the target data. Even if a skip list is constructed, due to the design of the index information, the index information in each index linked list is stored independently. In the process of accessing the index linked list layer by layer from top to bottom, frequent memory access is involved, resulting in a large overhead. However, in this embodiment of the disclosure, based on the design of the index feature value and the setting that the index information of each layer of the same data node is stored in the same memory space as the data node, the frequent memory access during the data query process can be reduced. In summary, querying target data in a skip list is an iterative process. Each iteration requires identifying the data node to be compared with the target data, its corresponding index feature value in the current index list, and generating the target data's index feature value relative to that data node. The comparison of these index feature values ​​determines which level of the index list and which data node to jump to in the next iteration. This iterative process continues until the target data query is complete. The query result is either finding the target data's corresponding data node in the skip list or determining that the skip list does not contain the target data's corresponding data node. As the example above shows, when the string length corresponding to a data node is very long, each index information of the data node only needs to occupy 64 bits, with the index feature value possibly occupying only 10-34 bits. Even considering only the length of the compared data, the overhead of comparing the strings of data nodes is higher than the overhead of comparing based on index feature values. Specifically, assuming the first index feature value determined from the skip list that needs to be compared with the target data is the first index feature value corresponding to the first data node, where the first index feature value is the index feature value corresponding to the first data node in the first index linked list that needs to be compared, then the second index feature value of the target data relative to the first data node is determined. Then, based on the comparison result between the second and first index feature values, the target data is queried. As described above regarding the skip list structure, the first index feature value includes the length of the first common prefix string between the second and first data nodes, where the second data node is the next data node in the first index linked list after the first data node.The second index feature value includes the length of the second common prefix string of the target data relative to the first data node. Based on this, the target data is queried according to the comparison result between the second index feature value and the first index feature value. Due to different comparison results, there are three possible cases: First, the length of the second common prefix string is greater than the length of the first common prefix string. This indicates that the target data has a longer second common prefix string compared to the first and second data nodes. Moreover, the string after the length of the first common prefix string of the second data node is larger than that of the first data node, indicating that the second data node is larger than the target data. Therefore, it is not necessary to compare the string content of the target data with that of the second data node. Instead, the process directly moves "down" from the current first index linked list to the next level of the index linked list, i.e., jumps to the index feature value corresponding to the first data node in the next index linked list, which is called the third index feature value. In other words, the third index feature value corresponding to the first data node is determined from the jump list. The third index feature value is the index feature value corresponding to the first data node in the second index linked list, which is the next level of the first index linked list. Next, the next iteration begins: Based on the comparison between the second and third index feature values, the target data is queried. In the second case, the length of the second common prefix string is less than the length of the first common prefix string. This indicates that the target data has a shorter second common prefix string than the first data node and the first data node, and the first common prefix string contains this second common prefix string. Therefore, the target data is larger than both the first and second data nodes. The process continues "to the right" along the current first index linked list, determining the fourth index feature value corresponding to the second data node (i.e., the index feature value corresponding to the second data node in the first index linked list) from the jump list. Since the data node being compared with the target data has changed from the first data node to the second data node, the second index feature value of the target data relative to the first data node needs to be updated to the fifth index feature value of the target data relative to the second data node. Then, the next iteration begins: Based on the comparison between the fifth and fourth index feature values, the target data is queried. In the third case, the length of the second common prefix string is equal to the length of the first common prefix string. This means that the target data and the next data node pointed to by the first data node in the first index linked list, i.e., the second data node, may be equal.Optionally, the second data node is read to compare the target data with the second data node. If the target data and the second data node are identical, the second data node is determined to be the query result. If they are inconsistent, it is necessary to decide whether to "go down" or "go right" based on the comparison result. Furthermore, as mentioned above, the index feature value can include not only the length of the common prefix string of the two data nodes, but also a subsequent string of a set length that exists after the common prefix string in the subsequent data node. Therefore, the first index feature value also includes the first subsequent string that exists after the first common prefix string in the second data node. Similarly, the second index feature value also includes the second subsequent string that exists after the second common prefix string in the target data. Based on this, the timing for reading the second data node can be: if the length of the second common prefix string is equal to the length of the first common prefix string, and the first subsequent string is the same as the second subsequent string, then the second data node is read to compare the target data with the second data node. In other words, if the two subsequent strings are the same, it means that the second data node is more likely to be the same as the target data. Only then is a comparison between the target data and the second data node performed. Otherwise, if the two subsequent strings are different, the size relationship between the target data and the second data node can be determined directly based on the comparison result of the two subsequent strings: if the first subsequent string is greater than the second subsequent string, it means that the second data node is greater than the target data; conversely, if the first subsequent string is less than the second subsequent string, it means that the second data node is less than the target data. Then, the subsequent query logic continues to be executed. Specifically: In the first sub-case, the target data is less than the second data node. In this case, it is not necessary to continue to jump from the first index feature value of the first data node to the index feature value of the second data node along the first index linked list, because the second data node and subsequent data nodes are all greater than the target data. At this time, it is necessary to "go down" from the first index linked list to jump to the third index feature value of the first data node in the next level of the second index linked list. Since the data node compared with the target data is still the first data node, it is not necessary to update the second index feature value of the target data. Proceed to the next iteration. Based on the comparison results between the second index feature value and the third index feature value, the target data is queried, which is the same as the first case mentioned above.In the second sub-case, if the target data is greater than the second data node, it means we need to search for the target data from a data node larger than the second data node. This requires continuing to "go right" along the first index linked list to jump from the first data node to the fourth index feature value corresponding to the next second data node. Since the comparison object changes from the first data node to the second data node, we need to determine the fifth index feature value of the target data relative to the second data node. Then, we proceed to the next iteration: based on the comparison result of the fifth index feature value and the fourth index feature value, we query the target data, similar to the second case described above. To facilitate understanding of the above query process, we will illustrate it with an example in Figure 3b. Assuming the target data S is the string ABDXYOXY corresponding to data node d, the query process is as follows: In the first round of query, the query starts from the head data node a of the skip list. Specifically, it starts by comparing the index feature value (0, ABE) corresponding to data node a in the third-level index linked list (higher level). First, the index feature value of the target data S relative to the head data node a is calculated: (0, ABD). Then, (0, ABE) is compared with (0, ABD). Since the length of the common prefix string is equal, and ABD is less than ABE, it is determined that the target data S < the next data node e under the third-level index linked list of data node a (this is the first sub-case mentioned above). Therefore, the process "goes down" and jumps to the index feature value (0, ABD) corresponding to data node a in the second-level index linked list. In the second round of querying: the index feature value (0, ABD) of target data S relative to data node a is the same as the index feature value (0, ABD) of data node a in the second-level index linked list. Since the next data node pointed to by data node a in the second-level index linked list is data node c, the size of data node c is compared with that of target data S. The comparison shows that target data S > data node c (this is the second sub-case mentioned above), so the process "goes right": jumps to the index feature value (2, EOX) of the next data node c pointed to by data node a in the second-level index linked list, and updates and calculates the index feature value (4, YOX) of target data S relative to data node c. o The third round of query: Compare the index feature value (4, YOX) of the target data S relative to data node c with the index feature value (2, EOX) of data node c in the second-level index linked list. The comparison result corresponds to the first case above: "go down", so jump to the index feature value (4, YOX) of data node c in the first-level index linked list.oThe fourth round of query: Compare the feature value (4, YOX) corresponding to the target data S with the index feature value (4, YOX) corresponding to data node c in the first-level index linked list. If they are equal, read the next data node d pointed to by data node c in the first-level index linked list. If data node d is found to be consistent with the target data S, the query ends. Since the same data node and its associated index information are stored in the same memory space, as shown in the example above, in the process of querying the target data S, only one jump is needed between the memory spaces of data node a and data node c to hit the target data S. However, if the query is based on the general data linked list shown in Figure 2, it requires three accesses to the memory space: from data node a to data node b, from data node b to data node c, and from data node c to data node d. In comparison, the number of memory accesses is reduced, and the access overhead is reduced. Intuitively, based on the skip list structure design of this disclosure, multiple index information (multiple index feature values ​​and the position pointer of the next data node) of a data node are stored together (in the same memory space). If the result of the connection is "go down", then it is not necessary to access the memory space pointed to by the position pointer of the next data node. However, without this skip list structure design, in order to compare, whether going down or right, different memory spaces need to be accessed based on the position pointer, which may lead to cache misses. In the above scheme, the same data node and its corresponding index information are stored in the same memory space. When accessing the skip list, the number of memory accesses can be reduced. In other words, when it is necessary to load certain data nodes and their corresponding index information from memory into the cache, the same data node and its corresponding index information have a higher probability of being loaded into the same cache line. Therefore, when querying target data based on the skip list, there is a higher probability that the index feature values ​​can be compared in the same cache line, thereby reducing the overhead of frequent access to different cache lines. Furthermore, querying target data by comparing index feature values ​​achieves higher query efficiency and lower performance overhead compared to directly comparing the target data sequentially with each data node in a regular data linked list. The data access and index building apparatus of one or more embodiments of this disclosure will be described in detail below. Those skilled in the art will understand that these apparatuses can be configured using commercially available hardware components through the steps taught in this solution.Figure 5 is a schematic diagram of an index building device provided in an embodiment of this disclosure. As shown in Figure 5, the device includes: a determining module 11 and a generating module 12. The determining module 11 is used to determine multiple data nodes corresponding to multiple strings in a sequential arrangement. The generating module 12 is used to determine the index level corresponding to each of the multiple data nodes, and generate a skip list containing multiple index linked lists based on the index level corresponding to each of the multiple data nodes. Each index linked list includes multiple index information arranged sequentially according to the order of the corresponding data nodes. The index information includes an index feature value and a next data node position identifier. For any two adjacent data nodes corresponding to any index linked list, the index feature value corresponding to the preceding data node includes the length of the common prefix string of the two data nodes. In an optional embodiment, the device further includes: a storage module 13, used to store any data node and at least one index information corresponding to the any data node in the same memory space, wherein at least one index information corresponding to any data node corresponds one-to-one with at least one level of the index linked list corresponding to the any data node. Optionally, the index feature value corresponding to the previous data node further includes: a subsequent string of a set length existing after the common prefix string in the next data node. Optionally, the determining module 11 is further configured to: in response to the insertion request of a new data node, determine the insertion position of the new data node based on the comparison result between the new data node and the plurality of data nodes. Correspondingly, the generating module 12 is further configured to: determine the index level of the new data node, and update the skip list based on the insertion position and the index level of the new data node. The device shown in Figure 5 can execute the steps of the index building method in the foregoing embodiments. For detailed execution process and technical effects, please refer to the description in the foregoing embodiments, which will not be repeated here. Figure 6 is a schematic diagram of the structure of a data access device provided in an embodiment of this disclosure. As shown in Figure 6, the device includes: a receiving module 21, an acquiring module 22, and a querying module 23. The receiving module 21 is configured to receive a query request for target data.Acquisition module 22 is used to acquire a skip list containing multiple index linked lists. Each skip list includes multiple data nodes corresponding to multiple strings arranged sequentially. Each index linked list includes multiple index information arranged in order of the corresponding data nodes. The index information includes an index feature value and a next data node position identifier. For any two adjacent data nodes corresponding to any index linked list, the index feature value corresponding to the preceding data node includes the length of the common prefix string of the two data nodes. Query module 23 is used to determine from the skip list the first index feature value corresponding to the first data node to be compared with the target data, determine the second index feature value of the target data relative to the first data node, and perform a query for the target data based on the comparison result of the second index feature value and the first index feature value. The first index feature value is the index feature value corresponding to the first data node in the first index linked list. The device shown in Figure 6 can execute the steps of the data access method in the foregoing embodiments. Detailed execution processes and technical effects are described in the foregoing embodiments and will not be repeated here. In one possible design, the structure of the device shown in Figures 5 and 6 can be implemented as an electronic device. As shown in Figure 7, the electronic device may include: a processor 31, a memory 32, and a communication interface 33. The memory 32 stores executable code. When the executable code is executed by the processor 31, the processor 31 can at least implement the data access and indexing methods provided in the foregoing embodiments. Additionally, this disclosure provides a non-transitory machine-readable storage medium storing executable code. When the executable code is executed by the processor of the electronic device, the processor can at least implement the data access and indexing methods provided in the foregoing embodiments. The device embodiments described above are merely illustrative. The network elements described as separate components may or may not be physically separate. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without creative effort. Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of necessary general-purpose hardware platforms, or by a combination of hardware and software.Based on this understanding, the above-mentioned technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a computer product. This disclosure can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code. Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this disclosure, and not to limit them; although this disclosure has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this disclosure.

Claims

Claims 1. An index creation method, wherein, include: Determine the sequential arrangement of multiple data nodes corresponding to multiple strings; Determine the index layer number corresponding to each of the plurality of data nodes; Based on the index level corresponding to each of the multiple data nodes, a skip list containing multiple index linked lists is generated. Each index linked list includes multiple index information arranged sequentially according to the order of the corresponding data nodes. The index information includes an index feature value and the position identifier of the next data node. For any two adjacent data nodes corresponding to any index linked list, the index feature value corresponding to the preceding data node includes the length of the common prefix string of the two data nodes.

2. The method according to claim 1, wherein, The method further includes: storing any data node and at least one index information corresponding to the data node in the same memory space, wherein at least one index information corresponding to the data node corresponds to at least one level of the index linked list corresponding to the data node.

3. The method according to claim 1 or 2, wherein, The index feature value corresponding to the previous data node also includes: a subsequent string of a set length that exists after the common prefix string in the next data node.

4. The method according to any one of claims 1-3, wherein, The method further includes: in response to an insertion request for a new data node, determining the insertion position of the new data node based on a comparison result between the new data node and the plurality of data nodes; determining the index level of the new data node; and updating the skip list based on the insertion position and the index level of the new data node.

5. A data access method, wherein, include: Receive query requests for target data; Obtain a skip list containing multiple indexed linked lists, each skip list including multiple data nodes corresponding to multiple strings arranged sequentially. Each indexed linked list includes multiple index information items arranged in order of the corresponding data nodes, and each index information item contains an index feature value and a next data node position identifier. For any two adjacent data nodes corresponding to any indexed linked list, the index feature value corresponding to the preceding data node includes the length of the common prefix string of the two data nodes. Determine the first index feature value corresponding to the first data node to be compared with the target data from the skip list, wherein the first index feature value is the index feature value corresponding to the first data node in the first indexed linked list. Determine the second index feature value of the target data relative to the first data node. Based on the comparison result between the second index feature value and the first index feature value, perform a query on the target data.

6. The method according to claim 5, wherein, The first index feature value includes the length of the first common prefix string of the second data node relative to the first data node, and the second data node is the next data node of the first data node in the first index linked list; The second index feature value includes the length of the second common prefix string of the target data relative to the first data node; The target is then determined based on the comparison result between the second index feature value and the first index feature value. The data query includes: if the length of the second common prefix string is greater than the length of the first common prefix string, then determining the third index feature value corresponding to the first data node from the skip list, wherein the third index feature value is the index feature value corresponding to the first data node in the second index linked list, and the second index linked list is the next level index linked list of the first index linked list; and querying the target data based on the comparison result between the second index feature value and the third index feature value.

7. The method according to claim 6, wherein, The step of querying the target data based on the comparison result between the second index feature value and the first index feature value includes: if the length of the second common prefix string is less than the length of the first common prefix string, then determining the fourth index feature value corresponding to the second data node from the skip list; determining the fifth index feature value of the target data relative to the second data node; and querying the target data based on the comparison result between the fifth index feature value and the fourth index feature value.

8. The method according to claim 6 or 7, wherein, The step of querying the target data based on the comparison result between the second index feature value and the first index feature value includes: if the length of the second common prefix string is equal to the length of the first common prefix string, then reading the second data node to compare the target data with the second data node; if the target data is consistent with the second data node, then determining the second data node as the query result.

9. The method according to claim 8, wherein, The first index feature value also includes a first subsequent string existing after the first common prefix string in the second data node, and the second index feature value also includes a second subsequent string existing after the second common prefix string in the target data; the step of reading the second data node to compare the target data with the second data node if the length of the second common prefix string is equal to the length of the first common prefix string includes: if the length of the second common prefix string is equal to the length of the first common prefix string, and the first subsequent string is the same as the second subsequent string, then reading the second data node to compare the target data with the second data node; if the first subsequent string is different from the second subsequent string, then determining the comparison result O between the target data and the second data node based on the comparison result of the first subsequent string and the second subsequent string.

10. The method according to claim 8 or 9, wherein, The method further includes: if the target data is less than the second data node, then determining the third index feature value corresponding to the first data node from the skip list, wherein the third index feature value is the index feature value corresponding to the first data node in the second index linked list, and the second index linked list is the next level index linked list of the first index linked list; and querying the target data based on the comparison result between the second index feature value and the third index feature value.

11. The method according to claim 8 or 9, wherein, The method further includes: if the target data is greater than the second data node, then determining the second data node from the skip list. The corresponding fourth index feature value; determine the fifth index feature value of the target data relative to the second data node; and perform a query on the target data based on the comparison result between the fifth index feature value and the fourth index feature value.

12. The method according to any one of claims 5-11, wherein, Any data node and at least one index information corresponding to the data node are stored in the same memory space, wherein at least one index information corresponding to the data node corresponds one-to-one with at least one level of the index linked list corresponding to the data node.

13. An electronic device, wherein, include: The system includes a memory, a processor, and a communication interface; wherein the memory stores executable code, which, when executed by the processor, causes the processor to perform an index building method as described in any one of claims 1 to 4, or a data access method as described in any one of claims 5 to 12.

14. A non-transitory machine-readable storage medium, wherein, The non-transitory machine-readable storage medium stores executable code that, when executed by a processor of an electronic device, causes the processor to perform an indexing method as described in any one of claims 1 to 4, or a data access method as described in any one of claims 5 to 12.

15. A computer program product, wherein, include: A computer program, when executed by a processor of an electronic device, causes the processor to perform the index building method as described in any one of claims 1 to 4, or the data access method as described in any one of claims 5 to 12. 17

Images