A data storage method and apparatus

Abstract

This invention discloses a data storage method and apparatus, relating to the field of computer technology. One specific embodiment of the method includes: calculating the promotion probability of inserting data into a skip list; in response to a user's storage operation on target data, inserting the target data into the skip list according to the promotion probability; and persisting the skip list to a front-end database; wherein the front-end database is located on a local disk, and the skip list is stored in memory. This embodiment can solve the technical problems of low query efficiency and inability to persist storage.

Description

Technical Field

[0001] This invention relates to the field of computer technology, and in particular to a data storage method and apparatus. Background Technology

[0002] Currently, the following methods are commonly used to store front-end user data: arrays, linked list data structures, and browser-related storage using localStorage.

[0003] In the process of realizing this invention, the inventors discovered at least the following problems in the prior art:

[0004] Arrays require contiguous memory addresses, and adding or deleting data requires moving memory, resulting in low query efficiency and the inability to be stored persistently. Linked list data structures have a query time complexity of O(n), resulting in low query efficiency and the inability to be stored persistently. LocalStorage has memory size limitations, and the data structures stored are not clearly related, requiring users to design their own data structures. Summary of the Invention

[0005] In view of this, embodiments of the present invention provide a data storage method and apparatus to solve the technical problems of low query efficiency and inability to persist storage.

[0006] To achieve the above objectives, according to one aspect of the present invention, a data storage method is provided, comprising:

[0007] Calculate the promotion probability of inserting data into the skip list;

[0008] In response to a user's storage operation on target data, the target data is inserted into the jump list according to the promotion probability;

[0009] Persist the skip list to the front-end database;

[0010] The front-end database is located on the local disk, and the skip list is stored in memory.

[0011] Optionally, calculating the promotion probability of inserting data into the skip list includes:

[0012] Based on the number of levels in the skip list and the cross-level value, determine the target level from which the promotion probability is to be calculated;

[0013] Calculate the promotion probability of data in the target layer when inserting data into the bottom linked list of the jump list.

[0014] Optionally, in response to a user's storage operation on the target data, inserting the target data into the skip list according to the promotion probability includes:

[0015] In response to the user's storage operation on the target data, the target node corresponding to the target data is inserted into the underlying linked list of the skip linked list;

[0016] Determine whether the target node needs to be promoted based on the promotion probability; if so, insert the target node into the indexed linked list of the skip list.

[0017] Optionally, inserting the target node corresponding to the target data into the underlying linked list of the skip list includes:

[0018] Determine whether the number of nodes in the underlying linked list of the skip list is greater than the node count threshold;

[0019] If not, the target node corresponding to the target data is inserted at the end of the underlying linked list;

[0020] If so, then find the insertion position in the underlying linked list and insert the target node corresponding to the target data into the insertion position.

[0021] Optionally, finding the insertion position in the underlying linked list and inserting the target node corresponding to the target data into the insertion position includes:

[0022] The insertion position is determined by traversing the index values ​​of each node in the underlying linked list.

[0023] Determine whether the span between the index values ​​of two nodes adjacent to the insertion position is greater than the index value span threshold;

[0024] If so, the target node corresponding to the target data is inserted between the two nodes;

[0025] If not, the target node corresponding to the target data is inserted at the end of the underlying linked list.

[0026] Optionally, inserting the target node into the indexed linked list of the skip list includes:

[0027] For each level of the skip list, the index value of each node in the index list is traversed to insert the target node into the index list.

[0028] Based on the promotion probability, determine whether the target node needs to continue to be promoted; if so, insert the target node into the next level of the indexed linked list until the target node no longer needs to be promoted.

[0029] Optionally, before calculating the promotion probability of inserting data into the skip list, the method further includes:

[0030] Retrieve the skip list from the front-end database and load the skip list into memory.

[0031] Additionally, according to another aspect of the present invention, a data storage device is provided, comprising:

[0032] The calculation module is used to calculate the promotion probability of inserting data into the skip list;

[0033] An insertion module is used to insert the target data into the jump list according to the promotion probability in response to a user's storage operation on the target data;

[0034] The persistence module is used to persist the skip list to the front-end database;

[0035] The front-end database is located on the local disk, and the skip list is stored in memory.

[0036] Optionally, the computing module is further configured to:

[0037] Based on the number of levels in the skip list and the cross-level value, determine the target level from which the promotion probability is to be calculated;

[0038] Calculate the promotion probability of data in the target layer when inserting data into the bottom linked list of the jump list.

[0039] Optionally, the insertion module is further configured to:

[0040] In response to the user's storage operation on the target data, the target node corresponding to the target data is inserted into the underlying linked list of the skip linked list;

[0041] Determine whether the target node needs to be promoted based on the promotion probability; if so, insert the target node into the indexed linked list of the skip list.

[0042] Optionally, the insertion module is further configured to:

[0043] Determine whether the number of nodes in the underlying linked list of the skip list is greater than the node count threshold;

[0044] If not, the target node corresponding to the target data is inserted at the end of the underlying linked list;

[0045] If so, then find the insertion position in the underlying linked list and insert the target node corresponding to the target data into the insertion position.

[0046] Optionally, the insertion module is further configured to:

[0047] The insertion position is determined by traversing the index values ​​of each node in the underlying linked list.

[0048] Determine whether the span between the index values ​​of two nodes adjacent to the insertion position is greater than the index value span threshold;

[0049] If so, the target node corresponding to the target data is inserted between the two nodes;

[0050] If not, the target node corresponding to the target data is inserted at the end of the underlying linked list.

[0051] Optionally, the insertion module is further configured to:

[0052] For each level of the skip list, the index value of each node in the index list is traversed to insert the target node into the index list.

[0053] Based on the promotion probability, determine whether the target node needs to continue to be promoted; if so, insert the target node into the next level of the indexed linked list until the target node no longer needs to be promoted.

[0054] Optionally, a loading module is also included for:

[0055] Before calculating the promotion probability of inserting data into the skip list, the skip list is retrieved from the front-end database and loaded into memory.

[0056] According to another aspect of the present invention, an electronic device is also provided, comprising:

[0057] One or more processors;

[0058] Storage device for storing one or more programs.

[0059] When the one or more programs are executed by the one or more processors, the one or more processors implement the method described in any of the above embodiments.

[0060] According to another aspect of the present invention, a computer-readable medium is also provided, on which a computer program is stored, which, when executed by a processor, implements the methods described in any of the above embodiments.

[0061] According to another aspect of the present invention, a computer program product is also provided, including a computer program that, when executed by a processor, implements the methods described in any of the above embodiments.

[0062] One embodiment of the above invention has the following advantages or beneficial effects: because it employs a technique of first calculating the promotion probability of inserting data into the skip list, then inserting the target data into the skip list according to the promotion probability, and finally persisting the skip list in memory to the front-end database on the local disk, it overcomes the technical problems of low query efficiency and inability to persist storage in the prior art. This embodiment of the invention uses a skip list to store data to achieve hierarchical skip queries, thereby reducing query time complexity; moreover, by calculating the promotion probability of inserting data into the skip list before insertion, it improves the data structure storage efficiency.

[0063] The further effects of the aforementioned unconventional alternative methods will be explained below in conjunction with specific implementation methods. Attached Figure Description

[0064] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. Wherein:

[0065] Figure 1 This is a flowchart of a data storage method according to an embodiment of the present invention;

[0066] Figure 2 This is a schematic diagram of the structure of a skip list according to an embodiment of the present invention;

[0067] Figure 3 This is a flowchart of a data storage method according to a possible embodiment of the present invention;

[0068] Figure 4 This is a schematic diagram of a data storage tool according to an embodiment of the present invention;

[0069] Figure 5 This is a schematic diagram of a data storage device according to an embodiment of the present invention;

[0070] Figure 6 This is an exemplary system architecture diagram in which embodiments of the present invention can be applied;

[0071] Figure 7 This is a schematic diagram of the structure of a computer system suitable for implementing terminal devices or servers of the present invention. Detailed Implementation

[0072] The following description, in conjunction with the accompanying drawings, illustrates exemplary embodiments of the present invention, including various details to aid understanding. These details should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the invention. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0073] Figure 1 This is a flowchart of a data storage method according to an embodiment of the present invention. As one embodiment of the present invention, such as... Figure 1 As shown, the data storage method may include:

[0074] Step 101: Calculate the promotion probability of inserting the data into the skip list.

[0075] A skip list is a multi-level singly linked list, a linked list structure that trades space for time, and is stored in memory. Figure 2 This is a schematic diagram of the structure of a skip list according to an embodiment of the present invention, as shown below. Figure 2 As shown, a skip list consists of a bottom-level linked list and several levels of indexed linked lists (first-level indexed linked list, second-level indexed linked list, third-level indexed linked list, etc.). When the number of linked list levels of the skip list exceeds a certain number (such as 5, 8, or 10 levels), the promotion algorithm preprocess can be triggered, that is, the promotion probability of inserting data into the skip list is calculated before inserting data, thereby improving data storage efficiency.

[0076] Optionally, step 101 may include: determining the target layer for which the promotion probability is to be calculated based on the number of levels and the span value of the skip list; and calculating the promotion probability of the data in the target layer when the data is inserted into the bottom-level linked list of the skip list. For example, if the span value is 4, then when the number of levels in the storage structure of the skip list exceeds 5, the number of levels in the skip list (a total of 7 levels) is divided by the span value to calculate the target layers as the first level (i.e., the bottom-level linked list) and the fifth level (i.e., the fourth-level indexed linked list). The promotion probability of the data in the first and fifth levels is calculated based on the span value, which can improve data storage efficiency. Similarly, if the span value is 3 and the number of levels in the skip list is 10, then the target layers are the first, fourth, seventh, and tenth levels (from bottom to top), and the promotion probability of the data in the first, fourth, seventh, and tenth levels is calculated based on the span value.

[0077] Furthermore, if the calculated promotion probability is less than the promotion probability threshold, the promotion probability is set to zero. This can effectively control the space waste caused by insufficient data and excessively high levels.

[0078] Alternatively, the promotion algorithm is as follows:

[0079] The probability of a node's promotion = 1 - p * (1 - (index of the previous node - index of the previous promoted node - 1) / index span)

[0080] Where p is a random number between 0 and 1.

[0081] If p is 0.5, then the promotion probability of a node is 50%. We can set the index value span to 5 to ensure that at least one of the 5 nodes is successfully promoted.

[0082] For example: If nodes 1, 2, 3, 4, and 5 are already in the underlying linked list and node 5 has been successfully promoted, then:

[0083] The promotion probability of inserting node 6 = 1 - 0.5 * (1 – (6 - 5 - 1) / 5) = 50%;

[0084] The promotion probability of inserting node 7 = 1 - 0.5 * (1 – (7 - 5 - 1) / 5) = 60%;

[0085] The promotion probability of inserting node 8 = 1 - 0.5 * (1 – (8 - 5 - 1) / 5) = 70%;

[0086] ...

[0087] The promotion probability of inserting node 11 is 1 - 0.5 * (1 – (11 - 5 - 1) / 5) = 100%.

[0088] The promotion probability of compute nodes in the index linked list is similar and will not be elaborated further.

[0089] Step 102: In response to the user's storage operation on the target data, insert the target data into the jump list according to the promotion probability.

[0090] When a user performs a storage operation on target data, the target data is inserted into the jump list according to the promotion probability of each target layer calculated in step 101.

[0091] Optionally, step 102 may include: in response to a user's storage operation on the target data, inserting the target node corresponding to the target data into the bottom-level linked list of the skip list; determining whether the target node needs to be promoted based on the promotion probability; if so, inserting the target node into the index linked list of the skip list. In this step, in response to a user's storage operation on the target data, the target node corresponding to the target data is first inserted into the bottom-level linked list, such as... Figure 2As shown, then based on the promotion probability of the target node in each target layer calculated in step 101, it is determined whether the target node needs to be promoted. If so, the target node is inserted into the first-level index linked list. Next, based on the promotion probability of the target node in each target layer calculated in step 101, it is determined whether the target node needs to be promoted. If so, the target node is inserted into the second-level index linked list, and so on, until promotion is no longer needed.

[0092] Optionally, inserting the target node corresponding to the target data into the bottom linked list of the skip list includes: determining whether the number of nodes in the bottom linked list of the skip list is greater than a node count threshold; if not, inserting the target node corresponding to the target data into the tail of the bottom linked list; if so, finding an insertion position in the bottom linked list and inserting the target node corresponding to the target data into that insertion position. If the node count threshold is 3, determining whether the number of nodes in the bottom linked list is greater than 3; if not, directly inserting the target node corresponding to the target data into the tail of the bottom linked list; if so, finding an insertion position in the bottom linked list and inserting the target node corresponding to the target data into that insertion position. It should be noted that the node count threshold can be set as needed, and this embodiment of the invention does not impose any restrictions on it.

[0093] Optionally, finding the insertion position in the underlying linked list and inserting the target node corresponding to the target data into the insertion position includes: traversing the index values ​​of each node in the underlying linked list to determine the insertion position; determining whether the index value span between two nodes adjacent to the insertion position is greater than an index value span threshold; if yes, inserting the target node corresponding to the target data between the two nodes; if no, inserting the target node corresponding to the target data at the end of the underlying linked list. When inserting a target node into the underlying linked list, first traversing the index values ​​of each node in the underlying linked list (the index values ​​of each node increase sequentially) to determine the insertion position (e.g., between node 1 and node 9), then determining whether the index value span between two nodes adjacent to the insertion position (e.g., the index value span between node 1 and node 9) is greater than an index value span threshold; if yes, inserting the target node corresponding to the target data after node 1 (which could be the position of node 2, node 3, or node 4, and this embodiment of the invention does not limit this); if no, inserting the target node corresponding to the target data at the end of the underlying linked list.

[0094] It's important to note that while each node insertion creates an index value, users can delete nodes, leading to large gaps between the index values ​​of two nodes. An index value span threshold can be preset, for example, by multiplying the skip list's depth by 3. Therefore, as the skip list's depth increases, the index value span threshold will also increase.

[0095] Optionally, inserting the target node into the index linked list of the skip list includes: for each level of the skip list, traversing the index values ​​of each node in the index linked list to insert the target node into the index linked list; determining whether the target node needs to continue promotion based on the promotion probability; if so, inserting the target node into the next higher level index linked list until the target node no longer needs promotion. The process of inserting the target node into each level of the index linked list is similar to the process of inserting the target node into the bottom level linked list. Taking the first-level index linked list as an example, traversing the index values ​​of each node in the first-level index linked list to insert the target node into the index linked list, so that the index values ​​of each node increase sequentially, and then determining whether the target node needs to be promoted based on the promotion probability of each target level calculated in step 101. If so, inserting the target node into the second-level index linked list, and then again determining whether the target node needs to be promoted based on the promotion probability of each target level calculated in step 101. If so, inserting the target node into the third-level index linked list until the target node no longer needs promotion.

[0096] It should be noted that if the total number of levels in the skip list is 7, the calculation result in step 101 shows that the promotion probability of the first and fifth levels is greater than or equal to the promotion probability threshold. This indicates that the target node needs to be promoted after being inserted into the first and fifth levels. Therefore, to improve data storage efficiency, after inserting the target node into the first, second, third, fourth, and fifth levels, it is not necessary to determine whether the target node needs to be promoted based on the promotion probability. Instead, the target node is directly determined to need to be promoted, because if the target node needs to be promoted in the fifth level, then the target node must need to be promoted in the first, second, third, and fourth levels. Therefore, the target node can be directly determined to need to be promoted. However, after inserting the target node into the sixth level, the promotion algorithm needs to be called to determine whether the target node needs to be promoted. The calculation process of the promotion algorithm is similar to the calculation process described in step 101, and will not be repeated here.

[0097] Step 103: Persist the skip list to the front-end database. The front-end database is located on the local disk.

[0098] After inserting data into the skip list, the skip list is persisted to the front-end database. To facilitate loading data from the front-end database into memory in subsequent steps, the entire skip list can be directly persisted to the front-end database. Therefore, the front-end database stores the entire processed list hierarchy. Optionally, the front-end database can be an IndexDB database.

[0099] When a user queries a node, the data is compared starting from the top level of the skip list to find the region, and then the lower-level list is searched. This process of comparing data at different levels continues until the node data is found. This reduces the query time complexity to O(logn).

[0100] Based on the various embodiments described above, it can be seen that the embodiments of the present invention solve the technical problems of low query efficiency and inability to persist storage in the prior art by first calculating the promotion probability of inserting data into the skip list, then inserting the target data into the skip list according to the promotion probability, and finally persisting the skip list in memory to the front-end database on the local disk. The embodiments of the present invention use skip lists to store data to achieve hierarchical skip queries, thereby reducing query time complexity; moreover, by calculating the promotion probability of inserting data into the skip list in advance before inserting data, the data structure storage efficiency is improved.

[0101] Figure 3 This is a flowchart of a data storage method according to a possible embodiment of the present invention. As another embodiment of the present invention, such as... Figure 3 As shown, the data storage method may include:

[0102] Step 301: Obtain the skip list from the front-end database and load the skip list into memory. The front-end database is located on the local disk.

[0103] Browser memory is cleared when the page is refreshed. The browser will first synchronize the skip list stored in the front-end database from the local disk to memory.

[0104] It should be noted that since the front-end database stores all the already processed linked list hierarchical structures, the link relationships do not need to be recalculated; the skip linked lists stored in the front-end database are directly loaded into memory.

[0105] Step 302: Check if the number of levels in the skip list is greater than the level threshold; if yes, proceed to step 303; if no, proceed to step 305.

[0106] You can preset a level threshold, such as 3, 5, 7 or 12 levels, to determine whether the current level of the skip list is greater than the level threshold.

[0107] Step 303: Calculate the promotion probability of inserting the data into the skip list.

[0108] To improve data storage efficiency, if the current level of the skip list is greater than a preset level threshold, the promotion algorithm preprocess is triggered. This means that the promotion probability of inserting data into the skip list is calculated before data is inserted, thereby improving data storage efficiency.

[0109] Specifically, based on the number of levels in the skip list and the cross-level value, the target level for calculating the promotion probability is determined. The promotion probability of data in the target level when inserted into the bottom-level list of the skip list is calculated. For example, if the cross-level value is 3, then when the storage structure of the skip list exceeds 5 levels, the number of levels in the skip list (a total of 8 levels) is divided by the cross-level value to calculate the target levels as the first level (i.e., the bottom-level list), the fourth level (i.e., the third-level index list), and the seventh level (i.e., the sixth-level index list). Therefore, the promotion probability of data in the first, fourth, and seventh levels is calculated based on the cross-level value, which improves data storage efficiency.

[0110] Step 304: In response to the user's storage operation on the target data, insert the target data into the jump list according to the promotion probability.

[0111] When a user performs a storage operation on target data, the target data is inserted into the jump list according to the promotion probability of each target layer calculated in step 303.

[0112] In this step, in response to the user's storage operation on the target data, the target node corresponding to the target data is first inserted into the underlying linked list. Then, based on the promotion probability of the target node in each target layer (such as the first layer, the fourth layer, and the seventh layer) calculated in step 303, it is determined whether the target node needs to be promoted. If so, the target node is inserted into the first-level index linked list. Next, based on the promotion probability of the target node in each target layer calculated in step 303, it is determined again whether the target node needs to be promoted. If so, the target node is inserted into the second-level index linked list, and so on, until promotion is no longer needed.

[0113] It should be noted that if the calculation result of step 303 shows that the promotion probability of the first and fourth levels is greater than or equal to the promotion probability threshold, it means that the target node needs to be promoted after being inserted into the first and fourth levels. Therefore, to improve data storage efficiency, after inserting the target node into the first, second, third, and fourth levels, it is not necessary to determine whether the target node needs to be promoted based on the promotion probability. Instead, the target node is directly determined to need to be promoted, because if the target node needs to be promoted in the fourth level, then the target node must need to be promoted in the first, second, and third levels. Therefore, the target node can be directly determined to need to be promoted. However, after inserting the target node into the fifth level, it is necessary to call the promotion algorithm to determine whether the target node needs to be promoted.

[0114] Step 305: In response to the user's storage operation on the target data, insert the target data into the skip list.

[0115] If the current level of the skip list is less than or equal to a preset level threshold, the promotion algorithm preprocess does not need to be triggered. Instead, when the user performs a storage operation on the target data, the promotion algorithm is invoked to insert the target node corresponding to the target data into the skip list.

[0116] Step 306: Persist the skip list to the front-end database.

[0117] After inserting data into the skip list, the skip list is persisted to the front-end database. To facilitate loading data from the front-end database into memory in subsequent steps, the entire skip list can be directly persisted to the front-end database. Therefore, the front-end database stores the entire processed list hierarchy. Optionally, the front-end database can be an IndexDB database.

[0118] Furthermore, the specific implementation details of the data storage method in one of the reference embodiments of the present invention have been described in detail in the data storage method described above, so the details will not be repeated here.

[0119] Figure 4 This is a schematic diagram of a data storage tool according to an embodiment of the present invention. Figure 4 As shown, the data storage tool includes a functional layer, an algorithm layer, a built-in processor, and a persistence controller. The functional layer provides functions such as querying, modifying, deleting, inserting, and configuring. The algorithm layer uses a promotion algorithm to calculate the promotion probability of nodes. The built-in processor includes a task scheduler, an asynchronous arithmetic unit, and an initialization loader.

[0120] The process of using this data storage tool includes:

[0121] Step 1: Plugin Installation and Plugin Referencing

[0122] Step 1.1 Users can install data storage tools via npm.

[0123] Step 1.2: Reference it in the project or via a script.<scriptsrc="https: / / cdn.zhangyun.com / bigdatatools.js">

[0124] Step 2: Use the data storage tool and initialize the configuration.

[0125] Step 2.1 Initialize the data storage tool installed in Step 1 in the front-end project. The tool provides: initialization configuration, whether to enable persistence mode, whether to replace the promotion algorithm, etc.

[0126] If persistent storage is enabled, the system initially retrieves the skip list from the browser's front-end database (such as IndexDB) and loads it into memory. Browser memory is cleared when the page is refreshed. If persistent storage is enabled, the data storage tool synchronizes a copy to the browser's built-in front-end database every time the user inserts data, ensuring persistent data storage. When the user refreshes the page, the data storage tool first synchronizes the data stored before the refresh from the front-end database.

[0127] Step 3: Node Insertion Workflow

[0128] Step 3.1 The user calls the built-in insertion method of the data storage tool to create a node and determines whether the number of nodes is greater than 3; if it is greater than 3, proceed to step 3.2; otherwise, insert a node at the end of the underlying linked list.

[0129] Step 3.2 Find the insertion position of the node in the underlying linked list. If the index value span of two nodes adjacent to the insertion position is greater than the index value span threshold, then insert the node between these two nodes; otherwise, insert the node at the tail of the underlying linked list.

[0130] Step 3.3 After inserting a node at the tail of the underlying linked list, the promotion algorithm is called to determine whether the node needs to be promoted. If it does, step 3.4 is executed; otherwise, the process ends.

[0131] Step 3.4 If the node inserted in the underlying linked list needs to be promoted, then insert the node in the first-level indexed linked list, i.e., repeat steps 3.2 and 3.3.

[0132] If the user enables persistent mode, proceed to step 4.1; otherwise, proceed to step 5.1.

[0133] Step 4: Persistent storage procedure

[0134] Step 4.1 If the user enables persistent mode, the data storage tool will notify the asynchronous processor to create an asynchronous task to deliver the latest data to the persistent controller every time the storage structure is operated. The asynchronous queue will not interrupt the user's synchronous modification operation.

[0135] Step 4.2 Each time an asynchronous task is created, the offset in the processor is incremented by 1. When the user's synchronous task ends, the processor will obtain the latest data for synchronization based on the offset. For previous task cancellation operations, simply hand over the latest data to the persistence controller.

[0136] Step 4.3 The persistence controller will use the browser's requestIdleCallback function to process the data and synchronize it to the front-end database.

[0137] Step 5: Algorithm Pre-process

[0138] Step 5.1 When the skip list has more than 5 levels, the data storage tool will initiate the promotion algorithm pre-process. Whenever a user operation changes the number of levels in the list, the data storage tool will create an asynchronous task in the asynchronous processor to perform a cross-level promotion algorithm calculation, pre-calculating the promotion probability of inserting data into the skip list. This reduces the number of algorithm operations and sets the probability of being less than the promotion probability threshold to 0, thus effectively controlling space waste caused by small data volume and excessively high levels.

[0139] Step 5.2 When inserting a node, directly insert the node into the skip list based on the region and level of the node's index value, as well as the calculation results in Step 5.1.

[0140] Step 6, Query Process

[0141] When a user queries a node, the data storage tool starts by comparing data from the top level of the skip list to find the region, and then skips to search the lower-level list, continuously comparing data at each level until the node data is found. This reduces the query time complexity to O(logn).

[0142] In addition, the specific implementation details of the data storage method in another reference embodiment of the present invention have been described in detail in the data storage method described above, so the details will not be repeated here.

[0143] Figure 5 This is a schematic diagram of a data storage device according to an embodiment of the present invention. Figure 5As shown, the data storage device 500 includes a calculation module 501, an insertion module 502, and a persistence module 503; wherein, the calculation module 501 is used to calculate the promotion probability of inserting data into the skip list; the insertion module 502 is used to insert the target data into the skip list according to the promotion probability in response to the user's storage operation on the target data; the persistence module 503 is used to persist the skip list to the front-end database; wherein, the front-end database is located on the local disk, and the skip list is stored in memory.

[0144] Optionally, the calculation module 501 is further configured to:

[0145] Based on the number of levels in the skip list and the cross-level value, determine the target level from which the promotion probability is to be calculated;

[0146] Calculate the promotion probability of data in the target layer when inserting data into the bottom linked list of the jump list.

[0147] Optionally, the insertion module 502 is further configured to:

[0148] In response to the user's storage operation on the target data, the target node corresponding to the target data is inserted into the underlying linked list of the skip linked list;

[0149] Determine whether the target node needs to be promoted based on the promotion probability; if so, insert the target node into the indexed linked list of the skip list.

[0150] Optionally, the insertion module 502 is further configured to:

[0151] Determine whether the number of nodes in the underlying linked list of the skip list is greater than the node count threshold;

[0152] If not, the target node corresponding to the target data is inserted at the end of the underlying linked list;

[0153] If so, then find the insertion position in the underlying linked list and insert the target node corresponding to the target data into the insertion position.

[0154] Optionally, the insertion module 502 is further configured to:

[0155] The insertion position is determined by traversing the index values ​​of each node in the underlying linked list.

[0156] Determine whether the span between the index values ​​of two nodes adjacent to the insertion position is greater than the index value span threshold;

[0157] If so, the target node corresponding to the target data is inserted between the two nodes;

[0158] If not, the target node corresponding to the target data is inserted at the end of the underlying linked list.

[0159] Optionally, the insertion module 502 is further configured to:

[0160] For each level of the skip list, the index value of each node in the index list is traversed to insert the target node into the index list.

[0161] Based on the promotion probability, determine whether the target node needs to continue to be promoted; if so, insert the target node into the next level of the indexed linked list until the target node no longer needs to be promoted.

[0162] Optionally, a loading module is also included for:

[0163] Before calculating the promotion probability of inserting data into the skip list, the skip list is retrieved from the front-end database and loaded into memory.

[0164] It should be noted that the specific implementation details of the data storage device described in this invention have been described in detail in the data storage method described above, so the details will not be repeated here.

[0165] Figure 6 An exemplary system architecture 600 is shown that can be applied to the data storage method or data storage device of the present invention.

[0166] like Figure 6 As shown, system architecture 600 may include terminal devices 601, 602, and 603, a network 604, and a server 605. Network 604 serves as the medium for providing communication links between terminal devices 601, 602, and 603 and server 605. Network 604 may include various connection types, such as wired or wireless communication links or fiber optic cables, etc.

[0167] Users can use terminal devices 601, 602, and 603 to interact with server 605 via network 604 to receive or send messages, etc. Various communication client applications can be installed on terminal devices 601, 602, and 603, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, social media platform software, etc. (for example only).

[0168] Terminal devices 601, 602, and 603 can be various electronic devices with displays and web browsing capabilities, including but not limited to smartphones, tablets, laptops, and desktop computers.

[0169] Server 605 can be a server that provides various services, such as a backend management server that supports shopping websites browsed by users using terminal devices 601, 602, and 603 (for example only). The backend management server can analyze and process data such as received item information query requests, and feed back the processing results (such as target push information and item information—for example only) to the terminal devices.

[0170] It should be noted that the data storage method provided in this embodiment of the invention is executed by terminal devices 601, 602, and 603, and correspondingly, the data storage device is disposed in terminal devices 601, 602, and 603.

[0171] It should be understood that Figure 6 The number of terminal devices, networks, and servers shown is merely illustrative. Depending on implementation needs, any number of terminal devices, networks, and servers can be included.

[0172] The following is for reference. Figure 7 It shows a schematic diagram of the structure of a computer system 700 suitable for implementing a terminal device of the present invention. Figure 7 The terminal device shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of the present invention.

[0173] like Figure 7 As shown, the computer system 700 includes a central processing unit (CPU) 701, which can perform various appropriate actions and processes based on programs stored in read-only memory (ROM) 702 or programs loaded from storage section 708 into random access memory (RAM) 703. The RAM 703 also stores various programs and data required for the operation of the system 700. The CPU 701, ROM 702, and RAM 703 are interconnected via a bus 704. An input / output (I / O) interface 705 is also connected to the bus 704.

[0174] The following components are connected to the I / O interface 705: an input section 706 including a keyboard, mouse, etc.; an output section 707 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and speakers, etc.; a storage section 708 including a hard disk, etc.; and a communication section 709 including a network interface card such as a LAN card, modem, etc. The communication section 709 performs communication processing via a network such as the Internet. A drive 710 is also connected to the I / O interface 705 as needed. A removable medium 711, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on the drive 710 as needed so that computer programs read from it can be installed into the storage section 708 as needed.

[0175] In particular, according to the embodiments disclosed in this invention, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments disclosed in this invention include a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication section 709, and / or installed from removable medium 711. When the computer program is executed by central processing unit (CPU) 701, it performs the functions defined above in the system of this invention.

[0176] It should be noted that the computer-readable medium shown in this invention can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this invention, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this invention, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media can also be any computer-readable medium other than computer-readable storage media, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wireless, wire, optical fiber, RF, etc., or any suitable combination thereof.

[0177] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer programs according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0178] The modules described in the embodiments of the present invention can be implemented in software or hardware. The described modules can also be located in a processor; for example, a processor can be described as including a computing module, an insertion module, and a persistence module, wherein the names of these modules do not necessarily limit the module itself.

[0179] In another aspect, the present invention also provides a computer-readable medium, which may be included in the device described in the above embodiments; or it may exist independently and not assembled into the device. The computer-readable medium carries one or more programs, which, when executed by the device, implement the following method: calculating the promotion probability of inserting data into a skip list; in response to a user's storage operation on target data, inserting the target data into the skip list according to the promotion probability; persisting the skip list to a front-end database; wherein the front-end database is located on a local disk, and the skip list is stored in memory.

[0180] In another aspect, embodiments of the present invention also provide a computer program product, including a computer program that, when executed by a processor, implements the methods described in any of the above embodiments.

[0181] According to the technical solution of this invention, by first calculating the promotion probability of inserting data into the skip list, then inserting the target data into the skip list according to the promotion probability, and finally persisting the skip list in memory to the front-end database on the local disk, the technical problems of low query efficiency and inability to persist storage in the prior art are overcome. This invention uses a skip list to store data to achieve hierarchical skip queries, thereby reducing query time complexity; moreover, by calculating the promotion probability of inserting data into the skip list before insertion, the data structure storage efficiency is improved.

[0182] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can occur depending on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A data storage method, characterized in that, include: Calculate the promotion probability of inserting data into a skip list; the skip list includes a bottom-level list and several levels of indexed lists; In response to the user's storage operation on the target data, the target node corresponding to the target data is inserted into the underlying linked list of the skip linked list; Determine whether the target node needs to be promoted based on the promotion probability; if so, insert the target node into the indexed linked list of the skip list. Persist the skip list to the front-end database; The front-end database is located on the local disk, and the skip list is stored in memory; Inserting the target node corresponding to the target data into the bottom linked list of the skip list includes: Determine whether the number of nodes in the underlying linked list of the skip list is greater than the node count threshold; If not, the target node corresponding to the target data is inserted at the end of the underlying linked list; If so, then find the insertion position in the underlying linked list and insert the target node corresponding to the target data into the insertion position.

2. The method according to claim 1, characterized in that, Calculating the promotion probability of inserting data into the skip list includes: Based on the number of levels in the skip list and the cross-level value, determine the target level from which the promotion probability is to be calculated; Calculate the promotion probability of data in the target layer when inserting data into the bottom linked list of the jump list.

3. The method according to claim 1, characterized in that, Finding the insertion position in the underlying linked list and inserting the target node corresponding to the target data into the insertion position includes: The insertion position is determined by traversing the index values ​​of each node in the underlying linked list. Determine whether the span between the index values ​​of two nodes adjacent to the insertion position is greater than the index value span threshold; If so, the target node corresponding to the target data is inserted between the two nodes; If not, the target node corresponding to the target data is inserted at the end of the underlying linked list.

4. The method according to claim 1, characterized in that, Inserting the target node into the indexed linked list of the skip list includes: For each level of the skip list, the index value of each node in the index list is traversed to insert the target node into the index list. Based on the promotion probability, determine whether the target node needs to continue to be promoted; if so, insert the target node into the next level of the indexed linked list until the target node no longer needs to be promoted.

5. The method according to claim 1, characterized in that, Before calculating the promotion probability of inserting data into the skip list, the following steps are also included: Retrieve the skip list from the front-end database and load the skip list into memory.

6. A data storage device, characterized in that, include: A calculation module is used to calculate the promotion probability of inserting data into a skip list; the skip list includes a bottom-level list and several levels of indexed lists. An insertion module is used to insert the target node corresponding to the target data into the underlying linked list of the skip linked list in response to the user's storage operation on the target data. Determine whether the target node needs to be promoted based on the promotion probability; if so, insert the target node into the indexed linked list of the skip list. The persistence module is used to persist the skip list to the front-end database; The front-end database is located on the local disk, and the skip list is stored in memory; The insertion module is also used for: Determine whether the number of nodes in the underlying linked list of the skip list is greater than the node count threshold; If not, the target node corresponding to the target data is inserted at the end of the underlying linked list; If so, then find the insertion position in the underlying linked list and insert the target node corresponding to the target data into the insertion position.

7. An electronic device, characterized in that, include: One or more processors; Storage device for storing one or more programs. When the one or more programs are executed by the one or more processors, the one or more processors implement the method as described in any one of claims 1-5.

8. A computer-readable medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the method as described in any one of claims 1-5.

9. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1-5.

Images