Data storage method and device of electronic equipment and computer readable storage medium

Abstract

The invention discloses a data storage method and device of electronic equipment and a computer readable storage medium, and relates to the technical field of computers. A specific embodiment of the method comprises the following steps: in response to starting of the electronic equipment, establishing intermediate node data and root node data of a multi-way tree structure in a volatile memory according to data of a plurality of leaf nodes of the multi-way tree structure which is pre-persisted in a persistent memory; and generating auxiliary data of the leaf node in the persistent memory so as to execute a read operation or a write operation on the leaf node. According to the implementation mode, the volatile memory and the persistent memory can be combined to realize data persistence, and meanwhile, the access speed approximate to that of the volatile memory is kept.

Description

Technical Field

[0001] The present invention relates to the field of computer technologies, and in particular, to a data storage method, apparatus, and computer-readable storage medium for an electronic device. Background Art

[0002] In-memory KV (Key-Value) storage is deeply used as a cache system in various links of a computer system. Most in-memory KV storage solutions in the industry use Redis as the kernel, which features using volatile memory (such as dynamic random access memory DRAM) as the storage medium, a single-threaded processing model, and data loss when power is off. In the actual application process, it is found that in some scenarios, the KV storage solution needs to have the characteristic of data persistence and support multi-threaded concurrent access. In addition, as the application scale continues to grow, the hardware cost of the in-memory KV storage solution is also very high. Summary of the Invention

[0003] In view of this, embodiments of the present invention provide a data storage method, apparatus, and computer-readable storage medium for an electronic device, which can combine volatile memory and persistent memory to achieve data persistence while maintaining an access speed approximate to that of volatile memory.

[0004] To achieve the above object, according to one aspect of the present invention, there is provided a data storage method for an electronic device. The electronic device includes: a volatile memory and a persistent memory sharing a memory address space, and the volatile memory and the persistent memory are used to store data having a multi-tree structure.

[0005] The data storage method for the electronic device according to the embodiments of the present invention includes: in response to the startup of the electronic device, establishing intermediate node data and root node data of the multi-tree structure in the volatile memory according to data of a plurality of leaf nodes of the multi-tree structure pre-persisted in the persistent memory; generating auxiliary data of the leaf nodes in the persistent memory to perform a read operation or a write operation on the leaf nodes.

[0006] Optionally, any leaf node contains at least one element, and any element has a keyword and a value corresponding to the keyword. The plurality of leaf nodes are arranged in a chain structure according to the size of the keywords of the contained elements; and, the elements within the same leaf node are unordered.

[0007] Optionally, in the multi-way tree structure: keywords for performing indexing and without corresponding values exist in the elements of the intermediate nodes and the root node; any keyword in any parent node exists in a child node of the parent node and serves as the maximum keyword or the minimum keyword in the child node; and, based on the data of multiple leaf nodes of the multi-way tree structure pre-persisted in the persistent memory, the data of the intermediate nodes and the root node of the multi-way tree structure are established in the volatile memory, including: establishing underlying intermediate nodes that are the parent nodes of the multiple leaf nodes according to the maximum keyword or the minimum keyword in the multiple leaves, generating a parent node pointer in any leaf node to point to the parent node of the leaf node, and layer by layer establishing the remaining intermediate nodes and the root node of the multi-way tree structure based on the underlying intermediate nodes.

[0008] Optionally, the auxiliary data is not persisted in the persistent memory; the auxiliary data of any leaf node includes at least one of the following data: the version number of the leaf node, the fingerprint values of the elements in the leaf node, and the parent node pointer of the leaf node.

[0009] Optionally, the method further includes: in response to a read operation on the multi-way tree structure, comparing the read keyword indicated by the read operation with the root node and intermediate nodes in the volatile memory to determine the target leaf node; in the case where there is no lock mark on the target leaf node, recording the current version number of the target leaf node as the first version number; calculating the fingerprint value of the read keyword, and determining the target element in the target leaf node whose fingerprint value is consistent with the read keyword; in the case where it is determined that the keyword of the target element is consistent with the read keyword, obtaining the value in the target element, and recording the current version number of the target leaf node as the second version number; in the case where it is determined that the second version number is equal to the first version number, returning the value of the obtained target element.

[0010] Optionally, the method further includes: in response to a write operation on the multi-way tree structure, comparing the write keyword indicated by the write operation with the root node and intermediate nodes in the volatile memory to determine the destination leaf node, and adding a lock mark to the destination leaf node in the case where there is no lock mark on the destination leaf node; writing the data to be written indicated by the write operation into the free element slot of the destination leaf node, and persisting the data to be written in the persistent memory; in the case where the current number of elements in the destination leaf node is greater than the preset element number threshold, performing a split operation on the destination leaf node; in the persistent memory, determining the fingerprint value of the data to be written according to the write keyword, updating the version number of the destination leaf node, and clearing the lock mark of the destination leaf node.

[0011] Optionally, the elements of the leaf node further include: the execution sequence number and the written data for any write operation of the multi-way tree structure, and the execution sequence number and the written data are used to form a snapshot of the multi-way tree structure at a historical moment or the current moment.

[0012] To achieve the above object, according to another aspect of the present invention, there is provided a data storage device for an electronic device. The electronic device includes: a volatile memory and a persistent memory sharing a memory address space, and the volatile memory and the persistent memory are used to store data having a multi-way tree structure.

[0013] The data storage device of the electronic device according to an embodiment of the present invention includes: a multi-way tree building unit, configured to, in response to the startup of the electronic device, establish intermediate node data and root node data of the multi-way tree structure in the volatile memory according to data of a plurality of leaf nodes of the multi-way tree structure pre-persisted in the persistent memory; an auxiliary data generation unit, configured to generate auxiliary data of the leaf nodes in the persistent memory to perform read operations or write operations on the leaf nodes.

[0014] To achieve the above object, according to still another aspect of the present invention, there is provided a computer-readable storage medium.

[0015] The computer-readable storage medium according to an embodiment of the present invention includes: a volatile memory and a persistent memory sharing a memory address space, and the volatile memory and the persistent memory are used to store data having a multi-way tree structure; wherein, the persistent memory is used to persist data of a plurality of leaf nodes of the multi-way tree structure, and is further used to store auxiliary data of a plurality of leaf nodes generated when the computer-readable storage medium is started; the volatile memory is used to store: intermediate node data and root node data of the multi-way tree structure established according to the data of the plurality of leaf nodes when the computer-readable storage medium is started.

[0016] Optionally, any leaf node contains at least one element, and any element has a keyword and a value corresponding to the keyword. The plurality of leaf nodes are arranged in a chain structure according to the size of the keywords of the included elements; the elements within the same leaf node are arranged disorderly; the auxiliary data is not persisted in the persistent memory; the auxiliary data of any leaf node includes at least one of the following data: the version number of the leaf node, the fingerprint values of the elements in the leaf node, and the pointer to the parent node of the leaf node.

[0017] To achieve the above object, according to still another aspect of the present invention, there is provided an electronic device.

[0018] An electronic device according to the present invention includes: one or more processors; a storage device for storing one or more programs, which when executed by the one or more processors, cause the one or more processors to implement the data storage method provided by the present invention.

[0019] To achieve the above object, according to another aspect of the present invention, a computer-readable storage medium is provided.

[0020] A computer-readable storage medium according to the present invention has a computer program stored thereon, and when the program is executed by a processor, it implements the data storage method provided by the present invention.

[0021] According to the technical solution of the present invention, the embodiments in the above invention have the following advantages or beneficial effects:

[0022] The electronic device in the embodiment of the present invention includes a volatile memory and a persistent memory sharing a memory address space. The persistent memory is used to persistently store data of multiple leaf nodes of a multi-way tree structure, and the volatile memory is used to store intermediate node data and root node data of the multi-way tree structure established according to the leaf node data when the electronic device is started. In this way, the main data is persistently stored in the leaf nodes of the persistent memory to achieve the data persistence feature, and at the same time, it has an access speed similar to that of the volatile memory; the root node and intermediate nodes for locating the leaf nodes are stored in the volatile memory to accelerate the indexing speed. These data with a small amount are lost when the power is off and are rebuilt according to the leaf nodes after restart. The above method avoids the large use of expensive volatile memory and reduces the hardware cost. The leaf nodes are sorted in order of the keyword size in the contained elements, but the elements within the same leaf node are not sorted. This can further improve the write operation speed of the persistent memory and avoid the frequent movement of element positions caused by element sorting during write operations. The embodiment of the present invention also generates auxiliary data such as version numbers, fingerprint values, and parent node pointers of leaf nodes in the persistent memory. Although these data are written in the persistent memory, they are not persisted. When the system restarts, they are rebuilt according to the leaf node data. This method can reduce the persistent operation instructions of the persistent memory and improve the write operation performance of the persistent memory. In addition, the embodiment of the present invention adopts a version number mechanism and a lock mechanism to execute read operations and write operations on the multi-way tree structure respectively, and has good multi-threaded concurrent access performance.

[0023] The further effects of the above non-conventional optional methods will be described in conjunction with specific embodiments below. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] The drawings are used to better understand the present invention and do not constitute an improper limitation of the present invention. Among them:

[0025] Figure 1It is a schematic diagram of the main steps of the data storage method in an embodiment of the present invention;

[0026] Figure 2 It is a schematic diagram of the multi - fork tree structure in an embodiment of the present invention;

[0027] Figure 3 It is a schematic diagram of the data reading process in an embodiment of the present invention;

[0028] Figure 4 It is a schematic diagram of the data writing process in an embodiment of the present invention;

[0029] Figure 5 It is a schematic diagram of the components of the data storage device in an embodiment of the present invention;

[0030] Figure 6 It is an exemplary system architecture diagram to which an embodiment of the present invention can be applied;

[0031] Figure 7 It is a schematic diagram of the structure of an electronic device for implementing the data storage method in an embodiment of the present invention. Detailed implementation manners

[0032] The following describes exemplary embodiments of the present invention with reference to the accompanying drawings. Various details of the embodiments of the present invention are included to facilitate understanding, and they should be considered merely exemplary. Therefore, those of ordinary skill in the art should recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present invention. Similarly, for clarity and conciseness, descriptions of well - known functions and structures are omitted below.

[0033] The electronic device in an embodiment of the present invention includes: a central processing unit CPU, a disk, a memory, etc. The memory includes volatile memory and persistent memory sharing a memory address space. Volatile memory is a memory hardware for quickly acquiring and storing data. When the system is shut down, the data in volatile memory will be automatically deleted. The volatile memory in the present invention refers to RAM (Random Access Memory), including dynamic random access memory DRAM, static random access memory SRAM, etc.

[0034] Persistent Memory (PMEM), also called Non - Volatile Memory (NVM), is a type of storage hardware that supports byte - addressable, can be directly operated through CPU instructions, and the data does not get lost after power failure. Generally speaking, the read - write performance of persistent memory is lower than that of volatile memory, and the CPU needs additional instructions to persist the data in the CPU cache to persistent memory.

[0035] In an embodiment of the present invention, volatile memory and persistent memory are used to store data having a multi-way tree structure. In the above multi-way tree structure, the root node and any intermediate node have at least two child nodes at least. The keywords in the root node and intermediate nodes are arranged in an orderly manner. The root node and intermediate nodes are only used to perform keyword-based indexing and do not store the main data (such as the value in the key-value pair). The main data is stored in the leaf nodes. The leaf nodes contain main data information, and the leaf nodes form a linked structure in the order of the size of the keywords. Any keyword of any intermediate node exists in a child node of the intermediate node and is the largest or smallest keyword in the child node. It can be seen that the above multi-way tree structure is similar to the known B+ tree. One of the differences between the two is that in the above multi-way tree structure, the elements in the same leaf node are not arranged in an orderly manner according to the size of the corresponding keywords, but are arranged disorderly.

[0036] It should be noted that, without conflict, the embodiments of the present invention and the technical features in the embodiments can be combined with each other.

[0037] Figure 1 It is a schematic diagram of the main steps of the data storage method according to the embodiment of the present invention.

[0038] As Figure 1 shown, the data storage method of the embodiment of the present invention can be executed by the central processing unit of the electronic device, and the specific execution steps are as follows:

[0039] Step S101: In response to the startup of the electronic device, based on the data of the multiple leaf nodes of the multi-way tree structure pre-persisted in the persistent memory, establish the intermediate node data and root node data of the multi-way tree structure in the volatile memory.

[0040] Referring to Figure 2 , in the embodiment of the present invention, the data of the leaf nodes (such as KV data) in the multi-way tree structure is persisted in the persistent memory, and the data of the root node and intermediate nodes (including keywords and pointers, and the intermediate nodes can be one or more layers) are stored in the volatile memory. When the electronic device is shut down, the leaf node data will not be lost while the root node and intermediate node data will be lost. When the electronic device starts up (such as powering on or restarting), the central processing unit reconstructs the intermediate node data and root node data of the multi-way tree structure in the volatile memory based on the leaf node data persisted in the persistent memory, so as to perform high-speed indexing based on the volatile memory.

[0041] In one embodiment, any leaf node contains at least one element. Any element has a keyword (key) and a corresponding value (value) for the keyword. Multiple leaf nodes are arranged in a linked structure according to the size of the keywords of the contained elements, while the elements within the same leaf node are unordered. This data structure can improve the write operation speed of persistent memory and avoid the frequent movement of element positions caused by element sorting during write operations. Each leaf node also has a pointer Pnext for realizing the ordered connection of each leaf node. Optionally, the elements of the leaf node may further contain: the execution sequence number and the written data (such as the key-value pair data or value data written) for any write operation of the multi-way tree structure. The above execution sequence number and written data can be used to form a snapshot of the multi-way tree structure at a historical moment or the current moment. The above key-value pair data, execution sequence number, and pointer Pnext in the leaf node element are the leaf node data persisted to the persistent memory.

[0042] In the above multi-way tree structure, the elements of the intermediate node and the root node have keywords for performing indexing and no corresponding values; any keyword in any parent node exists in a child node of the parent node and serves as the largest keyword or the smallest keyword in the child node. Correspondingly, the central processing unit can first establish the underlying intermediate node as the parent node of the above multiple leaf nodes according to the largest keyword or the smallest keyword in the multiple leaves, and generate a parent node pointer in any leaf node to point to the parent node of the leaf node. Thereafter, the central processing unit gradually establishes the remaining intermediate nodes and the root node of the multi-way tree structure based on the underlying intermediate node according to the requirements of the multi-way tree structure.

[0043] The above data storage method can use leaf nodes to achieve the persistence of the main data. The persistent memory used to store the leaf nodes has a much higher access speed than the disk, solving the problem of the read-write performance bottleneck caused by disk I / O in the prior art. The root node and intermediate nodes for locating the leaf nodes are stored in volatile memory to accelerate the indexing speed. These data with a small amount are lost when the power is off and can be quickly rebuilt according to the leaf nodes after restart. This way of combining volatile memory and persistent memory can also avoid the large use of expensive volatile memory and reduce the hardware cost.

[0044] Step S102: Generate auxiliary data of the leaf node in the persistent memory to perform read or write operations on the leaf node.

[0045] Exemplarily, the auxiliary data of any leaf node may include at least one of the following data: the version number of the leaf node, the fingerprint values of the elements in the leaf node, and the pointer to the parent node of the leaf node. The above version number is used to implement data reading with an optimistic locking mechanism. When responding to a read operation, the version numbers at two moments before and after are recorded. If the version numbers are the same, it indicates that no write operation has occurred during the reading process, and the value of the response is returned; if the version numbers are different, it means that data has been updated during the reading process, and the read operation is performed again. The above fingerprint value is an integer value, which is used to compare data during data reading. Optimizing the traditional string comparison to integer value comparison based on fingerprint values helps to improve the data search speed; the above pointer to the parent node is used to implement the connection between the leaf node and its parent node.

[0046] Specifically, although the above auxiliary data is written into the persistent memory after the electronic device is started, it is not persisted. After that, if the electronic device is turned off, these auxiliary data will be lost, and these auxiliary data can be reconstructed when the electronic device is restarted. This method can reduce the persistent operation instructions of the persistent memory and improve the write operation performance of the persistent memory. Exemplarily, the auxiliary data of any leaf node may further include the following data: the current sequence number of each element in the leaf node, the lock flag of the leaf node (used during read and write operations), the flag indicating whether it is currently a data insertion, the flag indicating whether to perform splitting of the leaf node currently, the cumulative number of data insertions of the leaf node, the cumulative number of splits of the leaf node, and the flag indicating whether the current root node has changed. In a specific application, after the electronic device is started, before the central processing unit reconstructs the intermediate node, the root node, and the auxiliary data, it can first clear the historical data such as the version number and fingerprint value that may exist in the leaf node.

[0047] As a preferred solution, the data reading process of the above multi-way tree structure is as Figure 3As shown in the figure. In step S301, the central processing unit monitors a read operation on the multi-way tree structure; in step S302, the central processing unit compares the read key indicated by the read operation with the root node and intermediate nodes in the volatile memory to determine the target leaf node; in step S303, the central processing unit determines whether there is a lock flag on the target leaf node; if it exists, it means that a write operation is currently being executed, and it can return to step S301 after waiting for a certain period of time; if it does not exist, step S304 is executed; in step S304, the central processing unit records the current version number of the target leaf node as the first version number; in step S305, the central processing unit calculates the fingerprint value of the read key; in step S306, the central processing unit determines whether there is a target element in the target leaf node whose fingerprint value is the same as the fingerprint value of the read key; if it exists, step S307 is executed; if it does not exist, it means that the result is not hit, and a null value is returned; in step S307, the central processing unit determines whether the key of the target element is the same as the read key; if they are the same, step S308 is executed; if they are not the same, it means that the result is not hit, and a null value is returned; in step S308, the central processing unit obtains the value in the target element; in step S309, the central processing unit records the current version number of the target leaf node as the second version number; in step S310, the central processing unit determines whether the second version number is equal to the first version number; if they are equal, step S311 is executed; if they are not equal, it means that data update has occurred during the reading process, and it can return to step S303 to read again; in step S311, the central processing unit returns the value of the target element obtained. During the above read operation process, the optimistic lock mechanism based on version number comparison and the integer value comparison based on fingerprint value can achieve high-speed multi-threaded concurrent access performance.

[0048] In an alternative technical solution, the data writing process of the multi-way tree structure is as Figure 4As shown. In step S401, the central processor detects a write operation to a multi-branch tree structure; in step S402, the central processor compares the write keyword indicated by the write operation with the root node and the intermediate node in the volatile memory to determine the destination leaf node; in step S403, the central processor determines whether there is a lock mark on the destination leaf node; if there is, it means that another write operation is currently being executed, and then returns to step S401 after waiting for a certain period of time; if not, execute step S404; in step S404, the central processor adds a lock mark to the destination leaf node in the persistent memory; in step S405, the central processor writes the write operation to the free element slot of the destination leaf node. Indicates the data to be written; in step S406, the central processing unit persists the data to be written in the persistent memory; in step S407, the central processing unit determines whether the current number of elements of the destination leaf node is greater than the preset element number threshold; if greater, the destination node is split and the multi-branch tree structure is adaptively adjusted before executing step S408; if not greater, directly execute step S408; in step S408, the central processing unit determines the fingerprint value of the data to be written in the persistent memory according to the write keyword; in step S409, the central processing unit updates the version number of the destination leaf node in the persistent memory; in step S410, the central processing unit clears the lock mark of the destination leaf node. In the above read operation process, multi-threaded concurrent write operations are implemented through the lock mechanism, and the data consistency of the write operation process is guaranteed.

[0049] In the technical solution of the embodiment of the present invention, a high-performance persistent KV storage method is implemented by using a persistent memory device, which not only ensures the persistence and consistency of data in abnormal situations such as power failure and system crash, but also has an access speed similar to that of volatile memory and excellent multi-threaded concurrency performance.

[0050] It should be noted that the collection, collection, update, analysis, processing, use, transmission, storage and other aspects of user personal information that may be involved in the technical solution of the present invention are in compliance with the provisions of relevant laws and regulations, are used for legitimate purposes, and do not violate public order and good morals. Necessary measures are taken for user personal information to prevent illegal access to user personal information data and maintain the security of user personal information, network security and national security.

[0051] For the above-mentioned method embodiments, for the convenience of description, they are expressed as a series of action combinations, but those skilled in the art should know that the present invention is not limited by the order of the actions described, and some steps can actually be performed in other orders or simultaneously. In addition, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily necessary to implement the present invention.

[0052] To facilitate better implementation of the above solutions of the embodiments of the present invention, the following also provides related devices for implementing the above solutions.

[0053] Please refer to Figure 5 As shown, the data storage device 500 of the electronic device provided by the embodiment of the present invention may include: a multi-way tree building unit 501 and an auxiliary data generation unit 502. The electronic device includes: a volatile memory and a persistent memory sharing a memory address space, and the volatile memory and the persistent memory are used to store data having a multi-way tree structure.

[0054] The multi-way tree building unit 501 can be used to, in response to the startup of the electronic device, establish the intermediate node data and root node data of the multi-way tree structure in the volatile memory according to the data of multiple leaf nodes of the multi-way tree structure pre-persisted in the persistent memory; the auxiliary data generation unit 502 can be used to generate auxiliary data of the leaf nodes in the persistent memory to perform read or write operations on the leaf nodes.

[0055] In the embodiment of the present invention, any leaf node contains at least one element, and any element has a keyword and a value corresponding to the keyword. The multiple leaf nodes are arranged in a chain structure according to the size of the keywords of the contained elements; and, the elements within the same leaf node are arranged disorderly.

[0056] As a preferred solution, in the multi-way tree structure: the elements of the intermediate node and the root node have keywords for performing indexing and no corresponding values; any keyword in any parent node exists in a child node of the parent node and serves as the largest keyword or the smallest keyword in the child node; the multi-way tree building unit 501 can be further used to: establish a bottom-layer intermediate node serving as the parent node of the multiple leaf nodes according to the largest keyword or the smallest keyword among the multiple leaves, generate a parent node pointer in any leaf node to point to the parent node of the leaf node, and layer by layer establish the remaining intermediate nodes and the root node of the multi-way tree structure based on the bottom-layer intermediate node.

[0057] Preferably, the auxiliary data is not persisted in the persistent memory; the auxiliary data of any leaf node includes at least one of the following data: the version number of the leaf node, the fingerprint values of the elements in the leaf node, the parent node pointer of the leaf node.

[0058] In a specific application, the device 500 may further include: a reading unit, configured to: in response to a read operation on the multi-way tree structure, compare the read key indicated by the read operation with the root node and intermediate nodes in the volatile memory to determine a target leaf node; in the case where there is no lock mark on the target leaf node, record the current version number of the target leaf node as the first version number; calculate the fingerprint value of the read key, and determine a target element in the target leaf node whose fingerprint value is consistent with the read key; in the case where it is determined that the key of the target element is consistent with the read key, obtain the value in the target element, and record the current version number of the target leaf node as the second version number; in the case where it is determined that the second version number is equal to the first version number, return the value of the target element obtained.

[0059] In a specific application, the device 500 may further include: a writing unit, configured to: in response to a write operation on the multi-way tree structure, compare the write key indicated by the write operation with the root node and intermediate nodes in the volatile memory to determine a destination leaf node, and add a lock mark to the destination leaf node in the case where there is no lock mark on the destination leaf node; write the data to be written indicated by the write operation into the free element slot of the destination leaf node, and persist the data to be written in the persistent memory; in the case where the current number of elements in the destination leaf node is greater than a preset element number threshold, perform a split operation on the destination leaf node; in the persistent memory, determine the fingerprint value of the data to be written according to the write key, update the version number of the destination leaf node, and clear the lock mark of the destination leaf node.

[0060] In addition, in an embodiment of the present invention, the element in the leaf node further includes: an execution sequence number and written data for any write operation on the multi-way tree structure, and the execution sequence number and written data are used to form a snapshot of the multi-way tree structure at a historical moment or the current moment.

[0061] According to the physical characteristics of the persistent memory medium, an embodiment of the present invention implements a high-performance persistent KV persistent storage engine. The persistent memory is used to persist the KV main data, and the DRAM stores the reconstructable index nodes (including the root node and intermediate nodes), fully utilizing the performance advantages of the DRAM to accelerate the lookup process. The data structure in the persistent memory is also divided into a persistent main part and a non-persistent reconstructable part (auxiliary data), further reducing the execution of persistent instructions. The hash value of the key is stored in the reconstructable part of the persistent memory to implement data query, thereby accelerating the read operation of the persistent memory. An optimistic lock mechanism is implemented based on the reconstructable part in the persistent memory, thereby improving the multi-threaded concurrent access performance.

[0062] The present invention further provides a computer-readable storage medium. Refer to Figure 2 . The computer-readable storage medium includes: volatile memory and persistent memory sharing a memory address space, where the volatile memory and the persistent memory are used to store data having a multi-way tree structure; wherein, the persistent memory is used to persist data of multiple leaf nodes of the multi-way tree structure, and is further used to store auxiliary data of multiple leaf nodes generated when the computer-readable storage medium is started; the volatile memory is used to store: intermediate node data and root node data of the multi-way tree structure established according to the data of the multiple leaf nodes when the computer-readable storage medium is started.

[0063] In an embodiment of the present invention, any leaf node contains at least one element, and any element has a keyword and a value corresponding to the keyword. The multiple leaf nodes are arranged in a chain structure according to the size of the keywords of the contained elements; the elements within the same leaf node are arranged disorderly; the auxiliary data is not persisted in the persistent memory; the auxiliary data of any leaf node includes at least one of the following data: the version number of the leaf node, the fingerprint values of the elements in the leaf node, and the pointer to the parent node of the leaf node. Optionally, the elements of the leaf node further contain: the execution sequence number and the written data for any write operation on the multi-way tree structure, and the execution sequence number and the written data are used to form a snapshot of the multi-way tree structure at a historical moment or the current moment.

[0064] In an alternative technical solution, in the multi-way tree structure: the elements of the intermediate node and the root node have keywords for performing indexing and having no corresponding values; any keyword in any parent node exists in a child node of the parent node and serves as the largest keyword or the smallest keyword in the child node; the intermediate node data and the root node data of the multi-way tree structure are reconstructed by a central processing unit through the following steps: establishing a bottom intermediate node serving as the parent node of the multiple leaf nodes according to the largest keyword or the smallest keyword in the multiple leaves, generating a parent node pointer in any leaf node to point to the parent node of the leaf node, and layer by layer establishing the remaining intermediate nodes and the root node of the multi-way tree structure based on the bottom intermediate node.

[0065] The data reading steps of the computer-readable storage medium are as follows: In response to a read operation on the multi-way tree structure, the central processing unit compares the read key indicated by the read operation with the root node and intermediate nodes in the volatile memory to determine the target leaf node; in the case where there is no lock mark on the target leaf node, the central processing unit records the current version number of the target leaf node as the first version number; the central processing unit calculates the fingerprint value of the read key and determines the target element in the target leaf node whose fingerprint value is consistent with the read key; in the case where it is determined that the key of the target element is consistent with the read key, the central processing unit obtains the value in the target element and records the current version number of the target leaf node as the second version number; in the case where it is determined that the second version number is equal to the first version number, the central processing unit returns the value of the target element obtained.

[0066] The data writing steps of the computer-readable storage medium are as follows: In response to a write operation on the multi-way tree structure, the central processing unit compares the write key indicated by the write operation with the root node and intermediate nodes in the volatile memory to determine the destination leaf node, and adds a lock mark to the destination leaf node in the case where there is no lock mark on the destination leaf node; the central processing unit writes the data to be written indicated by the write operation into the empty element slot of the destination leaf node and persists the data to be written in the persistent memory; in the case where the current number of elements in the destination leaf node is greater than the preset element number threshold, the central processing unit performs a split operation on the destination leaf node; in the persistent memory, the central processing unit determines the fingerprint value of the data to be written according to the write key, updates the version number of the destination leaf node, and clears the lock mark of the destination leaf node.

[0067] In the technical solution of the embodiment of the present invention, the main body data is persisted in the leaf nodes of the persistent memory to achieve the data persistence feature, and at the same time has an access speed similar to that of volatile memory; the root nodes and intermediate nodes used to locate the leaf nodes are stored in the volatile memory to accelerate the indexing speed. These data with a small amount are lost when the power is off and rebuilt according to the leaf nodes after restart. The above method avoids the large use of expensive volatile memory and reduces the hardware cost. The above leaf nodes are sorted in order of the keyword size in the contained elements, and the elements within the same leaf node are unsorted, which can further improve the write operation speed of the persistent memory and avoid the frequent movement of element positions caused by element sorting during the write operation. The embodiment of the present invention also generates auxiliary data such as version numbers, fingerprint values, and parent node pointers of the leaf nodes in the persistent memory. Although these data are written in the persistent memory, they are not persisted. When the system restarts, they are rebuilt according to the leaf node data. This method can reduce the persistent operation instructions of the persistent memory and improve the write operation performance of the persistent memory. In addition, the embodiment of the present invention adopts a version number mechanism and a lock mechanism to execute read operations and write operations on the multi-way tree structure respectively, and has good multi-threaded concurrent access performance.

[0068] Figure 6 FIG. 600 shows an exemplary system architecture to which the data storage method or data storage device according to the embodiment of the present invention can be applied.

[0069] As Figure 6 shown, the system architecture 600 may include terminal devices 601, 602, 603, a network 604, and a server 605 (this architecture is only an example, and the components included in the specific architecture can be adjusted according to the specific situation of the invention). The network 604 is used to provide a medium for communication links between the terminal devices 601, 602, 603 and the server 605. The network 604 may include various connection types, such as wired, wireless communication links, or fiber optic cables, etc.

[0070] Users can use the terminal devices 601, 602, 603 to interact with the server 605 through the network 604 to receive or send messages, etc. Various client applications, such as data operation-related applications (only examples), may be installed on the terminal devices 601, 602, 603.

[0071] The terminal devices 601, 602, 603 may be various electronic devices with a display screen and supporting web browsing, including but not limited to smart phones, tablet computers, laptop portable computers, and desktop computers, etc.

[0072] The server 605 can be a server that provides various services. For example, it can be a background server (only for example) that supports applications related to data operations performed by users using the terminal devices 601, 602, and 603. The background server can process the received data query requests and feedback the processing results (such as query results - only for example) to the terminal devices 601, 602, and 603.

[0073] It should be noted that the data storage method provided by the embodiments of the present invention is generally executed by the server 605. Correspondingly, the data storage device is generally arranged in the server 605.

[0074] It should be understood that Figure 6 the numbers of the terminal devices, the network, and the server in

[0075] The present invention also provides an electronic device. The electronic device according to the embodiments of the present invention includes: one or more processors; a storage device for storing one or more programs, and when the one or more programs are executed by the one or more processors, the one or more processors implement the data storage method provided by the present invention.

[0076] Next, refer to Figure 7 , which shows a schematic structural diagram of a computer system 700 of an electronic device suitable for implementing the embodiments of the present invention. Figure 7 The shown electronic device is only an example and should not impose any limitations on the functions and usage scope of the embodiments of the present invention.

[0077] As Figure 7 shown, the computer system 700 includes a central processing unit (CPU) 701, which can perform various appropriate actions and processes according to the programs stored in the read-only memory (ROM) 702 or the programs loaded from the storage section 708 into the random access memory (RAM) 703. In the RAM 703, various programs and data required for the operation of the computer system 700 are also stored. The CPU 701, the ROM 702, and the RAM 703 are connected to each other through a bus 704. The input / output (I / O) interface 705 is also connected to the bus 704.

[0078] The following components are connected to the I / O interface 705: an input section 706 including a keyboard, a mouse, etc.; an output section 707 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc. and a speaker, etc.; a storage section 708 including a hard disk, etc.; and a communication section 709 including a network interface card such as a LAN card, a modem, etc. The communication section 709 performs communication processing via a network such as the Internet. A drive 710 is also connected to the I / O interface 705 as required. A removable medium 711 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc. is installed on the drive 710 as required so that a computer program read therefrom is installed into the storage section 708 as required.

[0079] Specifically, according to the embodiments disclosed in the present invention, the process described in the above main step diagram can be implemented as a computer software program. For example, an embodiment of the present invention includes a computer program product which includes a computer program carried on a computer-readable medium, and the computer program includes program codes for performing the method shown in the main step diagram. In the above embodiment, the computer program can be downloaded and installed from the network through the communication section 709, and / or installed from the removable medium 711. When the computer program is executed by the central processing unit 701, the above functions defined in the system of the present invention are executed.

[0080] It should be noted that the computer-readable medium shown in the present invention can be a computer-readable signal medium, a computer-readable storage medium, or any combination of the above two. The computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above. More specific examples of the computer-readable storage medium can include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present invention, the computer-readable storage medium can be any tangible medium that contains or stores a program, and this program can be used by or in combination with an instruction execution system, apparatus, or device. In the present invention, the computer-readable signal medium can include a data signal propagated in a baseband or as part of a carrier wave, in which the computer-readable program code is carried. Such a propagated data signal can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The computer-readable signal medium can also be any computer-readable medium other than the computer-readable storage medium, and this computer-readable medium can send, propagate, or transmit a program for use by or in combination with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted by any appropriate medium, including but not limited to: wireless, wire, optical cable, RF, etc., or any suitable combination of the above.

[0081] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architectures, functions, and operations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagram can represent a module, a program segment, or a part of code, and the above module, program segment, or part of code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks can occur in a different order than marked in the accompanying drawings. For example, two consecutive blocks shown can actually be executed substantially in parallel, and they can sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram or flowchart, and the combination of blocks in the block diagram or flowchart, can be implemented by a dedicated hardware-based system for performing the specified functions or operations, or can be implemented by a combination of dedicated hardware and computer instructions.

[0082] The units involved in the embodiments of the present invention may be implemented by software or hardware. The units described may also be arranged in a processor, for example, it may be described as follows: a processor includes a multi-tree establishment unit and an auxiliary data generation unit.

[0083] As another aspect, the present invention also provides a computer-readable medium, which may be included in the device described in the above embodiment; or it may exist independently and not be assembled into the device. The above computer-readable medium carries one or more programs, and when the above one or more programs are executed by the device, the steps executed by the device include: in response to the electronic device starting up, according to the data of multiple leaf nodes of the multi-tree structure that are pre-persisted in the persistent memory, the intermediate node data and root node data of the multi-tree structure are established in the volatile memory; the auxiliary data of the leaf node is generated in the persistent memory to perform a read operation or a write operation on the leaf node.

[0084] In the technical solution of the embodiment of the present invention, the main data is persisted in the leaf node of the persistent memory to realize the data persistence feature, and has an access speed similar to that of volatile memory; the root node and intermediate node used to locate the leaf node are stored in the volatile memory to speed up the indexing speed. These data with a small amount of data are lost when the power is off and rebuilt according to the leaf node after restart. The above method avoids the large-scale use of expensive volatile memory and reduces hardware costs. The above leaf nodes are arranged in order according to the size of the keywords in the elements contained, and the elements inside the same leaf node are arranged in disorder. This can further improve the write operation speed of the persistent memory and avoid the frequent movement of the element position caused by the element sorting during the write operation. The embodiment of the present invention also generates auxiliary data such as the version number, fingerprint value, and parent node pointer of the leaf node in the persistent memory. Although these data are written in the persistent memory, they are not persisted. When the system is restarted, they are rebuilt according to the leaf node data. This method can reduce the persistent operation instructions of the persistent memory and improve the write operation performance of the persistent memory. In addition, the embodiment of the present invention adopts the version number mechanism and the lock mechanism to respectively perform read operations and write operations for the multi-tree structure, and has good multi-threaded concurrent access performance.

[0085] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions may occur depending on design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A data storage method for an electronic device, characterized in that, the electronic device includes: a volatile memory and a persistent memory sharing a memory address space, and the volatile memory and the persistent memory are used to store data having a multi - tree structure; the method includes: In response to the startup of the electronic device, based on the data of multiple leaf nodes of the multi - tree structure pre - persisted in the persistent memory, establish the intermediate node data and root node data of the multi - tree structure in the volatile memory; Generate auxiliary data for the leaf nodes in the persistent memory to perform read or write operations on the leaf nodes.

2. The method according to claim 1, characterized in that, any leaf node contains at least one element, and any element has a keyword and a value corresponding to the keyword. The multiple leaf nodes are arranged in a chain structure according to the size of the keywords of the contained elements; and, the elements within the same leaf node are arranged disorderly.

3. The method according to claim 2, characterized in that, in the multi - tree structure: the elements of the intermediate node and the root node have keywords for performing indexing and without corresponding values; any keyword in any parent node exists in a child node of the parent node and serves as the largest keyword or the smallest keyword in the child node; and, based on the data of multiple leaf nodes of the multi - tree structure pre - persisted in the persistent memory, establishing the intermediate node data and root node data of the multi - tree structure in the volatile memory includes: Establish a bottom - layer intermediate node serving as the parent node of the multiple leaf nodes according to the largest keyword or the smallest keyword among the multiple leaves. Generate a parent node pointer in any leaf node to point to the parent node of the leaf node, and layer by layer establish the remaining intermediate nodes and the root node of the multi - tree structure based on the bottom - layer intermediate node.

4. The method according to claim 2, characterized in that, the auxiliary data is not persisted in the persistent memory; the auxiliary data of any leaf node includes at least one of the following data: the version number of the leaf node, the fingerprint values of the elements in the leaf node, and the parent node pointer of the leaf node.

5. The method according to claim 4, characterized in that, the method further includes: In response to a read operation on the multi - tree structure, compare the read keyword indicated by the read operation with the root node and intermediate nodes in the volatile memory to determine the target leaf node; In the case where there is no lock mark on the target leaf node, record the current version number of the target leaf node as the first version number; Calculate the fingerprint value of the read keyword, and determine the target element in the target leaf node whose fingerprint value is consistent with the read keyword; In the case where it is determined that the keyword of the target element is consistent with the read keyword, obtain the value in the target element, and record the current version number of the target leaf node as the second version number; In the case where it is determined that the second version number is equal to the first version number, return the value of the obtained target element.

6. The method according to claim 4, wherein, the method further comprises: in response to a write operation on the multi-way tree structure, comparing the write key indicated by the write operation with the root node and intermediate nodes in the volatile memory to determine the destination leaf node, and adding a lock mark to the destination leaf node if there is no lock mark on the destination leaf node; writing the data to be written indicated by the write operation into the available element slot of the destination leaf node, and persisting the data to be written in the persistent memory; and performing a split operation on the destination leaf node if the current number of elements in the destination leaf node is greater than a preset element number threshold; in the persistent memory, determining a fingerprint value of the data to be written according to the write key, updating the version number of the destination leaf node, and clearing the lock mark of the destination leaf node.

7. The method according to claim 2, wherein, each element in the leaf node further contains: an execution sequence number and written data for any write operation on the multi-way tree structure, and the execution sequence number and written data are used to form a snapshot of the multi-way tree structure at a historical moment or the current moment.

8. A data storage device of an electronic device, wherein, the electronic device includes: a volatile memory and a persistent memory sharing a memory address space, and the volatile memory and the persistent memory are used to store data having a multi-way tree structure; the device includes: a multi-way tree building unit, configured to build intermediate node data and root node data of the multi-way tree structure in the volatile memory according to data of multiple leaf nodes of the multi-way tree structure pre-persisted in the persistent memory in response to startup of the electronic device; an auxiliary data generating unit, configured to generate auxiliary data of the leaf nodes in the persistent memory to perform read operations or write operations on the leaf nodes.

9. A computer-readable storage medium, wherein, the computer-readable storage medium includes: a volatile memory and a persistent memory sharing a memory address space, and the volatile memory and the persistent memory are used to store data having a multi-way tree structure; wherein, the persistent memory is used to persist data of multiple leaf nodes of the multi-way tree structure, and is further used to store auxiliary data of multiple leaf nodes generated when the computer-readable storage medium is started; the volatile memory is used to store: intermediate node data and root node data of the multi-way tree structure established according to data of the multiple leaf nodes when the computer-readable storage medium is started.

10. The computer-readable storage medium according to claim 9, wherein, any leaf node contains at least one element, any element has a key and a value corresponding to the key, and the multiple leaf nodes are arranged in a chain structure according to the size of the keys of the contained elements; the elements within the same leaf node are arranged disorderly; The auxiliary data is not persisted in the persistent memory; the auxiliary data of any leaf node includes at least one of the following data: the version number of the leaf node, the fingerprint values of the elements in the leaf node, and the pointer to the parent node of the leaf node.

11. An electronic device, characterized in that it includes: one or more processors; a storage device for storing one or more programs, when the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1-7.

12. A computer-readable storage medium, on which a computer program is stored, characterized in that when the program is executed by a processor, the method according to any one of claims 1-7 is implemented.

Images