Data storage method and device

Abstract

The present specification provides a data storage method and device, wherein the data storage method comprises: storing the i-th value of a target object to be stored in the n-th node at the j-th position, and creating an e-th pointer for the i-th value to point to the n-th node, wherein i, j, n, and e are all positive integers; when the value of the target object is updated, determining whether the n-th node is the last node at the j-th position, and if so, incrementing i, j, and e by 1, and continuing to execute the step of storing the i-th value of the target object to be stored in the n-th node at the j-th position; if not, incrementing i, n, and e by 1, and continuing to execute the step of storing the i-th value of the target object to be stored in the n-th node at the j-th position.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and more particularly to a data storage method and device. Background Art

[0002] With the development of computer technology, the computing speed of computers is constantly improving, but the data access speed of storage systems is improving slowly. The performance of modern computer systems is increasingly limited by the performance of memory systems based on DRAM (Dynamic Random Access Memory). The memory access speed has become one of the important bottlenecks restricting the development of computers. Therefore, many new memory devices have been proposed to replace traditional devices, among which NVM (non-volatile memory) memory devices are the most recognized type of memory devices. However, NVM storage media has a common problem: frequent updates of the same data block will cause the update speed of the data block to slow down, or even reduce its lifespan, and eventually become unusable. Especially in the existing KV (Key-Value system) storage system, frequent updates of hot spots will slow down read and write operations, and even affect the lifespan of NVM.

[0003] Therefore, a data storage method is urgently needed to solve the above problems. Summary of the invention

[0004] In view of this, an embodiment of the present specification provides a data storage method. The present specification also relates to a data storage device, a computing device, and a computer-readable storage medium to solve the technical defects existing in the prior art.

[0005] According to a first aspect of an embodiment of this specification, a data storage method is provided, including:

[0006] Store the i-th value of the target object to be stored in the n-th node at the j-th position, and create an e-th pointer for the i-th value to point to the n-th node, where i, j, n, and e are all positive integers;

[0007] When the value of the target object is updated, determine whether the nth node is the last node at the jth position.

[0008] If yes, then i, j, and e are all incremented by 1, and the step of storing the i-th value of the target object to be stored in the n-th node at the j-th position is continued.

[0009] If not, i, n, and e are all incremented by 1, and the step of storing the i-th value of the target object to be stored in the n-th node at the j-th position is continued.

[0010] Optionally, the j-th position is a target data block, the target data block includes multiple data groups, each data group includes n nodes, and the last node among the n nodes points to a node of the next target data block storing the value of the target object to be stored;

[0011] The step of storing the i-th value of the target object to be stored in the n-th node at the j-th position includes:

[0012] Obtaining a data key value corresponding to the i-th value of the target object to be stored, and calculating a hash value of the i-th value of the target object to be stored based on the data key value;

[0013] A storage tag of the i-th value of the target object to be stored is determined based on the hash value, and the i-th value of the target object to be stored is stored in the n-th node at the j-th position based on the storage tag.

[0014] Optionally, the pointer includes a storage flag bit;

[0015] Correspondingly, the step of creating the e-th pointer for the i-th value to point to the n-th node includes:

[0016] Create a pointer to the e-th storage flag bit based on the data obtained for the i-th time;

[0017] The pointer of the e-th storage flag bit is pointed to the n-th node.

[0018] Optionally, after the step of incrementing i, j, and e by 1 and continuing to store the i-th value of the target object to be stored in the n-th node at the j-th position, the step further includes:

[0019] In the case where the target data blocks before the i-th value of the target object to be stored are greater than or equal to a first preset number threshold, starting a compression thread;

[0020] All target data blocks storing the target object to be stored before the i-th value is taken are compressed.

[0021] Optionally, when the nth node is not the last node at the jth position, and after i, n, and e are all incremented by 1, the method further includes:

[0022] When the number of the nth node meets the second preset number threshold, and the target data block storing the target object to be stored before the i-th value is greater than or equal to the first preset number threshold, start the compression thread;

[0023] All target data blocks storing the target object to be stored before the i-th value is taken are compressed.

[0024] Optionally, compressing all target data blocks before the i-th value of the target object to be stored includes:

[0025] Generate a compressed target data block based on all target data blocks storing the target object to be stored before the i-th value is taken, and store the latest values ​​of all target data blocks storing the target object to be stored before the i-th value is taken into the compressed target data block;

[0026] Delete all target data blocks storing the target object to be stored before the i-th value is taken.

[0027] Optionally, the pointer includes an offset flag bit;

[0028] Correspondingly, determine whether n is the last node. If so, increment i, j, and e by 1. If the number of j is greater than or equal to the first preset number threshold, continue to store the i-th value of the target object to be stored in the n-th node at the j-th position, and update the offset flag of the e-th pointer based on the i-th value.

[0029] Based on the offset flag, an e-th pointer is created for the i-th value to point to the n-th node.

[0030] Optionally, the pointer includes an update flag bit;

[0031] After storing the latest values ​​of all target data blocks before the i-th value of the target object to be stored in the compressed target data block, the method further includes:

[0032] After the target data block is compressed and when the compressed target data block is generated, the update flag is adjusted.

[0033] Optionally, the target data block before the i-th value of the target object to be stored includes a head pointer;

[0034] Correspondingly, storing the latest values ​​of all target data blocks before the i-th value of the target object to be stored after the compressed target data block includes:

[0035] Moving the head pointer to the compression target data block;

[0036] Optionally, storing the i-th value of the target object to be stored after the n-th node at the j-th position includes:

[0037] The i-th value is read according to the j-th position pointed to by the e-th pointer.

[0038] According to a second aspect of an embodiment of this specification, there is provided a data storage device, including:

[0039] A storage module is configured to store the i-th value of the target object to be stored in the n-th node at the j-th position, and create an e-th pointer for the i-th value to point to the n-th node, wherein i, j, n, and e are all positive integers;

[0040] A judgment module is configured to judge whether the nth node is the last node at the jth position when the value of the target object is updated,

[0041] If yes, then i, j, and e are all incremented by 1, and the step of storing the i-th value of the target object to be stored in the n-th node at the j-th position is continued.

[0042] If not, i, n, and e are all incremented by 1, and the step of storing the i-th value of the target object to be stored in the n-th node at the j-th position is continued.

[0043] According to a third aspect of an embodiment of this specification, a computing device is provided, including:

[0044] Memory and processor;

[0045] The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions, wherein the processor implements the steps of the data storage method according to any one of claims 1 to 14 when executing the computer-executable instructions.

[0046] According to a fourth aspect of the embodiments of this specification, a computer-readable storage medium is provided, which stores computer-executable instructions, and when the instructions are executed by a processor, the steps of the data storage method are implemented.

[0047] The data storage method provided in the present specification stores the i-th value of the target object to be stored in the n-th node at the j-th position, and creates an e-th pointer for the i-th value to point to the n-th node, wherein i, j, n, and e are all positive integers; when the value of the target object is updated, it is determined whether the n-th node is the last node at the j-th position. If so, i, j, and e are all incremented by 1, and the step of storing the i-th value of the target object to be stored in the n-th node at the j-th position is continued. If not, i, n, and e are all incremented by 1, and the step of storing the i-th value of the target object to be stored in the n-th node at the j-th position is continued.

[0048] An embodiment of the present specification realizes that in a KV storage system, in order to avoid updating the acquired value of a storage target object in the same data block, the updated value of the storage target object is stored in a self-incremented data block, and at the same time, pointer data is created to point to the self-incremented data block to avoid frequent updates of the same data block for data storage, thereby improving the data update speed and extending the life of the NVM storage device. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 is a flow chart of a data storage method provided by an embodiment of this specification;

[0050] FIG. 2A to FIG. 2B This is a structural diagram of a data storage method provided by an embodiment of this specification;

[0051] FIG. 3A to FIG. 3C This is a metadata storage format structure diagram in a data storage method provided in an embodiment of this specification.

[0052] FIG. 4A to FIG. 4D This is a flowchart of compressing a target data block in a data storage method provided in an embodiment of the present specification.

[0053] Figure 5 It is a structural schematic diagram of a data storage device provided by an embodiment of this specification;

[0054] Figure 6 It is a structural block diagram of a computing device provided in an embodiment of this specification. DETAILED DESCRIPTION

[0055] Many specific details are described in the following description to facilitate a full understanding of this specification. However, this specification can be implemented in many other ways than those described herein, and those skilled in the art can make similar generalizations without violating the connotation of this specification, so this specification is not limited to the specific implementation disclosed below.

[0056] The terms used in one or more embodiments of this specification are only for the purpose of describing specific embodiments, and are not intended to limit one or more embodiments of this specification. The singular forms of "a", "said" and "the" used in one or more embodiments of this specification and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used in one or more embodiments of this specification refers to and includes any or all possible combinations of one or more associated listed items.

[0057] It should be understood that although the terms first, second, etc. may be used to describe various information in one or more embodiments of this specification, this information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of one or more embodiments of this specification, the first may also be referred to as the second, and similarly, the second may also be referred to as the first. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0058] First, the terms involved in one or more embodiments of this specification are explained.

[0059] NVM (full name in English: non-volatile memory, full name in Chinese: non-volatile memory) refers to computer memory in which the stored data does not disappear when the current is turned off.

[0060] KV system (English full name: Key-Value system, Chinese full name: key-value system): In the key-value system storage, its data is organized, indexed and stored in the form of key-value pairs.

[0061] CAS (Compare And Swap, Atomic Replacement Operation) refers to an operation that will not be interrupted by the thread scheduling mechanism. Once this operation starts, it will run until it ends without any thread context switching in the middle.

[0062] In this specification, a data storage method is provided. This specification also relates to a data storage device, a computing device, and a computer-readable storage medium, which are described in detail one by one in the following embodiments.

[0063] In the traditional data storage scenario, SRAM (full name: Static Random-Access Memory, Chinese full name: static random access memory), DRAM, and disks are widely used and have mature processes. They have become the mainstream of storage devices for many years. SRAM is usually used as a high-speed cache between the CPU (full name: Central Processing Unit) and the memory due to its fastest read and write speed. At the same time, due to its low integration and high price, it is rarely used as a memory. DRAM is generally used as a storage medium for memory and as a cache part of SSD (full name: Solid State Drives) hard disks due to its fast read and write speed and high integration. Disks are cheap, highly integrated, and have stable storage states and are not easily lost. NVM read and write access delays are close to DRAM, but the cost is cheaper. It is expected to replace the existing DRAM+SSD storage architecture in the future and provide large-capacity, persistent memory services. However, NVM storage media has a common problem: frequent updates to the same data block will cause the update speed of the data block to slow down, or even reduce its lifespan, and eventually become unavailable. This problem is more serious in existing KV storage systems. This is because there are frequent hotspot updates and read scenarios in KV storage systems. Frequent updates of hotspots will slow down read and write operations and even affect the life of NVM. One or more embodiments of this specification provide a data storage method to address the hot update problem of NVM, and propose a new KV system index design to avoid damage to NVM data blocks due to hot updates.

[0064] See also Figure 1 , Figure 1 A flowchart of a data storage method provided according to an embodiment of the present specification is shown, which specifically includes the following steps:

[0065] Step 102: Store the i-th value of the target object to be stored in the n-th node at the j-th position, and create an e-th pointer for the i-th value to point to the n-th node, where i, j, n, and e are all positive integers.

[0066] Among them, the target object can be understood as the subject that stores the data to be stored. For example, in the application scenario of storing lipstick inventory data, the target object is lipstick; in the application scenario of storing perfume inventory data, the target object is perfume; the i-th value of the target object to be stored can be understood as the data storing the lipstick inventory or the data storing the perfume inventory; the j-th position can be understood as the target data block that stores the i-th value of the target object to be stored; the n-th node can be understood as the node that stores the i-th value of the target object to be stored in the target data block; the e-th pointer can be understood as the pointer data created by the i-th value of the target object to be stored.

[0067] Specifically, taking the lipstick flash sale scenario during a shopping festival, storing lipstick inventory data as an example, when i, j, n, and e are 1, the data of the first value of the lipstick inventory is stored in the first node in the first target data block, and the first pointer is created for the first value to point to the first node in the first target data block.

[0068] Further, the j-th position is a target data block, the target data block includes multiple data groups, each data group includes n nodes, and the last node among the n nodes points to the node of the next target data block storing the value of the target object to be stored;

[0069] The step of storing the i-th value of the target object to be stored in the n-th node at the j-th position includes:

[0070] Obtaining a data key value corresponding to the i-th value of the target object to be stored, and calculating a hash value of the i-th value of the target object to be stored based on the data key value;

[0071] A storage tag of the i-th value of the target object to be stored is determined based on the hash value, and the i-th value of the target object to be stored is stored in the n-th node at the j-th position based on the storage tag.

[0072] The target data block may be understood as the corresponding data block stored by the i-th value of the target object to be stored, and the data group may be understood as a node group composed of multiple nodes.

[0073] Specifically, the target data block includes multiple data groups, each data group includes n nodes, and the last node among the n nodes will point to the node where the next target data block is stored in the target object value to be stored. In the process of storing the i-th value of the target object to be stored in the n-th node at the j-th position, it is necessary to obtain the data key value corresponding to the i-th value of the target object to be stored, and calculate the hash value corresponding to the i-th value of the target object to be stored based on the data key value. According to the hash value of the i-th value obtained after the calculation, the storage label of the i-th value of the target object to be stored is determined, and based on the storage label, the i-th value of the target object to be stored is stored in the n-th node at the j-th position.

[0074] In practical applications, if n is 16, the target data block can be divided into 16 nodes, where the last node points to the pointer of the next target data block, and the remaining 15 nodes are divided into 5 data groups, each with 3 nodes; according to the data key value corresponding to the i-th value of the target object to be stored, each data group can only store data with the same data key. If the data group is full, the next target data block is created.

[0075] For example, taking the storage of lipstick inventory data in the shopping festival flash sale scenario as an example, the first value of the lipstick inventory is obtained, and the hash value corresponding to the lipstick inventory data is calculated according to the data key value corresponding to the first value to determine the storage label of the lipstick inventory data, and the lipstick inventory data is stored in the first node at the first position based on the storage label. In actual applications, in the shopping festival flash sale scenario, the lipstick inventory data can be obtained in units of 1 second, that is, in the second second, the second value of the lipstick inventory is obtained, and the hash value corresponding to the lipstick inventory data is calculated according to the data key value corresponding to the second value to determine the storage label of the lipstick inventory data, and the lipstick inventory data is stored in the second node at the second position based on the storage label, and so on, the lipstick inventory is stored in different nodes in different target data blocks.

[0076] The data storage method provided in the embodiments of this specification, when data is frequently updated, frequently updates the same data in a data block, and the data block may be damaged due to hot updates. In the present application, the value data of the target object to be stored obtained at different times are stored in different data blocks and different nodes, thereby avoiding frequent data updates in the same data block and avoiding damage to the data block due to hot updates.

[0077] Further, the pointer includes a storage flag bit;

[0078] Correspondingly, the step of creating the e-th pointer for the i-th value to point to the n-th node includes:

[0079] Create a pointer to the e-th storage flag bit based on the data obtained for the i-th time;

[0080] The pointer of the e-th storage flag bit is pointed to the n-th node.

[0081] Specifically, the pointer created by taking the value of the target object to be stored includes a storage flag bit, and the pointer including the storage flag bit is pointed to the node where the target object to be stored is stored.

[0082] The data storage method provided in the embodiments of this specification creates a pointer with a storage flag bit to store the data of the i-th value of the target object to be stored in the node pointed to by the storage flag bit in order to more quickly find the node storing the value of the target object, thereby storing the value of the target object to be stored more quickly and conveniently.

[0083] Step 104: When the value of the target object is updated, determine whether the nth node is the last node at the jth position.

[0084] Specifically, following the above example, each time the acquired lipstick inventory data is updated, it is determined whether the node storing the updated lipstick inventory data in the target data block is the last node of the stored target data block.

[0085] Further, the pointer includes an offset flag bit;

[0086] Correspondingly, determine whether n is the last node. If so, increment i, j, and e by 1. If the number of j is greater than or equal to the first preset number threshold, continue to store the i-th value of the target object to be stored in the n-th node at the j-th position, and update the offset flag of the e-th pointer based on the i-th value.

[0087] Based on the offset flag, an e-th pointer is created for the i-th value to point to the n-th node.

[0088] Among them, the offset flag bit can be understood as representing the arrangement order of the target data blocks. For example, the offset flag bit of the first target data block is 0, the offset flag bit of the second target data block is 1, the offset flag bit of the third target data block is 2, and so on.

[0089] Specifically, determine whether the node in the target data block storing the target object to be stored is the last node of the target data block. If so, i, j, and e are all incremented by 1. When the number of target data blocks is greater than or equal to the first preset number threshold, continue to store the i-th value of the target object to be stored in the node corresponding to the corresponding target data block, and adjust the offset flag of the e-th pointer in the i-th value based on the updated value.

[0090] The data storage method provided in the embodiments of the present specification determines the specific location where the updated value of the target object to be stored is stored by judging whether the number of target data blocks is greater than or equal to a first preset number threshold value when the number of value acquisitions of the target object and the location of the target data block are continuously updated, thereby ensuring the accuracy of the storage location.

[0091] Step 106: If yes, then i, j, and e are all incremented by 1, and the step of storing the i-th value of the target object to be stored in the n-th node at the j-th position is continued.

[0092] Specifically, if the node where the i-th value of the target object to be stored is stored is the last node of the target data block, i, j, and e are all incremented by 1, and the step of storing the i-th value of the target object to be stored in the n-th node at the j-th position is continued.

[0093] Furthermore, after the step of incrementing i, j, and e by 1 and continuing to store the i-th value of the target object to be stored in the n-th node at the j-th position, the step further includes:

[0094] In the case where the target data blocks before the i-th value of the target object to be stored are greater than or equal to a first preset number threshold, starting a compression thread;

[0095] All target data blocks storing the target object to be stored before the i-th value is taken are compressed.

[0096] The compression thread may be understood as a thread for compressing target data blocks started by the background server when the background server continuously detects that the number of target data blocks increases.

[0097] Specifically, the i-th value of the target object to be stored is stored in the target data block. For example, following the above example, the lipstick inventory data is stored in the node of the target data block. When the lipstick inventory is continuously updated, the number of stored target data blocks is also continuously increasing. When the number of continuously stored target data blocks is greater than or equal to a first preset number threshold, wherein the first preset number threshold may be 2, 3, etc., the background server starts a compression thread for compressing the target data blocks stored in the lipstick inventory data. In actual applications, for example, if there are already three data blocks storing the lipstick inventory, the first target data block and the second target data block are compressed to obtain a compressed target data block.

[0098] The data storage method provided in the embodiments of the present specification compresses the stored target data blocks to reduce the number of stored target data blocks when the values ​​of the target objects to be stored are continuously updated in order to ensure the number of target data blocks storing the values ​​of the target objects to be stored. This not only reduces the storage space but also increases the data update speed when the values ​​of the target objects are continuously stored.

[0099] Further, the pointer includes an update flag bit;

[0100] After storing the latest values ​​of all target data blocks before the i-th value of the target object to be stored in the compressed target data block, the method further includes:

[0101] After the target data block is compressed and when the compressed target data block is generated, the update flag is adjusted.

[0102] The update flag can be understood as adjusting the update flag to represent that compression has been performed after the background server starts the compression thread and compresses two target data blocks into one target data block.

[0103] Specifically, the pointer to the i-th value of the target object to be stored includes an update flag. After the compression thread is started for all target data blocks before the i-th value of the target object to be stored, the compressed target data block is obtained, and the update flag in the pointer to the i-th value of the target object to be stored is adjusted.

[0104] The data storage method provided in the embodiments of the present specification compresses the target data blocks storing the values ​​of the target objects to be stored to ensure that the number of stored target data blocks is reduced, thereby not only increasing the storage space but also increasing the data update speed.

[0105] Furthermore, compressing all target data blocks before the i-th value of the target object to be stored includes:

[0106] Generate a compressed target data block based on all target data blocks storing the target object to be stored before the i-th value is taken, and store the latest values ​​of all target data blocks storing the target object to be stored before the i-th value is taken into the compressed target data block;

[0107] Delete all target data blocks storing the target object to be stored before the i-th value is taken.

[0108] Specifically, when all target data blocks before the i-th value of the target object to be stored are compressed to obtain a compressed target data block, the latest values ​​of all target data blocks before the i-th value of the target object to be stored are stored in the compressed target data block, and all target data blocks before the i-th value of the target object to be stored are deleted.

[0109] In actual applications, following the above example, the latest acquired lipstick inventory data is stored in the compressed target data block, and the data before the latest acquired lipstick inventory data is deleted to reduce storage space. For example, if the latest acquired lipstick inventory is A4, A4 is stored in the compressed target data block, and the lipstick inventory data values ​​A1, A2, and A3 are deleted.

[0110] The data storage method provided in the embodiment of this specification stores the latest acquired value in the compression target data block through the compression thread, which not only avoids the data storage from frequently updating the same data block, but also deletes the previously acquired value data to reduce the data storage space and speed up the data block update speed.

[0111] Step 108: If not, then i, n, and e are all incremented by 1, and the step of storing the i-th value of the target object to be stored in the n-th node at the j-th position is continued.

[0112] Specifically, if the node where the i-th value of the target object to be stored is stored is not the last node of the target data block, i, n, and e are all incremented by 1, and the step of storing the i-th value of the target object to be stored in the n-th node at the j-th position is continued.

[0113] Furthermore, when the nth node is not the last node at the jth position, and after i, n, and e are all incremented by 1, the method further includes:

[0114] When the number of the nth node meets the second preset number threshold, and the target data block storing the target object to be stored before the i-th value is greater than or equal to the first preset number threshold, start the compression thread;

[0115] All target data blocks storing the target object to be stored before the i-th value is taken are compressed.

[0116] Specifically, when the number of the nth node storing the i-th value of the data of the target object to be stored meets the second preset number threshold, and the target data blocks storing before the i-th value of the target object to be stored are greater than or equal to the first preset number threshold, the compression thread is started at the same time to compress all target data blocks storing before the i-th value of the target object to be stored.

[0117] The data storage method provided in the embodiment of this specification starts a compression thread to ensure the number of target data blocks and reduce storage space by judging whether the number of n-th nodes of the value data storage of the target object to be stored meets the second preset number threshold and whether the number of target data blocks is greater than or equal to the first preset number threshold.

[0118] Further, the target data block before the i-th value of the target object to be stored includes a head pointer;

[0119] Correspondingly, storing the latest values ​​of all target data blocks before the i-th value of the target object to be stored after the compressed target data block includes:

[0120] Moving the head pointer to the compression target data block;

[0121] Specifically, the target data block before the i-th value of the target object to be stored includes a head pointer, and after the compression thread is started to obtain the compressed target data block, the head pointer of the i-th value of the target object to be stored is moved to the compressed target data block.

[0122] The data storage method provided in the embodiments of the present specification can quickly search for the value data of the target object to be stored in the compressed target data block through the head pointer by moving the head pointer to point to the compressed target data block.

[0123] Furthermore, the step of storing the i-th value of the target object to be stored after the n-th node at the j-th position includes:

[0124] The i-th value is read according to the j-th position pointed to by the e-th pointer.

[0125] Specifically, after the i-th value of the target object to be stored is stored in the n-th node at the j-th position, the i-th value of the target object to be stored can be traversed and read according to the position of the j-th data block pointed to by the e-th pointer created by the i-th value of the target object to be stored.

[0126] The data storage method provided in the embodiments of the present specification can save data reading time by quickly finding the stored value data of the target object to be stored according to the data storage location pointed to by the created pointer data when reading the value data of the target object to be stored.

[0127] The following combination FIG. 2A to FIG. 2B , FIG. 2A to FIG. 2B A structural schematic diagram of a data storage method provided by an embodiment of the present specification is shown.

[0128] Figure 2A It is a cache tail data index structure in the DRAM system and stores a compression pool. Figure 2B It is a new index structure in NVM. The value data of the target object to be stored is created into a head pointer and stored in the hash table. The conflict chain consists of a 128-byte data block for each item. Each data block is divided into 16 nodes, and the last node is a pointer to the next data block. The remaining 15 nodes are divided into 5 data groups, each data group has 3 nodes. Each data group can only store data keys with the same label. If the data group is full, the next data block is created.

[0129] The following combination FIG. 3A to FIG. 3C , FIG. 3A to FIG. 3C A metadata storage format structure diagram in a data storage method provided in an embodiment of this specification is shown.

[0130] Figure 3A The data format of the cache index in the DRAM includes an offset byte, wherein the offset byte contains 16 bytes of data; Figure 3B The data format of the head pointer in NVM includes storage bytes, update bytes, and offset bytes; Figure 3CThe format of each data block in the collision chain, each data block contains 128 bytes.

[0131] Furthermore, the DRAM cache optimization process is as follows:

[0132] When a write operation creates a new data block, if the length of the conflict chain is less than the threshold, the DRAM cache is not started and normal traversal access is performed.

[0133] If the length of the conflict chain is greater than the threshold, the storage flag is set and the DRAM cache is started. The offset flag in the DRAM corresponds to the new data block, and the pointer also points to the new data block (after CAS successfully adds the data block, CAS modifies the head pointer to complete the storage flag setting). If the storage flag is already set, the DRAM cache can be updated.

[0134] When the compression thread deletes a data block, if the storage flag is set, the compression thread determines whether the cached offset flag is within the compression range. If it is within the compression range, the atomic replacement operation will set the storage flag to zero at the end of compression. If it is not within the compression range, the storage flag is not modified.

[0135] The following combination FIG. 4A to FIG. 4D , FIG. 4A to FIG. 4D A flowchart of compressing a target data block in a data storage method provided in an embodiment of the present specification is shown.

[0136] Figure 4A : Compression initialization.

[0137] Specifically, there are 5 data keys ABCDE in the conflict chain, among which A has been updated the most times, 4 times, and the latest data is A4. A and E have the same label, so they are in the same data group. Similarly, C and D are in the same data group.

[0138] Figure 4B : Prepare for compression.

[0139] Specifically, the offset flag bit in the head pointer is set to 2 to point to the latest allocated data block.

[0140] Figure 4C : Compression is performed.

[0141] Specifically, the data blocks with offset flags 0 and 1 are compressed. The compressed data blocks are first generated in DRAM and then written back to NVM. During this period, the newly updated data will be written to the latest allocated data block.

[0142] Figure 4D : Compression completed.

[0143] Specifically, the CAS operation modifies the head pointer to point to the compressed data block, and updates the target data block and offset flag at the same time.

[0144] Corresponding to the above method embodiment, this specification also provides a data storage device embodiment, Figure 5 FIG. 1 shows a schematic diagram of a data storage device provided by an embodiment of the present specification. Figure 5 As shown, the device comprises:

[0145] The storage module 502 is configured to store the i-th value of the target object to be stored in the n-th node at the j-th position, and create an e-th pointer for the i-th value to point to the n-th node, where i, j, n, and e are all positive integers;

[0146] The judgment module 504 is configured to judge whether the nth node is the last node at the jth position when the value of the target object is updated.

[0147] If yes, then i, j, and e are all incremented by 1, and the step of storing the i-th value of the target object to be stored in the n-th node at the j-th position is continued.

[0148] If not, i, n, and e are all incremented by 1, and the step of storing the i-th value of the target object to be stored in the n-th node at the j-th position is continued.

[0149] Optionally, the storage module 502 is further configured to:

[0150] Obtaining a data key value corresponding to the i-th value of the target object to be stored, and calculating a hash value of the i-th value of the target object to be stored based on the data key value;

[0151] A storage tag of the i-th value of the target object to be stored is determined based on the hash value, and the i-th value of the target object to be stored is stored in the n-th node at the j-th position based on the storage tag.

[0152] Optionally, the storage module 502 is further configured to:

[0153] Create a pointer to the e-th storage flag bit based on the data obtained for the i-th time;

[0154] The pointer of the e-th storage flag bit is pointed to the n-th node.

[0155] Optionally, the storage module 502 is further configured to:

[0156] In the case where the target data blocks before the i-th value of the target object to be stored are greater than or equal to a first preset number threshold, starting a compression thread;

[0157] All target data blocks storing the target object to be stored before the i-th value is taken are compressed.

[0158] Optionally, the storage module 502 is further configured to:

[0159] When the number of the nth node meets the second preset number threshold, and the target data block storing the target object to be stored before the i-th value is greater than or equal to the first preset number threshold, start the compression thread;

[0160] All target data blocks storing the target object to be stored before the i-th value is taken are compressed.

[0161] Optionally, the storage module 502 is further configured to:

[0162] Generate a compressed target data block based on all target data blocks storing the target object to be stored before the i-th value is taken, and store the latest values ​​of all target data blocks storing the target object to be stored before the i-th value is taken into the compressed target data block;

[0163] Delete all target data blocks storing the target object to be stored before the i-th value is taken.

[0164] Optionally, the determining module 504 is further configured to:

[0165] Correspondingly, determine whether n is the last node. If so, increment i, j, and e by 1. If the number of j is greater than or equal to the first preset number threshold, continue to store the i-th value of the target object to be stored in the n-th node at the j-th position, and update the offset flag of the e-th pointer based on the i-th value.

[0166] Based on the offset flag, an e-th pointer is created for the i-th value to point to the n-th node.

[0167] Optionally, the storage module 502 is further configured to:

[0168] After storing the latest values ​​of all target data blocks before the i-th value of the target object to be stored in the compressed target data block, the method further includes:

[0169] After the target data block is compressed and when the compressed target data block is generated, the update flag is adjusted.

[0170] Optionally, the storage module 502 is further configured to:

[0171] Moving the head pointer to the compression target data block;

[0172] Optionally, the storage module 502 is further configured to:

[0173] The i-th value is read according to the j-th position pointed to by the e-th pointer.

[0174] The above is a schematic scheme of a data storage device of this embodiment. It should be noted that the technical scheme of the data storage device and the technical scheme of the data storage method described above are of the same concept, and the details of the technical scheme of the data storage device that are not described in detail can be found in the description of the technical scheme of the data storage method described above.

[0175] Figure 6 The structure block diagram of a computing device 600 provided according to an embodiment of the present specification is shown. The components of the computing device 600 include but are not limited to a memory 610 and a processor 620. The processor 620 is connected to the memory 610 via a bus 630, and the database 650 is used to store data.

[0176] The computing device 600 also includes an access device 640 that enables the computing device 600 to communicate via one or more networks 660. Examples of these networks include a public switched telephone network (PSTN), a local area network (LAN), a wide area network (WAN), a personal area network (PAN), or a combination of communication networks such as the Internet. The access device 640 may include one or more of any type of network interface (e.g., a network interface card (NIC)) whether wired or wireless, such as an IEEE 802.11 wireless local area network (WLAN) wireless interface, a World Wide Interoperability for Microwave Access (Wi-MAX) interface, an Ethernet interface, a universal serial bus (USB) interface, a cellular network interface, a Bluetooth interface, a near field communication (NFC) interface, and the like.

[0177] In one embodiment of the present specification, the above components of the computing device 600 and Figure 6 Other components not shown in the figure may also be connected to each other, for example, via a bus. It should be understood that Figure 6 The computing device structure block diagram shown is only for the purpose of illustration, and is not intended to limit the scope of this specification. Those skilled in the art can add or replace other components as needed.

[0178] The computing device 600 may be any type of stationary or mobile computing device, including a mobile computer or mobile computing device (e.g., a tablet computer, a personal digital assistant, a laptop computer, a notebook computer, a netbook, etc.), a mobile phone (e.g., a smart phone), a wearable computing device (e.g., a smart watch, smart glasses, etc.), or other types of mobile devices, or a stationary computing device such as a desktop computer or PC. The computing device 600 may also be a mobile or stationary server.

[0179] The processor 620 is used to execute the following computer executable instructions, wherein the processor implements the steps of the data storage method when executing the computer executable instructions.

[0180] The above is a schematic scheme of a computing device of this embodiment. It should be noted that the technical scheme of the computing device and the technical scheme of the above data storage method belong to the same concept, and the details not described in detail in the technical scheme of the computing device can be referred to the description of the technical scheme of the above data storage method.

[0181] An embodiment of the present specification also provides a computer-readable storage medium storing computer instructions, which implement the steps of the data storage method when executed by a processor.

[0182] The above is a schematic scheme of a computer-readable storage medium of this embodiment. It should be noted that the technical scheme of the storage medium and the technical scheme of the above data storage method belong to the same concept, and the details not described in detail in the technical scheme of the storage medium can be referred to the description of the technical scheme of the above data storage method.

[0183] The above is a description of a specific embodiment of the specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0184] The computer instructions include computer program codes, which may be in source code form, object code form, executable files or some intermediate forms, etc. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium, etc. It should be noted that the content contained in the computer-readable medium may be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electric carrier signals and telecommunication signals.

[0185] It should be noted that, for the convenience of description, the aforementioned method embodiments are all described as a series of action combinations, but those skilled in the art should be aware that this specification is not limited by the order of the actions described, because according to this specification, some steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by this specification.

[0186] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0187] The preferred embodiments of this specification disclosed above are only used to help explain this specification. The optional embodiments do not describe all the details in detail, nor do they limit the invention to only the specific implementation methods described. Obviously, many modifications and changes can be made according to the content of this specification. This specification selects and specifically describes these embodiments in order to better explain the principles and practical applications of this specification, so that technicians in the relevant technical field can well understand and use this specification. This specification is limited only by the claims and their full scope and equivalents.

Claims

1. A data storage method, comprising: The i-th value of the target object to be stored is stored in the n-th node at the j-th position, and an e-th pointer is created for the i-th value to point to the n-th node, wherein i, j, n, and e are all positive integers, the j-th position is the target data block storing the i-th value of the target object to be stored, and the n-th node is the node storing the i-th value of the target object to be stored in the target data block; When the value of the target object is updated, determine whether the nth node is the last node at the jth position. If yes, then i, j, and e are all incremented by 1, and the step of storing the i-th value of the target object to be stored in the n-th node at the j-th position is continued, where n starts from 1. If not, i, n, and e are all incremented by 1, and the step of storing the i-th value of the target object to be stored in the n-th node at the j-th position is continued.

2. The data storage method according to claim 1, wherein the j-th position is a target data block, the target data block includes multiple data groups, the target data block includes n nodes, the last node among the n nodes points to a node of the next target data block storing the value of the target object to be stored, and the sum of the number of nodes included in the multiple data groups is n-1; The step of storing the i-th value of the target object to be stored in the n-th node at the j-th position includes: Obtaining a data key value corresponding to the i-th value of the target object to be stored, and calculating a hash value of the i-th value of the target object to be stored based on the data key value; A storage tag of the i-th value of the target object to be stored is determined based on the hash value, and the i-th value of the target object to be stored is stored in the n-th node at the j-th position based on the storage tag.

3. The data storage method according to claim 1, wherein the pointer comprises a storage flag bit; Correspondingly, the step of creating the e-th pointer for the i-th value to point to the n-th node includes: Create a pointer to the e-th storage flag bit based on the data obtained for the i-th time; The pointer of the e-th storage flag bit is pointed to the n-th node.

4. The data storage method according to claim 2, after the step of incrementing i, j, and e by 1 and continuing to execute the step of storing the i-th value of the target object to be stored in the n-th node at the j-th position, further comprises: In the case where the target data blocks before the i-th value of the target object to be stored are greater than or equal to a first preset number threshold, starting a compression thread; Compression processing is performed on all target data blocks storing the target object to be stored before the i-th value is taken.

5. The data storage method according to claim 1, wherein after the nth node is not the last node at the jth position and i, n and e are all incremented by 1, the method further comprises: When the number of the nth node meets the second preset number threshold, and the target data block storing the target object to be stored before the i-th value is greater than or equal to the first preset number threshold, start the compression thread; All target data blocks storing the target object to be stored before the i-th value is taken are compressed.

6. The data storage method according to claim 4 or 5, wherein compressing all target data blocks before the i-th value of the target object to be stored comprises: Generate a compressed target data block based on all target data blocks storing the target object to be stored before the i-th value is taken, and store the latest values ​​of all target data blocks storing the target object to be stored before the i-th value is taken into the compressed target data block; Delete all target data blocks storing the target object to be stored before the i-th value is taken.

7. The data storage method according to claim 1, wherein the pointer comprises an offset flag; Correspondingly, determine whether n is the last node. If so, increment i, j, and e by 1. If the number of j is greater than or equal to the first preset number threshold, continue to store the i-th value of the target object to be stored in the n-th node at the j-th position, and update the offset flag of the e-th pointer based on the i-th value. Based on the offset flag, an e-th pointer is created for the i-th value to point to the n-th node.

8. The data storage method according to claim 4 or 5, wherein the pointer includes an update flag bit, and further comprises: After the target data block is compressed and a compressed target data block is generated, the update flag is adjusted.

9. The data storage method according to claim 6, wherein the target data block before the i-th value of the target object to be stored comprises a head pointer; Correspondingly, storing the latest values ​​of all target data blocks before the i-th value of the target object to be stored after the compressed target data block includes: The head pointer is moved to the compression target data block.

10. The data storage method according to claim 1, wherein the step of storing the i-th value of the target object to be stored after the n-th node at the j-th position comprises: The i-th value is read according to the j-th position pointed to by the e-th pointer.

11. A data storage device, comprising: A storage module is configured to store the i-th value of the target object to be stored in the n-th node at the j-th position, and create an e-th pointer for the i-th value to point to the n-th node, wherein i, j, n, and e are all positive integers, the j-th position is a target data block storing the i-th value of the target object to be stored, and the n-th node is a node storing the i-th value of the target object to be stored in the target data block; A judgment module is configured to judge whether the nth node is the last node at the jth position when the value of the target object is updated, If yes, then i, j, and e are all incremented by 1, and the step of storing the i-th value of the target object to be stored in the n-th node at the j-th position is continued, where n starts from 1. If not, i, n, and e are all incremented by 1, and the step of storing the i-th value of the target object to be stored in the n-th node at the j-th position is continued.

12. A computing device comprising: Memory and processor; The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions, wherein the processor implements the steps of the data storage method according to any one of claims 1 to 10 when executing the computer-executable instructions.

13. A computer-readable storage medium storing computer instructions, which, when executed by a processor, implement the steps of the data storage method according to any one of claims 1 to 10.

Images