LBA full cache method and system based on append-write distributed storage

Abstract

The present invention provides an LBA full cache method and system based on append-write distributed storage, the method comprising: decomposing a shared block storage addressing space into multiple blocks and hashing them into a storage cluster; defining a data structure in the full cache as a linear table; storing an index of a chunk in a chunk table in an item item, and addressing an LBA address using the content identified by the item item; when addressing occurs overwriting, the original location pointing to the chunk is an idle location, when a chunk data is overwritten by LBA or integrated and completely moved away, the chunk is identified as an idle location in the chunk_table linear table, and a new chunk_id is stored when a new chunk is written; and the chunks are integrated so that the total size of the chunk_table linear table does not exceed a threshold. The present invention enables distributed shared storage to cache the LBA addressing information of all shared storage devices while consuming less storage space.

Description

Technical Field

[0001] The present invention relates to the field of computer storage technology, in particular, to the field of distributed storage technology; specifically, to an LBA full cache method and system based on append-write distributed storage. Background Art

[0002] Distributed shared storage refers to the use of network communication to connect heterogeneous devices based on distributed algorithms to form a large-scale shared storage system. In distributed shared storage, a large-scale storage system is shared by multiple storage devices, and each storage device maintains its own LBA (Logical Block Address) in the large-scale storage system. Logical Block Address (LBA) is a common mechanism for describing the block where data is located on a computer storage device, and is generally used in auxiliary memory devices such as hard disks. LBA can refer to the address of a data block or the data block pointed to by an address.

[0003] In a distributed shared storage system, each user's shared storage device will have a separate addressing space for addressing the physical location of the shared device's logical block data in the distributed storage. The usual practice is to divide the address space of the shared device into fixed-length units, each of which is similar to a page in the kernel page table, called a virtual block. Correspondingly, the physical hard disk can be regarded as a fixed-length unit, each of which is a physical chunk. In order to record the location of the virtual block mapped to the physical chunk, the shared storage system needs to maintain the addressing data structure. The main function of the addressing data structure is to save the conversion of virtual blocks to physical blocks, so that the shared storage can find the actual physical storage location of the data. The addressing data structure is described in the distributed shared storage system as: lba:value. lba is a monotonically increasing sequence number used to describe a logically continuous, fixed-length data block, and value is the physical storage address of a logically continuous, fixed-length data block, usually: chunk_id, chunk_offset. In a distributed shared storage system, the LBA logical addressing mapping relationship is usually stored in the database as a KV.

[0004] In order to improve the access performance of LBA addressing mapping relationship, the following LBA organization methods are usually adopted: LBA variable length storage, or LBA fixed length storage. In the two storage methods, in order to improve LBA access performance, LBA information is usually cached in memory. The data structure of cache memory generally adopts a linked list structure or a tree structure elimination algorithm, such as LRU, LFU, etc. The specific cache LBA algorithm is usually as follows:

[0005] The cache content is:

[0006] Key=LBA,

[0007] Value={physical_chunk,physical_chunk_offset}

[0008] Left=left_point

[0009] Right=right_point

[0010] Among them, value contains the physical location information of the LBA logical block, and left / right contains the left and right pointers of the data structure.

[0011] The query writing algorithm is described as:

[0012] 1. Use LBA as the key to find the corresponding value in memory. The time complexity of the query is related to the choice of data structure.

[0013] 2. Copy the value and make updates or further queries.

[0014] 3. Perform a trim operation on cached data based on the overall memory usage.

[0015] However, due to the choice of data structure, the amount of data in the LBA memory cache will consume some storage space to maintain the data structure in addition to the valid data. Secondly, the storage space consumed by the physical location information of the LBA logical block in the memory is related to the cluster addressing space, which is usually very large, resulting in the inability of the traditional LBA cache algorithm to cache all LBA information. The cache elimination algorithm is usually used, which is not conducive to the integrity and reliability of the data cache.

[0016] Therefore, there is an urgent need to design an LBA full cache method, system and electronic device based on append-write distributed storage to solve the above-mentioned difficulties and pain points of the prior art. Summary of the invention

[0017] In view of this, the purpose of the present invention is to propose an LBA full cache method, system and device based on append-write distributed storage, so as to improve the performance of LBA addressing of shared devices in distributed shared storage, and reduce the memory space usage while fully caching LBA in memory; utilize the kernel multi-level page table to perform full memory cache on the shared storage LBA, improve the addressing performance of LBA and reduce the memory storage space occupancy, and utilize the append-write feature to abstract the chunk_table layer, reduce the addressing space of a logical LBA physical address, save the memory space of LBA full memory cache, and enable the distributed shared storage to cache the LBA addressing information of all shared storage devices while consuming less storage space, thereby improving the integrity of data cache.

[0018] The present invention provides an LBA full cache method based on append-write distributed storage, comprising the following steps:

[0019] S1. Decompose the shared block storage address space requested by the user into multiple blocks of fixed size, hash them into the storage cluster, and define each block as a region; split the address space of each region into multiple logical blocks of fixed size;

[0020] S2, defining the data structure of the addressing space of the region in the full cache as a linear table of the kernel page table structure, wherein each sub-item of the linear table corresponds to the addressing structure of each logical block;

[0021] S3, storing the index of the physical chunk in the chunk table in the addressing structure item, and when the LBA addresses the item item, using the content of the item item identifier to address the physical address of the LBA, thereby reducing the range of the LBA addressing space;

[0022] The present invention does not directly store chunks in the item item of the addressing structure of the logic block, but stores the index of the chunk in the chunk table. When the LBA addresses the item item, the content of the item item identifier is used to address the physical address of the LBA, so as to save storage space and reduce the range of the LBA addressing space.

[0023] S4. When an overwrite occurs in LBA addressing, the location of the original pointed chunk is an idle location. When the data of a physical chunk is overwritten by LBA or integrated and moved away, the chunk is marked as an idle location in the chunk_table linear table. When a new chunk is written to the idle location, the new chunk address: chunk_id is stored.

[0024] LBA full cache chunk_table is a reusable linear table. When all the data of a chunk address is moved away, the chunk address is marked as an idle position in the chunk_table linear table. When a new chunk address is written, the idle position is used to store the new chunk_id.

[0025] S5. Based on the cluster append write feature, when a chunk is full, a new chunk is replaced for writing, and an available free position is found in the chunk_table linear table to store the new chunk_id.

[0026] When part of the data in a chunk is overwritten to release free space, the chunks are integrated so that a large number of lba items continue to point to a chunk_id in the chunk_table, so that the total size of the chunk_table linear table does not exceed a threshold.

[0027] In the linear list of a region, the unwritten parts will no longer be allocated memory storage space.

[0028] Furthermore, the linear list of the region in step S2 is created as a multi-level storage structure, and the sub-list of each level is a linear list. In the multi-level storage structure, the upper level stores the address and status information of the lower level.

[0029] The multi-level storage structure of the region's linear list can be 2 levels or more.

[0030] Furthermore, the number of blocks LBA of the logic block in step S2 is the subscript of the linear table, and the subscript of the linear table is used when initiating an addressing query for region, and the number of blocks LBA of the logic block is not stored in reigon.

[0031] Furthermore, the structure of the linear table of the region is described as follows:

[0032] region_x=[lg1,lg2,lg3,……,lgn]

[0033] Among them, region_x represents any region after hashing, and lg represents the addressing structure of the logical block;

[0034] Since region_x is a linear list structure, it contains multiple logical blocks. To query the logical block addressing structure in a region, you only need to use the block number of the logical block as the subscript number of the linear list for access, without storing the logical block number as the addressing key.

[0035] Furthermore, the multi-level storage structure of the linear table of the region is 2 levels, and lba_table includes directory_table and the lower leaf linear table item_table;

[0036] Each element in directory_table points to a leaf linear list, and each element contains redundant bit fields to identify the status information of the leaf linear list pointed to; when no element is written to the leaf linear list, the leaf linear list does not allocate memory.

[0037] Furthermore, the leaf linear list contains multiple item elements, each leaf linear list is of fixed length; item_table contains information for addressing physical chunks; chunk_table contains all chunk_ids referenced by this region; in the item_table, index idx points to the chunk_id in chunk_table that belongs to the LBA write.

[0038] The present invention also provides an LBA full cache system based on append-write distributed storage, which executes the LBA full cache method based on append-write distributed storage as described above, and is characterized in that it is deployed in a distributed shared storage cluster.

[0039] The present invention also provides an LBA full cache device based on append-write distributed storage, which carries the LBA full cache system based on append-write distributed storage as described above.

[0040] The present invention also provides a computer-readable storage medium having a computer program stored thereon, and when the program is executed by a processor, the steps of the LBA full cache method based on append-write distributed storage as described above are implemented.

[0041] The present invention also provides a computer device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, the steps of the LBA full cache method based on append-write distributed storage as described above are implemented.

[0042] Compared with the prior art, the present invention has the following beneficial effects:

[0043] The present invention is based on the LBA full cache method, system and device of append-write distributed storage, which improves the performance of LBA addressing of shared devices in distributed shared storage, and reduces the memory space usage while fully caching LBA in memory; the idea of ​​kernel multi-level page table is used to perform full memory cache on shared storage LBA, which improves the addressing performance of LBA and reduces the memory storage space occupation; the append-write feature is used, and the chunk_table layer is abstracted to reduce the addressing space of a logical LBA physical address, saving the memory space of LBA full memory cache, so that the distributed shared storage can cache the LBA addressing information of all shared storage devices while consuming less storage space, thereby improving the integrity and reliability of data cache. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Various other advantages and benefits will become apparent to those of ordinary skill in the art by reading the following detailed description of the preferred embodiment.The drawings are only for the purpose of illustrating the preferred embodiments and are not to be construed as limiting the invention.

[0045] In the attached picture:

[0046] Figure 1 This is a schematic diagram of a two-level region full memory cache structure of an application example of the present invention;

[0047] Figure 2 A schematic diagram of a structure for finding an idle position in a linear table for an application example of the present invention;

[0048] Figure 3 This is a schematic diagram of the structure of a distributed shared storage cluster in an application example of the present invention;

[0049] Figure 4 A schematic diagram of the structure of an LBA full cache device based on append-write distributed storage according to an embodiment of the present invention;

[0050] Figure 5 A schematic diagram of a process of reading and writing a region LBA full cache according to an embodiment of the present invention;

[0051] Figure 6 It is a flow chart of the LBA full cache method based on append-write distributed storage of the present invention;

[0052] Figure 7 The figure is a schematic diagram of the structure of a computer device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0053] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present disclosure. Instead, they are merely examples of systems and products consistent with some aspects of the present disclosure as detailed in the appended claims.

[0054] The terms used in this disclosure are for the purpose of describing specific embodiments only and are not intended to limit the disclosure. The singular forms of "a", "said" and "the" used in this disclosure and the appended claims are also intended to include plural forms unless the context clearly indicates otherwise. It should also be understood that the term "and / or" used herein refers to and includes any or all possible combinations of one or more associated listed items.

[0055] It should be understood that although the terms first, second, third, etc. may be used in the present disclosure to describe various information, such information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of the present disclosure, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0056] The embodiments of the present invention are described in further detail below.

[0057] The embodiment of the present invention provides an LBA full cache method based on append-write distributed storage, see Figure 6 As shown, the following steps are included:

[0058] S1. Decompose the shared block storage address space requested by the user into multiple blocks of fixed size, hash them into the storage cluster, and define each block as a region; split the address space of each region into multiple logical blocks of fixed size;

[0059] S2, defining the data structure of the addressing space of the region in the full cache as a linear table of the kernel page table structure, wherein each sub-item of the linear table corresponds to the addressing structure of each logical block;

[0060] The linear table of region is created as a multi-level storage structure, and each level of the sub-table is a linear table. In the multi-level storage structure, the upper level stores the address and status information of the lower level.

[0061] The block number offset of the logical block is the subscript of the linear table. The subscript of the linear table is used when initiating an addressing query for the region, and the block number offset of the logical block is not stored in the reigon.

[0062] The structure of the linear list of the region is described as follows:

[0063] region_x=[lg1,lg2,lg3,……,lgn]

[0064] Among them, region_x represents any region after hashing, and lg represents the addressing structure of the logical block;

[0065] Since region_x is a linear list structure, it contains multiple logical blocks. To query the logical block addressing structure in a region, you only need to use the block number of the logical block as the subscript number of the linear list for access, without storing the logical block number as the addressing key.

[0066] In this embodiment, the multi-level storage structure of the linear table of the region is 2 levels, and lba_table includes directory_table and the lower leaf linear table item_table;

[0067] Each element in directory_table points to a leaf linear list, and each element contains redundant bit fields to identify the status information of the leaf linear list pointed to; when no element is written to the leaf linear list, the leaf linear list does not allocate memory.

[0068] The leaf linear list contains multiple item elements, each of which has a fixed length; item_table contains information for addressing physical chunks; chunk_table contains all chunk_ids referenced by this region; in the item_table, the index idx points to the chunk_id in chunk_table that belongs to the LBA write.

[0069] S3, storing the index of the physical chunk in the chunk table in the addressing structure item, and when the LBA addresses the item item, using the content of the item item identifier to address the physical address of the LBA, thereby reducing the range of the LBA addressing space;

[0070] In this embodiment, the item item of the addressing structure of the logic block does not directly store the chunk, but stores the index of the chunk in the chunk table. When the LBA addresses the item item, the content of the item item identifier is used to address the physical address of the LBA, so as to save storage space and reduce the range of the LBA addressing space.

[0071] S4. When an overwrite occurs in LBA addressing, the location of the original pointed chunk is an idle location. When the data of a physical chunk is overwritten by LBA or integrated and moved away, the chunk is marked as an idle location in the chunk_table linear table. When a new chunk is written to the idle location, the new chunk address: chunk_id is stored.

[0072] LBA full cache chunk_table is a reusable linear table. When all the data of a chunk address is moved away, the chunk address is marked as an idle position in the chunk_table linear table. When a new chunk address is written, the idle position is used to store the new chunk_id.

[0073] S5. Based on the cluster append write feature, when a chunk is full, a new chunk is replaced for writing, and an available free position is found in the chunk_table linear table to store the new chunk_id.

[0074] When part of the data in a chunk is overwritten to release free space, the chunks are integrated so that a large number of lba items continue to point to a chunk_id in the chunk_table, so that the total size of the chunk_table linear table does not exceed a threshold.

[0075] In the linear list of a region, the unwritten parts will no longer be allocated memory storage space.

[0076] The embodiment of the present invention also provides an LBA full cache system based on append-write distributed storage, which executes the LBA full cache method based on append-write distributed storage as described above, and is characterized in that it is deployed in a distributed shared storage cluster.

[0077] like Figure 3 As shown, an embodiment of the present invention also provides an LBA full cache device based on append-write distributed storage, carrying the LBA full cache system based on append-write distributed storage as described above.

[0078] Another preferred embodiment of the present invention is as follows Figure 4As shown, the system device with LBA full cache can also be a separate device. This device is responsible for caching the LBA address information that has been allocated to all system devices in the distributed shared storage cluster. When a shared device starts to use the distributed shared storage cluster, its LBA addressing information will be cached in the device cluster with LBA full cache. When reading and writing, the shared device will first calculate which device in the device cluster with LBA full cache the LBA is cached in based on the written LBA range, IO to the device to read the LBA physical location information, and then interact with the cluster system device through the physical location information. In a distributed shared storage cluster, all chunks in the cluster are allocated by a centralized chunk allocation manager.

[0079] Application Examples

[0080] First, the shared block storage addressing space requested by the user will be hashed to the storage cluster according to a fixed size, and each hashed block is defined as a region. The addressing space of each region is divided into logical blocks according to a fixed size, and the data structure of the region's addressing space in the full cache is defined as a linear table, and each subitem of the linear table is the addressing structure of the logical block. When initiating an addressing query for a region, the block number (offset) of the logical block is the subscript of the linear table. Therefore, in a reigon, the block number (offset) of the logical block does not need to be stored.

[0081] The structure of the region linear table is described as follows:

[0082] region_x=[lg1,lg2,lg3,……,lgn]

[0083] Among them, region_x represents any region after hashing, and lg represents the addressing structure of the logical block. Since region_x is a linear table structure, it contains multiple logical blocks. To query the addressing structure of the logical block in a region, you only need to use the block number of the logical block as the subscript number of the linear table for access, without storing the block number of the logical block as the addressing key.

[0084] Secondly, similar to the kernel page table structure, the region linear table is a multi-level storage structure, and each level of the sub-table is a linear table. In this way, the unwritten part of the region linear table will no longer be allocated memory storage space.

[0085] The multi-level storage structure of the region linear table can be 2 levels or more. In the multi-level storage structure, the upper level is used to store the address, status and other information of the lower level.

[0086] For example, in a 2-level storage structure, the top-level storage structure is defined as:

[0087] directory_table=[dir_item_1, dir_item_2,..., dir_item_n]

[0088] Each item in the directory_table linear list is a directory item, and each item can be a structure dir_item. dir_item can be defined as:

[0089] Struct{

[0090] item_table*item_table;

[0091] }

[0092] item_table is a pointer to a section of logical blocks in the region. When item_table is overwritten, item_table is a null pointer, that is, no memory space allocation occurs.

[0093] lba is the size of the offset addressed in the region to the left alignment of the region's logical block, that is, the number of blocks that the offset maps to the region's logical block. When the region consists of a multi-level linear table, lba addresses the item of the corresponding logical block, which is a recursive addressing process from the root node of the multi-level linear table to the leaf node.

[0094] An lba, in the region multi-level linear table, the addressing algorithm at each level of the linear table is defined as:

[0095] idx=(lba&mask)>>shift

[0096] item = item_table[idx]

[0097] Among them, mask is the mask of lba at each level in the region multi-level linear table, and shift is the number of bits of the mask of all linear tables at the next level. When the index address of an lba in a linear table item_table at a level is calculated, the index address idx can be used to directly retrieve the table information of the next level from item_table (when this item_table is the last level, the retrieved item is a leaf node, that is, an item addressed by an lba).

[0098] Again, after LBA addresses the item, the content identified by the item is used to address the physical address of LBA. In order to save storage space, item does not directly store the physical chunk, but stores the index of the physical chunk in the chunk table, thereby reducing the range of LBA addressing space.

[0099] A physical chunk is a storage space on a physical disk on a physical machine in a distributed shared storage. A chunk can be a continuous physical storage space on a physical disk, or a chunk can be a logical continuous space. In a distributed shared storage system, as the size of the cluster increases, the number of chunks increases linearly. The addressing space for the number of cluster chunks is very large, and the addressing space of chunk_id is generally identified by uint64.

[0100] An LBA to physical chunk addressing is defined as:

[0101] key:lba

[0102] value:{chunk_id,chunk_offset}

[0103] In a region multi-level linear table, the number of chunks that can be referenced is limited. Therefore, another chunk_table is abstracted to reduce the addressing space of the region multi-level linear table lba.

[0104] The chunk_table of a region multi-level linear table is defined as:

[0105] chunk_table=[chunk_id_1, chunk_id_70, chunk_id_500,..., chunk_id_N]

[0106] In chunk_table, each item is a chunk_id number in the distributed shared storage, which is defined as uint64.

[0107] A region multi-level linear table contains a chunk_table. The contents stored in an lba leaf item of the region multi-level linear table are:

[0108] Item:

[0109] {ck_idx;

[0110] chunk_lba;

[0111] }

[0112] Among them, ck_idx is the index number of the lba in the chunk_table, addressing chunk_id, and chunk_lba is the first block of the physical chunk. Not directly storing chunk_offset can reduce memory usage.

[0113] The addressing algorithm for addressing a chunk is defined as:

[0114] chunk_id=chunk_table[item.ck_idx]

[0115] chunk_offset=item.chunk_lba*logical_block_size

[0116] Among them, logical_block_size is the size of a logical block in the region.

[0117] See also Figure 1 As shown in the figure above, the structure of a two-level region full memory cache is as shown in the figure above. The lba_table contains directory_table and the lower-level leaf linear list item_table. Each element in directory_table points to a leaf linear list. There can be extra bit fields to identify the status information of the leaf linear list it points to. When there is no element written in the leaf linear list, the leaf linear list does not allocate memory. A leaf linear list contains multiple item elements, and each leaf linear list has a fixed length. The item_table contains information on addressing physical chunks. The chunk_table contains all chunkids referenced by this region. Among them, in item_table, idx points to the chunkid in chunk_table that belongs to the lba write.

[0118] The capacity of a chunk should be an integer multiple of the region's minimum logical block (LBA block size). When the capacity of a chunk is an integer multiple of the region's minimum logical block, multiple item items in item_table will point to a chunk in chunk_table at the same time. Therefore, the capacity of chunk_table can be significantly reduced. Based on the cluster's append-write feature, when a region is full of a chunk, the chunk will be replaced.

[0119] Finally, the LBA full cache chunk_table is a reusable linear table. When LBA is overwritten, the location of the original pointed chunk is an idle location. When all the data of a chunk is moved away, this chunk can be marked as an idle location in the chunk_table linear table. When a new chunk is written, the idle location is used to store the new chunk_id.

[0120] Figure 5 The process of reading and writing a region LBA full cache for this application example.

[0121] Based on the characteristics of cluster append writing, when a chunk is full, a new chunk will be replaced for writing. When replacing a new chunk, an available free space will be found in the chunk_table linear table to store the new chunk_id. Or, when there is some data in a chunk, due to the free space released by overwriting, the system will integrate the chunks so that a considerable number of lba items continue to point to a chunk_id in the chunk_table. This ensures that the total size of the chunk_table linear table does not exceed a threshold.

[0122] like Figure 2 As shown in the figure, the timer will find out which slots are free in the chunk_table linear list in advance and can store new chunk_ids. The timer also has a garbage collection function, which will clear chunk_ids that have no reference relationship in chunk_table and mark them as available slots.

[0123] When updating key:lba value:{chunk_id,chunk_lba}, the write process is:

[0124] 1. Chunk_table update: Use the chunk_id carried in the update to find the writable slot (idx) of chunk_table. The slot search method is

[0125] a) Check whether chunk_id is equal to the active of this region. If they are equal, it means that the chunk is appended and there is no need to update chunk_table. The data is returned directly.

[0126] b) chunk_id is not equal to the active of this region, which means that this region has changed the chunk to write. From the collection queue of the timer, take a slot chunk_id_idx for update, and change the active of this region to the current chunk and the slot of this chunk: idx.

[0127] 2. Use lba to address the lba table, find the location of the item corresponding to the lba, update chunk_lba, and the slot of chunkid: idx.

[0128] For example: when writing lba 3,{chunk_id:1,chunk_off:20}:

[0129] (1) Determine whether the active chunk_id is 1. If it is 1, it means that the chunk has not been changed and it is an append write of chunk_id:1. There is no need to update chunk_table.

[0130] (2) Use lba to address the lba table and find the item corresponding to lba. Update chunk_lba and chunk_idx.

[0131] An embodiment of the present invention further provides a computer device, Figure 7 is a schematic diagram of the structure of a computer device provided by an embodiment of the present invention; see the accompanying drawings Figure 7 As shown, the computer device includes: an input system 23, an output system 24, a memory 22 and a processor 21; the memory 22 is used to store one or more programs; when the one or more programs are executed by the one or more processors 21, the one or more processors 21 implement the LBA full cache method based on append-write distributed storage provided in the above embodiment; wherein the input system 23, the output system 24, the memory 22 and the processor 21 can be connected by a bus or other means, Figure 7 The example of connecting through bus is taken in the following.

[0132] The memory 22 is a readable and writable storage medium of a computing device, which can be used to store software programs and computer executable programs, such as program instructions corresponding to the LBA full cache method based on append-write distributed storage described in the embodiment of the present invention; the memory 22 can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system and at least one application required for a function; the data storage area can store data created according to the use of the device, etc.; in addition, the memory 22 can include a high-speed random access memory, and can also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage device; in some instances, the memory 22 can further include a memory remotely arranged relative to the processor 21, and these remote memories can be connected to the device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0133] The input system 23 may be used to receive input digital or character information, and to generate key signal input related to user settings and function control of the device; the output system 24 may include display devices such as display screens.

[0134] The processor 21 executes various functional applications and data processing of the device by running software programs, instructions and modules stored in the memory 22, that is, realizes the above-mentioned LBA full cache method based on append-write distributed storage.

[0135] The computer device provided above can be used to execute the LBA full cache method based on append-write distributed storage provided in the above embodiment, and has corresponding functions and beneficial effects.

[0136] The embodiment of the present invention also provides a storage medium containing computer executable instructions, which are used to execute the LBA full cache method based on append-write distributed storage as provided in the above embodiment when executed by a computer processor. The storage medium is any of various types of memory devices or storage devices, and the storage medium includes: installation media, such as CD-ROM, floppy disk or tape system; computer system memory or random access memory, such as DRAM, DDRRAM, SRAM, EDO RAM, Rambus RAM, etc.; non-volatile memory, such as flash memory, magnetic media (such as hard disk or optical storage); registers or other similar types of memory elements, etc.; the storage medium may also include other types of memory or a combination thereof; in addition, the storage medium may be located in a first computer system in which the program is executed, or may be located in a different second computer system, the second computer system is connected to the first computer system via a network (such as the Internet); the second computer system may provide program instructions to the first computer for execution. The storage medium includes two or more storage media that can reside in different locations (for example, in different computer systems connected via a network). The storage medium can store program instructions (for example, specifically implemented as a computer program) that can be executed by one or more processors.

[0137] Of course, the storage medium containing computer executable instructions provided by an embodiment of the present invention is not limited to the LBA full cache method based on appending write distributed storage as described in the above embodiment, and can also execute related operations in the LBA full cache method based on appending write distributed storage provided by any embodiment of the present invention.

[0138] So far, the technical solutions of the present invention have been described in conjunction with the preferred embodiments shown in the accompanying drawings. However, it is easy for those skilled in the art to understand that the protection scope of the present invention is obviously not limited to these specific embodiments. Without departing from the principle of the present invention, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions after these changes or substitutions will fall within the protection scope of the present invention.

[0139] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. An LBA full cache method based on append-write distributed storage, characterized in that: The following steps are involved: S1. Decompose the shared block storage address space requested by the user into multiple blocks of fixed size, hash them into the storage cluster, and define each block as a region. Split the address space of each region into multiple logical blocks of fixed size; S2, defining the data structure of the addressing space of the region in the full cache as a linear table of the kernel page table structure, wherein each sub-item of the linear table corresponds to the addressing structure of each logical block; S3, storing the index of the physical chunk in the chunk_table linear table in the addressing structure item, when the LBA addresses the item item, using the content of the item item identifier to address the physical address of the LBA, reducing the range of the LBA addressing space; S4. When an overwrite occurs in LBA addressing, the location of the original pointed chunk is an idle location. When the data of a physical chunk is overwritten by LBA or integrated and moved away, the chunk is marked as an idle location in the chunk_table linear table. When a new chunk is written to the idle location, the new chunk address: chunk_id is stored. S5. Based on the feature of cluster append writing, when a chunk is full, a new chunk is replaced for writing, and an available free position is found in the chunk_table linear table to store the new chunk_id. When part of the data in a chunk is overwritten to release free space, the chunks are integrated so that a large number of LBAitem items continue to point to a chunk_id in the chunk_table, so that the total size of the chunk_table linear table does not exceed a threshold.

2. The LBA full cache method based on append-write distributed storage according to claim 1, characterized in that: The linear list of the region in step S2 is created as a multi-level storage structure, and the sub-table of each level is a linear list. In the multi-level storage structure, the upper level stores the address and status information of the lower level.

3. The LBA full cache method based on append-write distributed storage according to claim 1, characterized in that: The number of blocks LBA of the logic block in step S2 is the subscript of the linear table. The subscript of the linear table is used when initiating an addressing query for region, and the number of blocks LBA of the logic block is not stored in reigon.

4. The LBA full cache method based on append-write distributed storage according to claim 3, characterized in that: The structure of the linear list of the region is described as follows: region_x=[lg1,lg2,lg3,……,lgn] Among them, region_x represents any region after hashing, and lg1, lg2, lg3, ..., lgn represent the addressing structure of the logical block.

5. The LBA full cache method based on append-write distributed storage according to claim 2, characterized in that: The multi-level storage structure of the linear table of the region is 2 levels, LBA_table includes directory_table and the lower leaf linear table item_table; Each element in directory_table points to a leaf linear list, and each element contains redundant bit fields to identify the status information of the leaf linear list pointed to; when no element is written to the leaf linear list, the leaf linear list does not allocate memory.

6. The LBA full cache method based on append-write distributed storage according to claim 5, characterized in that: The leaf linear list contains multiple item elements, each of which has a fixed length; item_table contains information for addressing physical chunks; chunk_table contains all chunk_ids referenced by this region; in the item_table, the index idx points to the chunk_id in chunk_table that belongs to the LBA write.

7. An LBA full cache system based on append-write distributed storage, executing an LBA full cache method based on append-write distributed storage as claimed in any one of claims 1 to 6, characterized in that: Deployed in a distributed shared storage cluster.

8. An LBA full cache device based on append-write distributed storage, characterized in that: Carrying the LBA full cache system based on append-write distributed storage as described in claim 7.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the LBA full cache method based on append-write distributed storage described in any one of claims 1-6 are implemented.

10. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the steps of the LBA full cache method based on append-write distributed storage as described in any one of claims 1-6 are implemented.

Images